Neural Dynamics and Newton–Raphson Iteration for Nonlinear Optimization

Abstract

In this paper, a special type of neural dynamics (ND) is generalized and investigated for timevarying and static scalarvalued nonlinear optimization. In addition, for comparative purpose, the gradientbased neural dynamics (or termed gradient dynamics (GD)) is studied for nonlinear optimization. Moreover, for possible digital hardware realization, discretetime ND (DTND) models are developed. With the linear activation function used and with the step size being 1, the DTND model reduces to Newton–Raphson iteration (NRI) for solving the static nonlinear optimization problems. That is, the wellknown NRI method can be viewed as a special case of the DTND model. Besides, the geometric representation of the ND models is given for timevarying nonlinear optimization. Numerical results demonstrate the efficacy and advantages of the proposed ND models for timevarying and static nonlinear optimization.