A Modified Algorithm of Steepest Descent Method for Solving Unconstrained Nonlinear Optimization Problems

Abstract

The steepest descent method (SDM), which can be traced back to Cauchy (1847), is the simplest gradient method for unconstrained optimization problem. The SDM is effective for well-posed and low-dimensional nonlinear optimization problems without constraints; however, for a large-dimensional system, it converges very slowly. Therefore, a modified steepest decent method (MSDM) is developed to deal with these problems. Under the MSDM framework, the original global minimization problem is transformed into a quadratic-form minimization based on the SDM and the current iterative point. Our starting point is a manifold defined in terms of the quadratic function and a fictitious time variable. Thereafter, we can derive an iterative algorithm by including a parameter in the final stage. Through a Hopf bifurcation, this parameter indeed plays a major role to switch the situation of slow convergence to a new situation that the new algorithm converges faster. Several numerical examples are examined and compared with exact solutions. It is found that the new algorithm of the MSDM has better computational efficiency and accuracy, even for a large-dimensional non-convex minimization problem of the generalized Rosenbrock function.