Quasi-Newton methods based on ordinary differential equation approach for unconstrained nonlinear optimization

Abstract

In this paper, we propose a hybrid ODE-based quasi-Newton (QN) method for unconstrained optimization problems, which combines the idea of low-order implicit Runge–Kutta (RK) techniques for gradient systems with the QN type updates of the Jacobian matrix such as the symmetric rank-one (SR1) update. The main idea of this approach is to associate a QN matrix to approximate numerically the Jacobian matrix in the gradient system. Fundamentally this is a gradient system based on the first order optimality conditions of the optimization problem. To further extend the methods for solving large-scale problems, a feature incorporated to the proposed methods is that a limited memory setting for matrix–vector multiplications is used, thus avoiding the computational and storage issues when computing Jacobian information. Under suitable assumptions, global convergence of the proposed method is proved. Practical insights on the effectiveness of these approaches on a set of test functions are given by a numerical comparison with that of the limited memory BFGS algorithm (L-BFGS) and conjugate gradient algorithm (CG).