A class of diagonal preconditioners for limited memory BFGS method

Abstract

A major weakness of the limited memory BFGS (LBFGS) method is that it may converge very slowly on ill-conditioned problems when the identity matrix is used for initialization. Very often, the LBFGS method can adopt a preconditioner on the identity matrix to speed up the convergence. For this purpose, we propose a class of diagonal preconditioners to boost the performance of the LBFGS method. In this context, we find that it is appropriate to use a diagonal preconditioner, in the form of a diagonal matrix plus a positive multiple of the identity matrix, so as to fit information of local Hessian as well as to induce positive definiteness for the diagonal preconditioner at a whole. The property of hereditary positive definiteness is maintained by a careful choice of the positive scalar on the scaled identity matrix while the local curvature information is carried implicitly on the other diagonal matrix through the variational techniques, commonly employed in the derivation of quasi-Newton updates. Several preconditioning formulae are then derived and tested on a large set of standard test problems to access the impact of different choices of such preconditioners on the minimization performance.