Scaled memoryless BFGS preconditioned steepest descent method for very large-scale unconstrained optimization

Abstract

A preconditioned steepest descent (SD) method for solving very large (with dimensions up to 106 ) unconstrained optimization problems is presented. The basic idea is to incorpo1 rate the preconditioning technique in the framework of the SD method. The preconditioner, which is also a scaled memoryless BFGS updating matrix is selected despite the oftenly scaling strategy on SD method. Then the scaled memoryless BFGS preconditioned SD direction can be computed without any additional storage compared with a standard scaled SD direction. In very mild conditions it is shown that, for uniformly convex functions, the method is globally and linearly convergent. Numerical results are also given to illustrate the use of such preconditioning within the SD method. Our numerical study shows that the new proposed preconditioned SD method is significantly outperformed the SD method with Oren-Luenberger scaling and the conjugate gradient method, and comparable to the limited memory BFGS method