Oblique projections, Broyden restricted class and limited-memory quasi-Newton methods

Abstract

In the paper, a new form of the updating formula of the Broyden restricted class of methods is presented. It assumes the product form similar to that known for long time for the famous Broyden, Fletcher, Goldfarb, Shanno (BFGS) update. It is shown in the paper that similar product representation exists for the Davidon, Fletcher, Powell (DFP) formula and any member of the Broyden restricted class. For the BFGS update the projection uses vector of differences of variables, for DFP image of the previous inverse Hessian approximation on the difference of derivatives and the convex combination of those two vectors when we consider other members of the Broyden class. The formula relating the parameter in the oblique projections with the parameter in the classic form of Broyden restricted class is derived. It permits to use any Broyden update in the limited memory quasi-newton methods where up till now exclusively the BFGS update was exploited. Preliminary computational results of such numerical experiments on some convex functions with increasing dimensions are presented. The presented result gives deeper look inside the structure of variable metric updates. We hope that it would bring in the future simplification of the convergence proofs of the quasi-Newton methods.