New Approach for Secant Update Generalized Version of PSB

Abstract

Working with Quasi-Newton methods in optimization leads to one important challenge, being to find an estimate of the Hessian matrix as close as possible to the real matrix. While multisecant methods are regularly used to solve root finding problems, they have been little explored in optimization because the symmetry property of the Hessian matrix estimation is generally not compatible with the multisecant property. In this paper, we propose a solution to apply multisecant methods to optimization problems. Starting from the Powell-Symmetric-Broyden (PSB) update formula and adding pieces of information from the previous steps of the optimization path, we want to develop a new update formula for the estimate of the Hessian. A multisecant version of PSB is, however, generally mathematically impossible to build. For that reason, we provide a formula that satisfies the symmetry and is as close as possible to satisfy the multisecant condition and vice versa for a second formula. Subsequently, we add enforcement of the last secant equation to the symmetric formula and present a comparison between the different methods.