Practical quasi-Newton methods for solving nonlinear systems

Abstract

Practical quasi-Newton methods for solving nonlinear systems are surveyed. The definition of quasi-Newton methods that includes Newton's method as a particular case is adopted. However, especial emphasis is given to the methods that satisfy the secant equation at every iteration, which are called here, as usually, secant methods. The least-change secant update (LCSU) theory is revisited and convergence results of methods that do not belong to the LCSU family are discussed. The family of methods reviewed in this survey includes Broyden's methods, structured quasi-Newton methods, methods with direct updates of factorizations, row-scaling methods and column-updating methods. Some implementation features are commented. The survey includes a discussion on global convergence tools and linear-system implementations of Broyden's methods. In the final section, practical and theoretical perspectives of this area are discussed.