Convergence Of quasi-Newton Methods Research Articles

Quasi-Newton-type optimized iterative learning control for discrete linear time invariant systems

In this paper, a quasi-Newton-type optimized iterative learning control (ILC) algorithm is investigated for a class of discrete linear time-invariant systems. The proposed learning algorithm is to update the learning gain matrix by a quasi-Newton-type matrix instead of the inversion of the plant. By means of the mathematical inductive method, the monotone convergence of the proposed algorithm is analyzed, which shows that the tracking error monotonously converges to zero after a finite number of iterations. Compared with the existing optimized ILC algorithms, due to the superlinear convergence of quasi-Newton method, the proposed learning law operates with a faster convergent rate and is robust to the ill-condition of the system model, and thus owns a wide range of applications. Numerical simulations demonstrate the validity and effectiveness.

Damped techniques for enforcing convergence of quasi-Newton methods

This paper extends the technique used in the damped BFGS method of Powell [Algorithms for nonlinear constraints that use Lagrange functions, Math. Program. 14 (1978), 224–248] to the Broyden family of quasi-Newton methods with applications to unconstrained optimization problems. Appropriate conditions on the damped technique are proposed to enforce safely the positive definiteness property for all Broyden's updates. It is shown that this technique maintains the q-superlinear convergence property of the restricted Broyden family of methods for uniformly convex functions. It also extends the global convergence property to all members of the family. Preliminary numerical results are described which show that appropriate ways for employing the proposed technique improve the performance of all members of the Broyden family of methods substantially and significantly in certain cases. They also enforce convergence of divergent quasi-Newton methods.

A new class of quasi-Newton updating formulas

In this paper, we propose a derivative-free quasi-Newton condition, which results in a new class of quasi-Newton updating formulas for unconstrained optimization. Each updating formula in this class is a rank-two updating formula and preserves the positive definiteness of the second derivative matrix of the quadratic model. Its first two terms are the same as the first two terms of the BFGS updating formula. We establish global convergence of quasi-Newton methods based upon the updating formulas in this class, and superlinear convergence of a special quasi-Newton method among them. Then we propose a special quasi-Newton updating formula, which repetitively uses the new quasi-Newton condition. This updating formula is derivative-free. Numerical results are reported. Dedicated to Professor M.J.D. Powell on the occasion of his 70th birthday

Convergence of quasi-Newton method with new inexact line search

Quasi-Newton method is a well-known effective method for solving optimization problems. Since it is a line search method, which needs a line search procedure after determining a search direction at each iteration, we must decide a line search rule to choose a step size along a search direction. In this paper, we propose a new inexact line search rule for quasi-Newton method and establish some global convergent results of this method. These results are useful in designing new quasi-Newton methods. Moreover, we analyze the convergence rate of quasi-Newton method with the new line search rule.

On the convergence of quasi-newton methods for nonsmooth problems

We develop a theory of quasi-New ton and least-change update methods for solving systems of nonlinear equations F(x) = 0. In this theory, no differentiability conditions are necessary. Instead, we assume that Fcan be approximated, in a weak sense, by an affine function in a neighborhood of a solution. Using this assumption, we prove local and ideal convergence. Our theory can be applied to B-differentiable functions and to partially differentiable functions.

Local and Superlinear Convergence for Partially Known Quasi-Newton Methods

This paper develops a unified theory for establishing the local and q-superlinear convergence of quasi-Newton methods from the convex class when part of the Hessian matrix is known. One first proves the bounded deterioration principle due to Dennis (and popularized by Broyden, Dennis, and More) for the appropriate modifications of all update formulas in the convex Broyden class. Using standard conditions on the quasi-Newton updates, one then deduces local and q-superlinear convergence. Particular cases of these methods are the SQP augmented scale BFGS and DFP secant methods for constrained optimization problems introduced by Tapia and a generalization of the Al-Baali and Fletcher modification of the structured secant method considered by Dennis, Gay, and Welsch for the nonlinear least-squares problem. In all cases, bounded deterioration is proved for the approximate Hessian, not for its inverse.