quasi-Newton Equation Research Articles

A structured secant method based on a new quasi-Newton equation for nonlinear least squares problems

In this paper, a new quasi-Newton equation is applied to the structured secant methods for nonlinear least squares problems. We show that the new equation is better than the original quasi-Newton equation as it provides a more accurate approximation to the second order information. Furthermore, combining the new quasi-Newton equation with a “product structure”, a new algorithm is established. It is shown that the resulting algorithm is quadratically convergent for the zero-residual case and superlinearly convergent for the nonzero-residual case. In order to compare the new algorithm with some related methods, our preliminary numerical experiments are also reported.

Properties and numerical performance of quasi-Newton methods with modified quasi-Newton equations

Quasi-Newton (QN) equation plays a core role in contemporary nonlinear optimization. The usual QN equation employs only the gradients, but ignores the available function value information. In this paper, we derive a class of modified QN equations with a vector parameter which use both available gradient and function value information. The modified quasi-Newton methods maintain most properties of the usual quasi-Newton methods, meanwhile they achieve a higher-order accuracy in approximating the second-order curvature of the problem functions than the usual ones do. Numerical experiments are reported which support the theoretical analyses and show the advantages of the modified QN methods over the usual ones.

A Survey of Quasi-Newton Equations and Quasi-Newton Methods for Optimization

Quasi-Newton equations play a central role in quasi-Newton methods for optimization and various quasi-Newton equations are available. This paper gives a survey on these quasi-Newton equations and studies properties of quasi-Newton methods with updates satisfying different quasi-Newton equations. These include single-step quasi-Newton equations that use only gradient information and that use both gradient and function value information in one step, and multi-step quasi-Newton equations that use the gradient information in last m steps. Main properties of quasi-Newton methods with updates satisfying different quasi-Newton equations are studied. These properties include the finite termination property, invariance, heredity of positive definite updates, consistency of search directions, global convergence and local superlinear convergence properties.

New Quasi-Newton Equation and Related Methods for Unconstrained Optimization

In unconstrained optimization, the usual quasi-Newton equation is B k+1 s k=y k, where y k is the difference of the gradients at the last two iterates. In this paper, we propose a new quasi-Newton equation, $$B_{k + 1} s_k = \tilde y_k $$ , in which $$\tilde y_k $$ is based on both the function values and gradients at the last two iterates. The new equation is superior to the old equation in the sense that $$\tilde y_k $$ better approximates ∇ 2 f(x k+1)s k than y k. Modified quasi-Newton methods based on the new quasi-Newton equation are locally and superlinearly convergent. Extensive numerical experiments have been conducted which show that the new quasi-Newton methods are encouraging.

Family of optimally conditioned quasi-Newton updates for unconstrained optimization

In this note, a general optimal conditioning problem for updates which satisfy the quasi-Newton equation is solved. The new solution is a family of updates which contains other known optimally conditioned updates but also includes new formulas of increased rank. A new factorization formula for the Broyden family and some preliminary numerical results are also given.

A Modified Broyden Update with Interpolation

A new class of Hessian updates is introduced for quasi-Newton methods obtained by modifying the Broyden class of updates so that it interpolates function values. In general, the members of this new class do not satisfy the quasi-Newton equation. Through the use of a weak inverse updating strategy, a specific choice of update within this class is then introduced. Numerical results are given to illustrate the behaviour of this new update and to compare it with other updating sequences.

Explicit general solution of the Quasi-Newton equation with sparsity and symmetry

We consider the problem of solving the Quasi-Newton equation under the additional condition that some elements of the solution take prescribed values. The general solution is obtained in a straightforward way by using the Huang algorithm of the ABS class. It is given in explicit form without the need of solving intermediate linear systems. Particular cases include the general symmetric or unsymmetric full or sparse update. In the sparse symmetric case we show that while positive definiteness cannot generally be forced a weaker condition called quasi positive definiteness can be satisfied under a very mild condition. This condition guarantees that all eigenvalues, except at most two, are positive.

Partitioned quasi-Newton methods for nonlinear equality constrained optimization

We derive new quasi-Newton updates for the (nonlinear) equality constrained minimization problem. The new updates satisfy a quasi-Newton equation, maintain positive definiteness on the null space of the active constraint matrix, and satisfy a minimum change condition. The application of the updates is not restricted to a small neighbourhood of the solution. In addition to derivation and motivational remarks, we discuss various numerical subtleties and provide results of numerical experiments.

Finding general solutions of the Quasi-Newton equation via the ABS approach

In this paper we use the ABS algorithm to derive general and special solutions of single or multiple secant equations in Quasi-Newton methods for nonlinear equations or nonlinear optimization. The case where sparsity conditions are present is also considered.

A Modified BFGS Algorithm for Unconstrained Optimization

Computing Centre, Academia Sinica, Beijing 100080, China[Received 25 April 1989 and in revised form 16 October 1990]In this paper we present a modified BFGS algorithm for unconstrainedoptimization. The BFGS algorithm updates an approximate Hessian whichsatisfies the most recent quasi-Newton equation. The quasi-Newton condition canbe interpreted as the interpolation condition that the gradient value of the localquadratic model matches that of the objective function at the previous iterate.Our modified algorithm requires that the function value is matched, instead of thegradient value, at the previous iterate. The modified algorithm preserves theglobal and local superlinear convergence properties of the BFGS algorithm.Numerical results are presented, which suggest that a slight improvement hasbeen achieved.1. Introduction

Finite-Element Simulation of An In-Package Pasteurization Process

Abstract The problem considered in this paper is the simulation of the in-package pasteurization process for fluids contained in bottles or cans. In a typical pasteurization operation, the product enters the pasteurizer at a temperature of about 36°F and passes through several progressively hotter zones, which raise its temperature to approximately 140°F. This temperature of 140°F is maintained in the holding zone. The product then passes through several progressively cooler zones, which lower the temperature to 70-80°F This process was simulated by using a penalty Galerkin finite-element procedure. A variable step implicit time integration technique coupled with a quasi-Newton nonlinear equation solver was employed. The numerical results are in excellent agreement with experimental results.

A unified derivation of quasi-Newton methods for solving non-sparse and sparse nonlinear equations

An augmented quasi-Newton equation is solved by using theW-V generalized inverse to give a unified derivation of the known quasi-Newton methods for solving systems of nonlinear algebraic equations. This approach makes it possible to get new formulas for sparse and non-sparse systems, and also to determine what norms of the update matrices are minimized when several useful quasi-Newton update formulas are derived.

Minimum Norm Symmetric Quasi-Newton Updates Restricted to Subspaces

The Davidon-Fletcher-Powell and Broyden-Fletcher-Goldfarb-Shanno updates have been the two most successful quasi-Newton updates for a variety of applications. One reason offered in explanation is that they constitute, in an appropriate norm and metric, the minimum norm change to the matrix, or its inverse, being approximated which preserves symmetry and obeys the quasi-Newton equation. Recent methods have reason to consider updates restricted to certain subspaces. In this paper we derive the general minimum norm symmetric quasi-Newton updates restricted to such subspaces. In the same appropriate norm and metric, the minimum norm change update to the matrix or its inverse is shown to be, respectively, the rank-two update which is a particular projection of the DFP or BFGS onto this subspace.

Minimum norm symmetric quasi-Newton updates restricted to subspaces

The Davidon-Fletcher-Powell and Broyden-Fletcher-Goldfarb-Shanno updates have been the two most successful quasi-Newton updates for a variety of applications. One reason offered in explanation is that they constitute, in an appropriate norm and metric, the minimum norm change to the matrix, or its inverse, being approximated which preserves symmetry and obeys the quasi-Newton equation. Recent methods have reason to consider updates restricted to certain subspaces. In this paper we derive the general minimum norm symmetric quasi-Newton updates restricted to such subspaces. In the same appropriate norm and metric, the minimum norm change update to the matrix or its inverse is shown to be, respectively, the rank-two update which is a particular projection of the DFP or BFGS onto this subspace.