Newton-based Methods Research Articles

On the Performance of Tensor Methods for Solving Ill-Conditioned Problems

This paper investigates the performance of tensor methods for solving small- to large-scale systems of nonlinear equations where the Jacobian matrix at the root is ill-conditioned or singular. This condition occurs on many classes of problems, such as identifying or approaching turning points in path-following problems. The singular case has been studied more than the highly ill-conditioned case, for both Newton and tensor methods. It is known that Newton-based methods do not work well with singular problems because they converge linearly to the solution and, in some cases, with poor accuracy. On the other hand, direct tensor methods have performed well on singular problems and have superlinear convergence on such problems under certain conditions. This behavior originates from the use of a special, restricted form of the second-order term included in the local tensor model that provides information lacking in a (nearly) singular Jacobian. With several implementations available for large-scale problems, tensor methods now are capable of solving larger problems. We compare the performance of tensor methods and Newton-based methods for small- to large-scale problems over a range of conditionings, from well-conditioned to ill-conditioned to singular. Previous studies with tensor methods concerned only the ends of this spectrum. Our results show that tensor methods are increasingly superior to Newton-based methods as the problem grows more ill-conditioned.

Comparative studies of optimization methods for the optimal power flow problem

Abstract In this paper we have investigated three optimization approaches for solving the optimal power flow (OPF) problem: active set and penalty (ASP), primal–dual (PD), and primal–dual logarithmic-barrier (PDLB). All three approaches are Newton-based methods. This paper compares the main features of the above and reviews their methodologies. Results obtained in performance tests with traditional networks are summarized.

Tensor methods for large sparse systems of nonlinear equations

This paper introduces tensor methods for solving large sparse systems of nonlinear equations. Tensor methods for nonlinear equations were developed in the context of solving small to medium-sized dense problems. They base each iteration on a quadratic model of the nonlinear equations, where the second-order term is selected so that the model requires no more derivative or function information per iteration than standard linear model-based methods, and hardly more storage or arithmetic operations per iteration. Computational experiments on small to medium-sized problems have shown tensor methods to be considerably more efficient than standard Newton-based methods, with a particularly large advantage on singular problems. This paper considers the extension of this approach to solve large sparse problems. The key issue considered is how to make efficient use of sparsity in forming and solving the tensor model problem at each iteration. Accomplishing this turns out to require an entirely new way of solving the tensor model that successfully exploits the sparsity of the Jacobian, whether the Jacobian is nonsingular or singular. We develop such an approach and, based upon it, an efficient tensor method for solving large sparse systems of nonlinear equations. Test results indicate that this tensor method is significantly more efficient and robust than an efficient sparse Newton-based method, in terms of iterations, function evaluations, and execution time. © 1998 The Mathematical Programming Society, Inc. Published by Elsevier Science B.V.

The Effectiveness of Newton's Method for ImprovingAb InitioForce Fields with Applications to CO2and H2CO

Ab initioforce field parameters are refined using modifications to Newton's method in which a figure of merit function is approximated as a Taylor Series truncated at second order. We investigate two versions of Newton's method: the Discrete Newton method where the Hessian is approximated by finite differences and the Gauss–Newton method where the Hessian is approximated as the product of first derivatives. The applicability of both rests on the capability of currentab initiomethods to calculate quartic force fields that accurately reproduce experimental observables such as band centers and inertial constants. As examples, we calculate refined potential energy surfaces for CO2and H2CO. We show that an algorithm depending solely on Newton's methods requires only a small number of iterations. Additionally, Newton-based methods provide a great deal of information about the sensitivity of the force field parameters to the observables being fit. We also demonstrate that the less computationally demanding Gauss–Newton method gives results similar to the less approximate Discrete Newton method.