Newton-type Method Research Articles

Direct and Converse Theorems for Iterative Methods of Solving Irregular Operator Equations and Finite Difference Methods for Solving Ill-Posed Cauchy Problems

Results obtained in recent years concerning necessary and sufficient conditions for the convergence (at a given rate) of approximation methods for solutions of irregular operator equations are overviewed. The exposition is given in the context of classical direct and converse theorems of approximation theory. Due to the proximity of the resulting necessary and sufficient conditions to each other, the solutions on which a certain convergence rate of the methods is reached can be characterized nearly completely. The problems under consideration include irregular linear and nonlinear operator equations and ill-posed Cauchy problems for first- and second-order differential operator equations. Procedures for stable approximation of solutions of general irregular linear equations, classes of finite-difference regularization methods and the quasi-reversibility method for ill-posed Cauchy problems, and the class of iteratively regularized Gauss–Newton type methods for irregular nonlinear operator equations are examined.

Critical Multipliers in Semidefinite Programming

It was proved in Izmailov and Solodov (2014). Newton-Type Methods for Optimization and Variational Problems, Springer] that the existence of a noncritical multiplier for a (smooth) nonlinear programming problem is equivalent to an error bound condition for the Karush–Kuhn–Thcker (KKT) system without any assumptions. This paper investigates whether this result still holds true for a (smooth) nonlinear semidefinite programming (SDP) problem. The answer is negative: the existence of noncritical multiplier does not imply the error bound condition for the KKT system without additional conditions, which is illustrated by an example. In this paper, we obtain characterizations, in terms of the problem data, the critical and noncritical multipliers for a SDP problem. We prove that, for the SDP problem, the noncriticality property can be derived from the error bound condition for the KKT system without any assumptions, and we give an example to show that the noncriticality does not imply the error bound for the KKT system. We propose a set of assumptions under which the error bound condition for the KKT system can be derived from the noncriticality property. a Finally, we establish a new error bound for [Formula: see text]-part, which is expressed by both perturbation and the multiplier estimation.

Multipoint Fractional Iterative Methods with (2α + 1)th-Order of Convergence for Solving Nonlinear Problems

In the recent literature, some fractional one-point Newton-type methods have been proposed in order to find roots of nonlinear equations using fractional derivatives. In this paper, we introduce a new fractional Newton-type method with order of convergence α + 1 and compare it with the existing fractional Newton method with order 2 α . Moreover, we also introduce a multipoint fractional Traub-type method with order 2 α + 1 and compare its performance with that of its first step. Some numerical tests and analysis of the dependence on the initial estimations are made for each case, including a comparison with classical Newton ( α = 1 of the first step of the class) and classical Traub’s scheme ( α = 1 of fractional proposed multipoint method). In this comparison, some cases are found where classical Newton and Traub’s methods do not converge and the proposed methods do, among other advantages.

FBstab: A proximally stabilized semismooth algorithm for convex quadratic programming

This paper introduces the proximally stabilized Fischer–Burmeister method (FBstab); a new algorithm for convex quadratic programming that synergistically combines the proximal point algorithm with a primal–dual semismooth Newton-type method. FBstab is numerically robust, easy to warmstart, handles degenerate primal–dual solutions, detects infeasibility/unboundedness and requires only that the Hessian matrix be positive semidefinite. We outline the algorithm, provide convergence and convergence rate proofs, and report some numerical results from model predictive control benchmarks and from the Maros–Meszaros test set. We show that FBstab is competitive with state of the art methods and is especially promising for model predictive control and other parameterized problems.

Relaxed Gauss--Newton Methods with Applications to Electrical Impedance Tomography

As second-order methods, Gauss--Newton-type methods can be more effective than first-order methods for the solution of nonsmooth optimization problems with expensive-to-evaluate smooth components. Such methods, however, often do not converge. Motivated by nonlinear inverse problems with nonsmooth regularization, we propose a new Gauss--Newton-type method with inexact relaxed steps. We prove that the method converges to a set of disjoint critical points given that the linearisation of the forward operator for the inverse problem is sufficiently precise. We extensively evaluate the performance of the method on electrical impedance tomography (EIT).

Least Conservative Linearized Constraint Formulation for Real-Time Motion Generation

Abstract Today robotics has shown many successful strategies to solve several navigation problems. However, moving into a dynamic environment is still a challenging task. This paper presents a novel method for motion generation in dynamic environments based on real-time nonlinear model predictive control (NMPC). At the core of our approach is a least conservative linearized constraint formulation built upon the real-time iteration (RTI) scheme with Gauss-Newton Hessian approximation. We demonstrate that the proposed constraint formulation is less conservative for planners based on Newton-type method than for those based on a fully converged NMPC method. Additionally, we show the performance of our approach in simulation, in a scenario where the Crazyflie nanoquadcopter avoids balls and reaches its desired goal in spite of the uncertainty about when the balls will be thrown. The numerical results validate our theoretical findings and illustrate the computational efficiency of the proposed scheme.

A Data-Driven Distributed Adaptive Control Approach for Nonlinear Multi-Agent Systems

In this paper the distributed leader-follower consensus tracking problem is investigated for unknown nonlinear non-affine discrete-time multi-agent systems. Via a dynamic linearization method both for the agent system and the local ideal distributed controller, a distributed adaptive control scheme is proposed in this paper using the Newton-type optimization method. The proposed approach is data-driven since only the local measurement information among neighboring agents is utilized in the control system design. The consensus tracking stabilities of the proposed approach are rigorously guaranteed in the cases of fixed and switching communication topologies. The simulations are conducted to verify the effectiveness of the proposed approach.

Collision Avoidance for Multiple Micro Aerial Vehicles using Fast Centralized Nonlinear Model Predictive Control

Abstract This article proposes a novel control architecture using a centralized nonlinear model predictive control (CNMPC) scheme for controlling multiple micro aerial vehicles (MAVs). The control architecture uses an augmented state system to control multiple agents and performs both obstacle and collision avoidance. The optimization algorithm used is OpEn, based on the proximal averaged Newton type method for optimal control (PANOC) which provides fast convergence for non-convex optimization problems. The objective is to perform position reference tracking for each individual agent, while nonlinear constraints guarantee collision avoidance and smooth control signals. To produce a trajectory that satisfies all constraints a penalty method is applied to the nonlinear constraints. The efficacy of this proposed novel control scheme is successfully demonstrated through simulation results and comparisons, in terms of computation time and constraint violations, which are provided with respect to the number of agents.

OpEn: Code Generation for Embedded Nonconvex Optimization

We present Optimization Engine (OpEn): an open-source code generation framework for real-time embedded nonconvex optimization, which implements a novel numerical method. OpEn combines the proximal averaged Newton-type method for optimal control (PANOC) with the penalty and augmented Lagrangian methods to compute approximate stationary points of nonconvex problems. The proposed method involves very simple algebraic operations such as vector products, has a low memory footprint and exhibits very good convergence properties that allow the solution of nonconvex problems on embedded devices. OpEn’s core solver is written is Rust — a modern, high-performance, memory-safe and thread-safe systems programming language — while users can call it from Python, MATLAB, C, C++, ROS or over a TCP socket.

Basins of attraction and critical curves for Newton-type methods in a phase equilibrium problem

Many engineering problems are described by systems of nonlinear equations, which may exhibit multiple solutions, in a challenging situation for root-finding algorithms. The existence of several solutions may give rise to complex basins of attraction for the solutions in the algorithms, with severe influence in their convergence behavior. In this work, we explore the relationship of the basins of attractions with the critical curves (the locus of the singular points of the Jacobian of the system of equations) in a phase equilibrium problem in the plane with two solutions, namely the calculation of a double azeotrope in a binary mixture. The results indicate that the conjoint use of the basins of attraction and critical curves can be a useful tool to select the most suitable algorithm for a specific problem.

Basins of attraction and critical curves for Newton-type methods in a phase equilibrium problem

Many engineering problems are described by systems of nonlinear equations, which may exhibit multiple solutions, in a challenging situation for root-finding algorithms. The existence of several solutions may give rise to complex basins of attraction for the solutions in the algorithms, with severe influence on their convergence behaviour. In this work, we explore the relationship of the basins of attractions with the critical curves (the locus of the singular points of the Jacobian of the system of equations) in a phase equilibrium problem in the plane with two solutions, namely the calculation of a double azeotrope in a binary mixture. The results indicate that the conjoint use of the basins of attraction and critical curves can be a useful tool to select the most suitable algorithm for a specific problem.

Vision-driven NMPC for Autonomous Aerial Navigation in Subterranean Environments

Abstract This work establishes a novel robocentric Non-linear Model Predictive Control (NMPC) framework for fast fully autonomous navigation of quadrotors in featureless dark tunnel environments. Additionally, this work leverages the processing of a single camera to generate direction commands along the tunnel axis, while regulating the platform’s altitude. The extracted visual dynamics are coupled in the sequel with the NMPC problem, structured around the Proximal Averaged Newton-type method for Optimal Control (PANOC), which is a fast numerical optimization method that is not sensitive to ill conditioning and is suitable for embedded NMPC implementations. Multiple fully realistic simulation results demonstrate the effectiveness of the proposed method in challenging environments.

Two Iterative Methods to Solve Nonlinear Equations of Load Flows

This paper presents the results obtained when two iterative methods are applied to the solution of non-linear equations that model the load flow in electric power systems. Two iterative methods are applied; the first consists of a simplification of the rectangular form the traditional Newton-Raphson method, the second is a hybrid method and relates the simplified form proposed here and a four-step Newton-type iterative method. The convergence characteristic and the mathematical preliminaries of the iterative four-step method are included in the paper. The methods were used to calculate the voltages at each node of the IEEE test system of 118 nodes and a distribution system of 40 nodes. In each method, the formation of the Jacobian matrix, widely used in traditional forms of load flows, is avoided and only elementary operations are carried out, impacting the execution times for the test systems used, being of the order of 15.6 to 279 milliseconds. The maximum error found is for the 118 node system and is of the order of 3.7%.

Newton-type multilevel optimization method

Inspired by multigrid methods for linear systems of equations, multilevel optimization methods have been proposed to solve structured optimization problems. Multilevel methods make more assumptions regarding the structure of the optimization model, and as a result, they outperform single-level methods, especially for large-scale models. The impressive performance of multilevel optimization methods is an empirical observation, and no theoretical explanation has so far been proposed. In order to address this issue, we study the convergence properties of a multilevel method that is motivated by second-order methods. We take the first step toward establishing how the structure of an optimization problem is related to the convergence rate of multilevel algorithms.

Newton-Type Method for Solving Systems of Linear Equations and Inequalities

A Newton-type method is proposed for numerical minimization of convex piecewise quadratic functions, and its convergence is analyzed. Previously, a similar method was successfully applied to optimization problems arising in mesh generation. It is shown that the method is applicable to computing the projection of a given point onto the set of nonnegative solutions of a system of linear equations and to determining the distance between two convex polyhedra. The performance of the method is tested on a set of problems from the NETLIB repository.

Metrically regular mappings and its application to convergence analysis of a confined Newton-type method for nonsmooth generalized equations

Notion of metrically regular property and certain types of point-based approximations are used for solving the nonsmooth generalized equation f (x) + ℱ(x) ∋ 0, where X and Y are Banach spaces, and U is an open subset of X, f : U → Y is a nonsmooth function and ℱ : X ⇉ Y is a set-valued mapping with closed graph. We introduce a confined Newton-type method for solving the above nonsmooth generalized equation and analyze the semilocal and local convergence of this method. Specifically, under the point-based approximation of f on U and metrically regular property of f + ℱ, we present quadratic rate of convergence of this method. Furthermore, superlinear rate of convergence of this method is provided under the conditions that f admits p-point-based approximation on U and f + ℱ is metrically regular. An example of nonsmooth functions that have p-point-based approximation is given. Moreover, a numerical experiment is given which illustrates the theoretical result.

A two-step inexact Newton-Chebyshev-like method for inverse eigenvalue problems

A two-step inexact Newton-Chebyshev-like method is proposed to solve inverse eigenvalue problems. Unlike general Newton type methods which need to invert the Jacobian matrix that may lead to an unexpected instability, the proposed method can guarantee the numerical stability by exploiting the Chebyshev method to approximate the inverse of the Jacobian matrix. Theoretically, we prove that the proposed method converges cubically. Numerical examples are given to demonstrate the effectiveness of our method.

Tailored interior and boundary parameter transformations for iterative inversion in electrical impedance tomography

Electrical impedance tomography is a non-invasive method for imaging the electrical conductivity of an object from electrode measurements on its surface. The underlying mathematical problem is highly nonlinear, severely ill-posed, and several model parameters are usually not known accurately. Despite the strong nonlinearity, iterative Newton-type methods are widely used to tackle the problem numerically.This work presents and analyzes tailored transformations for the conductivity and for electrode parameters which are favourable in two regards: they remove the constrainedness of the unknown parameters and simultaneously decrease the nonlinearity of the underlying problem. We study the impact of various transformations on the nonlinearity of the problem and demonstrate improved speed of convergence for Newton-type methods while avoiding local minima in the solution space. The presented transformations can conveniently be incorporated into existing iterative solvers as they improve stability and do not require hand-tuned regularization parameters or line-search strategies, thereby bridging a gap between a variety of established conductivity estimation methods and practical applications.

Koiter’s Reduction Finite Element Method for Nonlinear Stability Analysis of Thin-Walled Shells

Thin-walled structures are widely used in aeronautical and aerospace engineering. Conical and cylindrical shells structures, under axial compression, are prone to failure by buckling and typically show a snap-back phenomenon in the end-shortening curve. Path-following technologies based on Newton-type methods have difficulties to trace reliably the snap-back response due to the extremely sharp turning angle near the limit point. In this paper, a Koiter’s reduction finite element method, termed the Koiter–Newton (KN) method, is presented to trace reliably the post-buckling path of cylinders and cones considering either linear buckling modes or dimples from lateral perturbation loads as geometric imperfection. A robust algorithm based on the bifurcation-detection technique is applied during the solution of the reduced order model to achieve a successful path-tracing. The numerical results presented reveal that the nonlinear prediction obtained from Koiter’s perturbation theory at the unloaded state of the structure is numerically accurate up to the buckling load and the initial post-buckling path.

Software system design based on patterns for Newton-type methods

A wide range of engineering applications uses optimisation techniques as part of their solution process. The researcher uses specialized software that implements well-known optimisation techniques to solve his problem. However, when it comes to develop original optimisation techniques that fit a particular problem the researcher has no option but to implement his own new method from scratch. This leads to large development times and error prone code that, in general, will not be reused for any other application. In this work, we present a novel methodology that simplifies, fasten and improves the development process of scientific software. This methodology guide us on the identification of design patterns. The application of this methodology generates reusable, flexible and high quality scientific software. Furthermore, the produced software becomes a documented tool to transfer the knowledge on the development process of scientific software. We apply this methodology for the design of an optimisation framework implementing Newton's type methods which can be used as a fast prototyping tool of new optimisation techniques based on Newton's type methods. The abstraction, reusability and flexibility of the developed framework is measured by means of Martin's metric. The results indicate that the developed software is highly reusable.