Newton Step Research Articles

A low‐rank method for parameter‐dependent fluid‐structure interaction discretizations with hyperelasticity

AbstractFluid‐structure interaction models are used to study how a material interacts with different fluids at different Reynolds numbers. Examining the same model not only for different fluids but also for different solids allows to optimize the choice of materials for construction even better. A possible answer to this demand is parameter‐dependent discretization. Furthermore, low‐rank techniques can reduce the complexity needed to compute approximations to parameter‐dependent fluid‐structure interaction discretizations. Low‐rank methods have been applied to parameter‐dependent linear fluid‐structure interaction discretizations. The linearity of the operators involved allows to translate the resulting equations to a single matrix equation. The solution is approximated by a low‐rank method. In this paper, we propose a new method that extends this framework to nonlinear parameter‐dependent fluid‐structure interaction problems by means of the Newton iteration. The parameter set is split into disjoint subsets. On each subset, the Newton approximation of the problem related to the median parameter is computed and serves as initial guess for one Newton step on the whole subset. This Newton step yields a matrix equation whose solution can be approximated by a low‐rank method. The resulting method requires a smaller number of Newton steps if compared with a direct approach that applies the Newton iteration to the separate problems consecutively. In the experiments considered, the proposed method allows to compute a low‐rank approximation up to twenty times faster than by the direct approach.

Yet another fast variant of Newton’s method for nonconvex optimization

Abstract A class of second-order algorithms is proposed for minimizing smooth nonconvex functions that alternates between regularized Newton and negative curvature steps in an iteration-dependent subspace. In most cases, the Hessian matrix is regularized with the square root of the current gradient and an additional term taking moderate negative curvature into account, a negative curvature step being taken only exceptionally. Practical variants are detailed where the subspaces are chosen to be the full space, or Krylov subspaces. In the first case, the proposed method only requires the solution of a single linear system at nearly all iterations. We establish that at most $\mathcal{O}\big ( |\!\log \epsilon |\,\epsilon ^{-3/2}\big )$ evaluations of the problem’s objective function and derivatives are needed for algorithms in the new class to obtain an $\epsilon $-approximate first-order minimizer, and at most $\mathcal{O}\big (|\!\log \epsilon |\,\epsilon ^{-3}\big )$ to obtain a second-order one. Encouraging initial numerical experiments with two full-space and two Krylov-subspaces variants are finally presented.

A Line-Search-Based Algorithm for Multiphase Flash Calculations with CO2-Hydrocarbon System.

Multiphase flash calculations are pivotal in compositional simulation, necessitating a robust and efficient computational algorithm. In this work, we have developed a line-search-based algorithm framework for stability analysis and multiphase flash calculations. This algorithm is rooted in the modified Newton step and line-search method. The modified Newton step, derived from modified Cholesky factorization, ensures a descent direction, while the line search determines the degree of decrease. This combination facilitates convergence even in challenging regions for phase stability analysis and multiphase flash calculations, exhibiting superlinear convergence speed. Unlike traditional approaches that rely on successive substitution iteration and may resort to Newton iteration only if the Hessian matrix is positive definite, our algorithm incorporates modification via modified Cholesky factorization upon encountering a nonpositive definite Hessian matrix. We tested our algorithm with several classical fluids, demonstrating its efficiency and robustness. Furthermore, we assessed the algorithm's performance by computing the pressure-composition diagram for the CO2-hydrocarbon system, where all calculations achieved rapid convergence without failure. This newly developed algorithm for phase stability analysis and multiphase flash calculations represents a significant advancement for compositional or chemical process simulations.

A Jacobian-free pseudo-arclength continuation method for phase transitions in inhomogeneous thermodynamic systems.

Developing phase diagrams for inhomogeneous systems in thermodynamics is difficult, in part, due to the large phase space and the possibility of unstable and metastable solutions arising from first-order phase transitions. Pseudo-arclength continuation (PAC) is a method that allows one to trace out stable and unstable solutions of nonlinear systems. Typically, PAC utilizes the Jacobian in order to implement Newton (or quasi-Newton) steps. In this work, we present a Jacobian-free PAC method that is amenable to the usual workflows in inhomogeneous thermodynamics. We demonstrate our method in systems that have first-order phase transitions, including a novel example of polyelectrolyte complex coacervation in confinement, where multiple surface phase transitions occur and can overlap with one another.

Achieving Optimal Order in a Novel Family of Numerical Methods: Insights from Convergence and Dynamical Analysis Results

In this manuscript, we introduce a novel parametric family of multistep iterative methods designed to solve nonlinear equations. This family is derived from a damped Newton’s scheme but includes an additional Newton step with a weight function and a “frozen” derivative, that is, the same derivative than in the previous step. Initially, we develop a quad-parametric class with a first-order convergence rate. Subsequently, by restricting one of its parameters, we accelerate the convergence to achieve a third-order uni-parametric family. We thoroughly investigate the convergence properties of this final class of iterative methods, assess its stability through dynamical tools, and evaluate its performance on a set of test problems. We conclude that there exists one optimal fourth-order member of this class, in the sense of Kung–Traub’s conjecture. Our analysis includes stability surfaces and dynamical planes, revealing the intricate nature of this family. Notably, our exploration of stability surfaces enables the identification of specific family members suitable for scalar functions with a challenging convergence behavior, as they may exhibit periodical orbits and fixed points with attracting behavior in their corresponding dynamical planes. Furthermore, our dynamical study finds members of the family of iterative methods with exceptional stability. This property allows us to converge to the solution of practical problem-solving applications even from initial estimations very far from the solution. We confirm our findings with various numerical tests, demonstrating the efficiency and reliability of the presented family of iterative methods.

Regularized online exponentially concave optimization

In this paper, we investigate regularized online exponentially concave (abbr. exp-concave) optimization, in which each loss function consists of a time-varying exp-concave function and a fixed convex regularization. If the whole loss function is exp-concave, a classical method called online Newton step (ONS) enjoys an O(dlogT) regret bound, where d is the dimensionality and T is the time horizon. However, in the regularized setting, the sum of an exp-concave function and a convex regularization is not necessarily an exp-concave function, which implies that ONS is not applicable. To address this problem, we propose the proximal online Newton step (ProxONS), and show that it can attain the same O(dlogT) regret bound for any convex regularization. The main idea is to first perform an iteration of ONS with the exp-concave part in each loss function and then perform a proximal mapping with the regularization part. Furthermore, we demonstrate that by utilizing the standard online-to-batch conversion, our ProxONS can be extended to solve stochastic optimization with a regularized exp-concave objective, and enjoy an O(dlogT/T) convergence rate with high probability. Experimental results on two real datasets verify the effectiveness of our ProxONS.

Qubit Count Reduction by Orthogonally Constrained Orbital Optimization for Variational Quantum Excited-State Solvers.

We propose a state-averaged orbital optimization scheme for improving the accuracy of excited states of the electronic structure Hamiltonian for use on near-term quantum computers. Instead of parameterizing the orbital rotation operator in the conventional fashion as an exponential of an antihermitian matrix, we parameterize the orbital rotation as a general partial unitary matrix. Whereas conventional orbital optimization methods minimize the state-averaged energy using successive Newton steps of the second-order Taylor expansion of the energy, the method presented here optimizes the state-averaged energy using an orthogonally constrained gradient projection method that does not require any expansion approximations. Through extensive benchmarking of the method on various small molecular systems, we find that the method is capable of producing more accurate results than fixed basis FCI while simultaneously using fewer qubits. In particular, we show that for H2, the method is capable of matching the accuracy of FCI in the cc-pVTZ basis (56 qubits) while only using 14 qubits.

Solving quantum optimal control problems using projection-operator-based Newton steps

Solving quantum optimal control problems using projection-operator-based Newton steps

Distributed Newton Step Projection Algorithm for Online Convex Optimization Over Time-Varying Unbalanced Networks

Distributed Newton Step Projection Algorithm for Online Convex Optimization Over Time-Varying Unbalanced Networks

A Special Interval Newton Step for Solving Nonlinear Equations

The problem of finding the zeros of a nonlinear equation has been extensively and thoroughly studied. Point methods constitute a significant and extensive category of techniques, allowing for the efficient finding of zeros with arbitrary precision under specific conditions. However, the limitation of these methods is that they typically yield a single zero. An alternative approach employs Interval Analysis, leveraging its properties to provide reliable and with certainty inclusions of all zeros within a given search interval. Interval methods, such as the Interval Newton method, exhibit quadratic convergence to the corresponding inclusions when monotonicity and simple zeros exist. Nonetheless, there exist pathological cases, like the existence of multiple zeros, where the obtained inclusions cannot be bounded with arbitrary precision, necessitating the adoption of bisection schemes to refine the search interval. These schemes not only increase computational time and cost but also result in a higher number of enclosures, enclosing sometimes the same zero more than once. The main objective of this work is to enhance the applicability of Interval Newton method in cases where no efficient alternative are available. Thus, in this paper, the Interval Newton method is studied and an adjusted perturbation technique is proposed to address the cases where multiple zeros exist. In particular, the given function is vertically shifted. Then, the Interval Newton operator is applied once to this shifted function. The resulting enclosures are then used to efficiently partition the search interval. The successful application of the Interval Newton method is expected to improve overall performance and reduce reliance on bisection schemes. Experimental results on a set of problems demonstrate the effectiveness of the proposed technique.

Optimal eighth-order multiple root finding iterative methods using bivariate weight function

In this contribution, a novel eighth-order scheme is presented for solving nonlinear equations with multiple roots. The proposed scheme comprises of three steps with the modified Newton method as its first step followed by two weighted Newton steps involving one univariate and one bivariate function respectively. Analysis of convergence confirms that the presented scheme obtains optimal computational order of convergence. The efficiency of presented scheme is compared numerically with recent eighth-order methods. Functions like population growth problem, Newton’s beam problem, etc., have been considered for numerical experimentation. For the comparative study in the complex plane, we employed the concept of basins of attraction.

A Fast Temporal Decomposition Procedure for Long-Horizon Nonlinear Dynamic Programming

We propose a fast temporal decomposition procedure for solving long-horizon nonlinear dynamic programs. The core of the procedure is sequential quadratic programming (SQP) that utilizes a differentiable exact augmented Lagrangian as the merit function. Within each SQP iteration, we approximately solve the Newton system using an overlapping temporal decomposition strategy. We show that the approximate search direction is still a descent direction of the augmented Lagrangian provided the overlap size and penalty parameters are suitably chosen, which allows us to establish the global convergence. Moreover, we show that a unit step size is accepted locally for the approximate search direction and further establish a uniform, local linear convergence over stages. This local convergence rate matches the rate of the recent Schwarz scheme (Na et al. 2022). However, the Schwarz scheme has to solve nonlinear subproblems to optimality in each iteration, whereas we only perform a single Newton step instead. Numerical experiments validate our theories and demonstrate the superiority of our method. Funding: This work was supported by the National Science Foundation [Grant CNS-1545046] and the U.S. Department of Energy, Office of Science, Office of Advanced Scientific Computing Research [Grant DE-AC02-06CH11347].

Structure-exploiting Newton-type method for optimal control of switched systems

ABSTRACT This study proposes an efficient Newton-type method for the optimal control of switched systems under a given mode sequence. A mesh-refinement-based approach is utilised to discretise continuous-time optimal control problems (OCPs) and formulate a nonlinear program (NLP), which guarantees the local convergence of a Newton-type method. A dedicated structure-exploiting algorithm (Riccati recursion) is proposed to perform a Newton-type method for the NLP efficiently because its sparsity structure is different from a standard OCP. The proposed method computes each Newton step with linear time-complexity for the total number of discretization grids as the standard Riccati recursion algorithm. Additionally, the computation is always successful if the solution is sufficiently close to a local minimum. Conversely, general quadratic programming (QP) solvers cannot accomplish this because the Hessian matrix is inherently indefinite. Moreover, a modification on the reduced Hessian matrix is proposed using the nature of the Riccati recursion algorithm as the dynamic programming for a QP subproblem to enhance the convergence. A numerical comparison is conducted with off-the-shelf NLP solvers, which demonstrates that the proposed method is up to two orders of magnitude faster. Whole-body optimal control of quadrupedal gaits is also demonstrated and shows that the proposed method can achieve the whole-body model predictive control (MPC) of robotic systems with rigid contacts.

A Family of Nested General Linear Methods for Solving Ordinary Differential Equations

General linear methods (GLMs) was introduced as a generalization of Runge{Kutta methods (RKMs) and linear multistep methods (LMMs). The discovery of general linear method gave insight into the discovery of new methods that are neither RKMs or LMMs. Here, new classes of GLMs that are nested in their stages and mono-implicit in the output are presented, these methods are referred to as nested general linear methods (NGLMs). Procedures for deriving members that are algebraically stable are discussed herein and algebraically stable NGLMs have been derived up to order p = 5. Implementation procedure of these nested general linear methods which include the solution of non-linear systems of equations by simplified Newton iterations and step size changing strategy are discussed. The order p = 3 NGLM has been implemented on two test problems by variable step size, and the results compared with the results of MATLAB ode15s and RADAU IIA.

Performance of a New Sixth-Order Class of Iterative Schemes for Solving Non-Linear Systems of Equations

This manuscript is focused on a new parametric class of multi-step iterative procedures to find the solutions of systems of nonlinear equations. Starting from Ostrowski’s scheme, the class is constructed by adding a Newton step with a Jacobian matrix taken from the previous step and employing a divided difference operator, resulting in a triparametric scheme with a convergence order of four. The convergence order of the family can be accelerated to six by setting two parameters, resulting in a uniparametric family. We performed dynamic and numerical development to analyze the stability of the sixth-order family. Previous studies for scalar functions allow us to isolate those elements of the family with stable performance for solving practical problems. In this regard, we present dynamical planes showing the complexity of the family. In addition, the numerical properties of the class are analyzed with several test problems.

Application of PDE Constrained Optimization in Internal Combustion Engine Pollution Control

Aiming at the problem of secondary pollution of waters due to the difficulty of controlling the dosage of purifiers in the treatment of internal combustion engine pollution, a partial differential equation (referred to as PDE) constrained optimization algorithm based on l 1 -norm is proposed. The algorithm first converts the internal combustion engine control model of the scavenger dose into a constrained optimization problem with a l 1 -penalty term. Secondly, it introduces a dose constraint condition based on PDE and uses the inherent property of Moreau-Yosida regularization to establish a smooth minimization function. Finally, the semismooth Newton method is used to iteratively find the optimal solution. The results of the comparison experiment show that the algorithm in this paper has a great improvement in the results of Newton step number and dose area percentage.

A hybrid method for the interior inverse scattering problem

<abstract><p>In this paper, the interior inverse scattering problem of a cavity is considered. By use of a general boundary condition, we can reconstruct the shape of the cavity without a priori information of the boundary condition type. The method of fundamental solutions (MFS) with regularization is formulated for solving the scattered field and its normal derivative on the cavity boundary. Newton's method is employed to reconstruct the cavity boundary by satisfying the general boundary condition. This hybrid method copes with the ill-posedness of the inverse problem in the MFS step and deals with the nonlinearity in the Newton's step. Some computational examples are presented to demonstrate the effectiveness of our method.</p></abstract>

Distributed optimization by Newton consensus over connected digraphs

In this study, a Newton consensus method is proposed for the distributed optimization of a multi-agent systems operating over strongly connected digraphs. The approach proposes an approximate Newton step for both the primal and dual problems that can be implemented in a completely decentralized fashion. The asymmetry in the communication network is addressed by computing an approximate Newton step that only requires the out-Laplacian. The proposed Newton consensus approach does not require the exchange of derivative information between agents. In addition, the optimization approach avoids the explicit inversion of the approximate Hessian information for the computation of the Newton step. A simulation study demonstrates the effectiveness of the technique.

A Structure Exploiting SDP Solver for Robust Controller Synthesis

In this paper, we revisit structure exploiting SDP solvers dedicated to the solution of Kalman-Yakubovic-Popov semi-definite programs (KYP-SDPs). These SDPs inherit their name from the KYP lemma and they play a crucial role in e.g. robustness analysis, robust state feedback synthesis, and robust estimator synthesis for uncertain dynamical systems. Off-the-shelve SDP solvers require O(n6) arithmetic operations per Newton step to solve this class of problems, where n is the state dimension of the dynamical system under consideration. Specialized solvers reduce this complexity to O(n3). However, existing specialized solvers do not include semi-definite constraints on the Lyapunov matrix, which is necessary for controller synthesis. In this paper, we show how to include such constraints in structure exploiting KYP-SDP solvers.

A new search direction for full-Newton step infeasible interior-point method in linear optimization

In this work, we investigate a full Newton step infeasible interior-point method for linear optimization based on a new search direction which is obtained from an algebraic equivalent transformation of the central path system. Furthermore, we prove that the proposed method obtains an ε-optimal solution to the original problem in polynomial time.