Inexact Newton Method Research Articles

Multipoint flux mixed finite element methods for slightly compressible flow in porous media

In this paper, we consider multipoint flux mixed finite element discretizations for slightly compressible Darcy flow in porous media. The methods are formulated on general meshes composed of triangles, quadrilaterals, tetrahedra or hexahedra. An inexact Newton method that allows for local velocity elimination is proposed for the solution of the nonlinear fully discrete scheme. We derive optimal error estimates for both the scalar and vector unknowns in the semidiscrete formulation. Numerical examples illustrate the convergence behavior of the methods, and their performance on test problems including permeability coefficients with increasing heterogeneity.

A new class of root-finding methods in $${\mathbb {R}}^n$$ R n : the inexact tensor-free Chebyshev–Halley class

A new class of methods for solving systems of nonlinear equations that can attain cubic convergence rate, named the inexact tensor-free Chebyshev–Halley class, is introduced. As its name implies, the methods belonging to this class do not require second-order derivative information (a third-order tensor of size $$n\times n\times n$$ ) and find the next iterate by approximately solving two linear systems. This new class can be seen as a generalization of the Chebyshev–Halley class. No norm reduction in the quadratic model is needed to ensure convergence as required by Steihaug and Suleiman (Appl Numer Math 67:230–242, 2013). This gives more flexibility for choosing the step than the Steihaug and Suleiman algorithm does. In addition, in the convergence analysis section, we show that, depending on reasonable assumptions, the methods of this class can have superlinear, quadratic, superquadratic, or cubic convergence rates. We presented numerical evidence that demonstrates significant improvement when utilizing the proposed inexact tensor-free methods when compared to the Steihaug and Suleiman algorithm. This new class of methods also enlarges the Inexact Newton step by an additional step. The role of this extra step is to accelerate the convergence of the Inexact Newton method, so that cubic convergence rate is achieved by the inexact tensor-free Chebyshev–Halley methods. For this reason, we also tested them numerically against Newton-GMRES and the results have shown that the extra-cost due to the computation of the further step is more than counterbalanced by the gain in convergence rate.

Toward Distributed OPF Using ALADIN

The present paper discusses the application of the recently proposed Augmented Lagrangian Alternating Direction Inexact Newton (ALADIN) method to non-convex AC Optimal Power Flow Problems (OPF) in a distributed fashion. In contrast to the often used Alternating Direction of Multipliers Method (ADMM), ALADIN guarantees locally quadratic convergence for AC OPF. Numerical results for 5 to 300 bus test cases indicate that ALADIN is able to outperform ADMM and to reduce the number of iterations by about one order of magnitude. We compare ALADIN to numerical results for ADMM documented in the literature. The improved convergence speed comes at the cost of increasing the communication effort per iteration. Therefore, we propose a variant of ALADIN that uses inexact Hessians to reduce communication. Additionally, we provide a detailed comparison of these ALADIN variants to ADMM from an algorithmic and communication perspective. Moreover, we prove that ALADIN converges locally at quadratic rate even for the relevant case of suboptimally solved local NLPs.

Nonlinear Electromagnetic Inverse Scattering Imaging Based on IN-LSQR

A nonlinear inversion scheme is proposed for electromagnetic inverse scattering imaging. It exploits inexact Newton (IN) and least square QR factorization (LSQR) methods to tackle the nonlinearity and ill-posedness of the electromagnetic inverse scattering problem. A nonlinear model of the inverse scattering in functional form is developed. At every IN iteration, the sparse storage method is adopted to solve the storage and computational bottleneck of Fréchet derivative matrix, a large-scale sparse Jacobian matrix. Moreover, to address the slow convergence problem encountered in the inexact Newton solution via Landweber iterations, an LSQR algorithm is proposed for obtaining a better solution of the internal large-scale sparse linear equations in the IN step. Numerical results demonstrate the applicability of the proposed IN-LSQR method to quantitative inversion of scatterer electric performance parameters. Moreover, compared with the inexact Newton method based on Landweber iterations, the proposed method significantly improves the convergence rate with less computational and storage cost.

A Low-Rank Inexact Newton–Krylov Method for Stochastic Eigenvalue Problems

Abstract This paper aims at the efficient numerical solution of stochastic eigenvalue problems. Such problems often lead to prohibitively high-dimensional systems with tensor product structure when discretized with the stochastic Galerkin method. Here, we exploit this inherent tensor product structure to develop a globalized low-rank inexact Newton method with which we tackle the stochastic eigenproblem. We illustrate the effectiveness of our solver with numerical experiments.

Parallelizing spectral deferred corrections across the method

In this paper we present two strategies to enable “parallelization across the method” for spectral deferred corrections (SDC). Using standard low-order time-stepping methods in an iterative fashion, SDC can be seen as preconditioned Picard iteration for the collocation problem. Typically, a serial Gauß–Seidel-like preconditioner is used, computing updates for each collocation node one by one. The goal of this paper is to show how this process can be parallelized, so that all collocation nodes are updated simultaneously. The first strategy aims at finding parallel preconditioners for the Picard iteration and we test three choices using four different test problems. For the second strategy we diagonalize the quadrature matrix of the collocation problem directly. In order to integrate non-linear problems we employ simplified and inexact Newton methods. Here, we estimate the speed of convergence depending on the time-step size and verify our results using a non-linear diffusion problem.

Comprehensive contact analysis for vertical-contact-mode triboelectric nanogenerators with micro-/nano-textured surfaces

The triboelectric nanogenerator (TENG) is a new energy technology to convert mechanical energy into electricity based on contact charging and electrostatic induction. To increase the effective power generation in a limited device size, micro-/nano-textures are often designed at the contact surfaces. As the contact region decreases to micro/nano scale, adhesive forces such as van der Waals and electrostatic forces become dominant. However, there is currently no comprehensive model including micro-/nano-textures and the above adhesive forces, and thus the selection and design of textures are still lack of theoretical guidance. In this paper, the numerical contact and electrical models considering van der Waals and electrostatic forces are established, and the computational methods such as inexact Newton method, bi-conjugate stabilized (Bi-CGSTAB) method and fast Fourier transform (FFT) technique are employed to quantitatively analyze the effects of charge densities, applied loads, texture shapes and sizes on both contact and electrical performance. It is shown that the electrostatic force can be ignored when the charge density is small, while with large charge density, the electrical performance is significantly overestimated if the electrostatic forces are neglected. Among the four selected types of textures (pyramid, cone, cylinder and cube), the pyramid and cube textures are found to provide the larger output voltage under different applied loads, and also the optimal sizes for high voltage are presented, respectively. This study can provide the guideline for the texture design of high power generation TENGs.

Inexact Newton regularization combined with gradient methods in Banach spaces

In this paper we investigate convergence and stability properties of an adaptation to Banach spaces of the algorithm REGINN (Rieder 1999 Inverse Problems 15 309–27). This inexact Newton method solves nonlinear inverse problems by means of linearizing the equation around the current iterate and subsequently applying a regularization technique in the so-called inner iteration in order to obtain a stable approximation of the resulting linearized system. The current iterate is then updated by adding this approximate solution.Using a generic gradient-like method as inner iteration, we provide a whole converge analysis of this Newton-type method, proving, under reasonable assumptions, strong convergence of the generated sequence to a solution of the inverse problem in the noiseless situation and the regularization property in the noisy-data case.Some numerical experiments are performed at the end of the paper for providing the necessary support to the theoretical results

Exact and inexact subsampled Newton methods for optimization

Abstract The paper studies the solution of stochastic optimization problems in which approximations to the gradient and Hessian are obtained through subsampling. We first consider Newton-like methods that employ these approximations and discuss how to coordinate the accuracy in the gradient and Hessian to yield a superlinear rate of convergence in expectation. The second part of the paper analyzes an inexact Newton method that solves linear systems approximately using the conjugate gradient (CG) method, and that samples the Hessian and not the gradient (the gradient is assumed to be exact). We provide a complexity analysis for this method based on the properties of the CG iteration and the quality of the Hessian approximation, and compare it with a method that employs a stochastic gradient iteration instead of the CG method. We report preliminary numerical results that illustrate the performance of inexact subsampled Newton methods on machine learning applications based on logistic regression.

An adaptive nonlinear elimination preconditioned inexact Newton algorithm for highly local nonlinear multicomponent PDE systems

This work aims to develop an adaptive nonlinear elimination preconditioned inexact Newton method as the numerical solution of large sparse multi-component partial differential equation systems with highly local nonlinearity. A nonlinear elimination algorithm used as a nonlinear preconditioner has been shown to be a practical technique for enhancing the robustness and improving the efficiency of an inexact Newton method for some challenging problems, such as the transonic full potential problems. The basic idea of our method is to remove some components causing troubles in order to decrease the impact of local nonlinearity on the global system. The two key elements of the method are the valid identification of the to-be-eliminated components and the choice of subspace correction systems, respectively. In the method, we employ the point-wise residual component of nonlinear systems as an indicator for selecting these to-be-eliminated components adaptively and build a subspace nonlinear system consisting of the components corresponding to the bad region and an auxiliary linearized subsystem to reduce the interfacial jump pollution. The numerical results demonstrate that the new approach significantly improves performance for incompressible fluid flow and heat transfer problems with highly local nonlinearity when compared to the classical inexact Newton method.

An inexact Newton method for solving complementarity problems in hydrodynamic lubrication

We present an iterative procedure based on a damped inexact Newton iteration for solving linear complementarity problems. We introduce the method in the framework of a popular problem arising in mechanical engineering: the analysis of cavitation in lubricated contacts. In this context, we show how the perturbation and the damping parameter are chosen in our method and we prove the global convergence of the entire procedure. A Fortran implementation of the method is finally analyzed. First, we validate the procedure and analyze all its components, performing also a comparison with a recently proposed technique based on the Fischer–Burmeister–Newton iteration. Then, we solve a 2D problem and provide some insights on an efficient implementation of the method exploiting routines of the Lapack and of the PETSc packages for the solution of inner linear systems.

Sparsity preserving optimal control of discretized PDE systems

We focus on the problem of optimal control of large-scale systems whose models are obtained by discretization of partial differential equations using the Finite Element (FE) or Finite Difference (FD) methods. The motivation for studying this pressing problem originates from the fact that the classical numerical tools used to solve low-dimensional optimal control problems are computationally infeasible for large-scale systems. Furthermore, although the matrices of large-scale FE or FD models are usually sparse banded or highly structured, the optimal control solution computed using the classical methods is dense and unstructured. Consequently, it is not suitable for efficient centralized and distributed real-time implementations. We show that the a priori (sparsity) patterns of the exact solutions of the generalized Lyapunov equations for FE and FD models are banded matrices. The a priori pattern predicts the dominant non-zero entries of the exact solution. We furthermore show that for well-conditioned problems, the a priori patterns are not only banded but also sparse matrices. On the basis of these results, we develop two computationally efficient methods for computing sparse approximate solutions of generalized Lyapunov equations. Using these two methods and the inexact Newton method, we show that the solution of the generalized Riccati equation can be approximated by a banded matrix. This enables us to develop a novel computationally efficient optimal control approach that is able to preserve the sparsity of the control law. We perform extensive numerical experiments that demonstrate the effectiveness of our approach.

Inexact Newton method for non-linear functions with values in a cone

ABSTRACTThe problem of finding a solution of non-linear inclusion problems in Banach space is considered in this paper. Using convex optimization techniques introduced by Robinson (Numer Math. 1972;19:341–347), a robust convergence theorem for the inexact Newton method is proved. As an application, an affine invariant version of Kantorovich’s theorem and Smale’s -theorem for the inexact Newton method is obtained.

Inexact Newton-Type Optimization with Iterated Sensitivities

This paper presents and analyzes an inexact Newton-type optimization method based on iterated sensitivities (INIS). A particular class of nonlinear programming (NLP) problems is considered, where a subset of the variables is defined by nonlinear equality constraints. The proposed algorithm considers any problem-specific approximation for the Jacobian of these constraints. Unlike other inexact Newton methods, the INIS-type optimization algorithm is shown to preserve the local convergence properties and the asymptotic contraction rate of the Newton-type scheme for the feasibility problem yielded by the same Jacobian approximation. The INIS approach results in a computational cost which can be made close to that of the standard inexact Newton implementation. In addition, an adjoint-free (AF-INIS) variant of the approach is presented which, under certain conditions, becomes considerably easier to implement than the adjoint based scheme. The applicability of these results is motivated specifically for dynamic ...

MARO: matrix rank optimization for the detection of small-size moving objects from aerial camera platforms

Moving objects detection from aerial camera platforms is a very challenging problem due to the small-size of the moving objects and the false motion of the static background elements. Although many methods have been proposed in this domain, they always have a trade-off between true detections and false detections. This paper proposes a novel solution called matrix rank optimization method (MARO) to achieve high true detections with low false detections. In MARO, the detection problem is formulated as a principal component pursuit with a transformation domain. The novelty of MARO is that it solves this problem by using the inexact Newton method and a backtracking behaviour in inexact augmented Lagrange multiplier. MARO has been extensively evaluated using DARPA VIVID, UCF aerial action, and VIRAT aerial datasets. The results show that MARO outperforms current-state-of-the-art methods, as well as lowers the execution time without sacrificing the accuracy.

Development of an efficient tightly coupled method for multiphysics reactor transient analysis

Picard Iteration is a widely used coupling method for multiphysics simulations. This method allows one to directly leverage existing and well-developed single-physics programs without re-writing large portions of the codes. In Picard Iteration, single-physics codes just iteratively pass solutions to each other as inputs until each code has reached a converged solution. However, multiphysics computation linked by Picard Iteration is susceptible to over-solving, which can make the overall computation much less efficient. Over-solving means that each single-physics code provides an accurate solution in each Picard Iteration, which is not necessary in practice. Solving the single-physics codes in an inexact manner, i.e. with relaxed termination criteria, can help avoid this problem. This work develops a modified Picard Iteration coupling method with adaptive, inexact termination criteria for the underlying single-physics codes. Also, nested within the inexact Picard Iteration, inexact Newton methods were applied in the single-physics codes. The effect on the overall computation efficiency due to the inexact (relaxed) termination criteria at both levels is investigated by applying them to solve reactor transient problems. A reactor dynamics problem with temperature feedback in one-dimensional slab geometry is used to scope the behavior of nested inexact solvers. Then these methods are applied to a larger two-dimensional Boiling Water Reactor (BWR) problem. Computational time savings reach 55% for the two-dimensional problem. Additionally, applying an inexact termination criterion (inexact Newton method) to each single-physics code results in a further time savings of up to 18%.

Numerical Simulation on the Adhesive Contact Between Rough Surfaces with Bi-Conjugate Gradient Stabilized Method

A numerical model with bi-conjugate gradient stabilized method (Bi-CGSTAB) is proposed to simulate the adhesive contact between rough surfaces. This model provides a self-consistent solution of surface separation and contact pressure throughout rough surface with the adhesive interactions governed by the Lennard-Jones potential. This model uses the inexact Newton method and the preconditioned Bi-CGSTAB method for each Newton iteration. Both left and right preconditioners are employed. This model is proved to be accurate and feasible and is suitable for the moderate adhesive contact between rough surfaces.

A two-component Matched Interface and Boundary (MIB) regularization for charge singularity in implicit solvation

We present a new Matched Interface and Boundary (MIB) regularization method for treating charge singularity in solvated biomolecules whose electrostatics are described by the Poisson–Boltzmann (PB) equation. In a regularization method, by decomposing the potential function into two or three components, the singular component can be analytically represented by the Green's function, while other components possess a higher regularity. Our new regularization combines the efficiency of two-component schemes with the accuracy of the three-component schemes. Based on this regularization, a new MIB finite difference algorithm is developed for solving both linear and nonlinear PB equations, where the nonlinearity is handled by using the inexact-Newton's method. Compared with the existing MIB PB solver based on a three-component regularization, the present algorithm is simpler to implement by circumventing the work to solve a boundary value Poisson equation inside the molecular interface and to compute related interface jump conditions numerically. Moreover, the new MIB algorithm becomes computationally less expensive, while maintains the same second order accuracy. This is numerically verified by calculating the electrostatic potential and solvation energy on the Kirkwood sphere on which the analytical solutions are available and on a series of proteins with various sizes.

Constraint-Preconditioned Inexact Newton Method for Hydraulic Simulation of Large-Scale Water Distribution Networks

Many sequential mathematical optimization methods and simulation-based heuristics for optimal control and design of water distribution networks rely on a large number of hydraulic simulations. In this paper, we propose an efficient inexact subspace Newton method for hydraulic analysis of water distribution networks. By using sparse and well-conditioned fundamental null space bases, we solve the nonlinear system of hydraulic equations in a lower-dimensional kernel space of the network incidence matrix. In the inexact framework, the Newton steps are determined by solving the Newton equations only approximately using an iterative linear solver. Since large water network models are inherently badly scaled, Jacobian regularization is employed to improve the condition number of these linear systems and guarantee positive definiteness. After presenting a convergence analysis of the regularized inexact Newton method, we use the conjugate-gradient (CG) method to solve the sparse reduced Newton linear systems. Since CG is not effective without good preconditioners, we propose tailored constraint preconditioners that are computationally cheap because they are based only on invariant properties of the null-space linear systems and do not change with flows and pressures. The preconditioners are shown to improve the distribution of eigenvalues of the linear systems and so enable a more efficient use of the CG solver. Since contiguous Newton iterates can have similar solutions, each CG call is warm-started with the solution for a previous Newton iterate to accelerate its convergence rate. Operational network models are used to show the efficacy of the proposed preconditioners and the warm-starting strategy in reducing computational effort.

A finite element multigrid-framework to solve the sea ice momentum equation

We present a multigrid framework for the solution of the momentum equation arising in Hibler's viscous-plastic (VP) sea ice model. This model is used in global climate models, for seasonal prediction and high-resolution short-term forecasting systems. The development of fast and robust solvers for the strongly nonlinear momentum equation is still a big issue. There are mainly three approaches to solve the momentum equation: a fixed-point iteration (Picard solver), an inexact Newton method and a subcycling procedure based on an elastic-viscous-plastic (EVP) model approximation. Simple fixed-point iterations call for a vast number of cycles. The Jacobian arising in Newton linearizations is ill-conditioned and unstructured. No efficient linear solvers are available up to date. One possibility to solve the linear systems arising within a Newton method is the preconditioned GMRES iteration that however still requires many steps. The commonly used line SOR preconditioner is computationally expensive. Especially on fine meshes, literature recommends to revise common solution strategies.We introduce a geometric multigrid method as a preconditioner to the GMRES iteration for accelerating the solution of the linear problems. We show that the convergence rate of the multigrid method is robust with respect to mesh refinement. This makes it an appealing method for high resolution simulations. We validate the robustness of the linear solver and compare the multigrid with ILU preconditioning. In particular on fine meshes (∼16 km–2 km), multigrid preconditioning can substantially reduce the computational effort and decreases iteration counts by 80%.