Nonlinear Preconditioner Research Articles

Comparison of nonlinear field-split preconditioners for two-phase flow in heterogeneous porous media

This work focuses on the development of a two-step field-split nonlinear preconditioner to accelerate the convergence of two-phase flow and transport in heterogeneous porous media. We propose a field-split algorithm named Field-Split Multiplicative Schwarz Newton (FSMSN), consisting in two steps: first, we apply a preconditioning step to update pressure and saturations nonlinearly by solving approximately two subproblems in a sequential fashion; then, we apply a global step relying on a Newton update obtained by linearizing the system at the preconditioned state. Using challenging test cases, FSMSN is compared to an existing field-split preconditioner, Multiplicative Schwarz Preconditioned for Inexact Newton (MSPIN), and to standard solution strategies such as the Sequential Fully Implicit (SFI) method or the Fully Implicit Method (FIM). The comparison highlights the impact of the upwinding scheme in the algorithmic performance of the preconditioners and the importance of the dynamic adaptation of the subproblem tolerance in the preconditioning step. Our results demonstrate that the two-step nonlinear preconditioning approach—and in particular, FSMSN—results in a faster outer-loop convergence than with the SFI and FIM methods. The impact of the preconditioners on computational performance–i.e., measured by wall-clock time–will be studied in a subsequent publication.

Deep preconditioners and their application to seismic wavefield processing

Seismic data processing heavily relies on the solution of physics-driven inverse problems. In the presence of unfavourable data acquisition conditions (e.g., regular or irregular coarse sampling of sources and/or receivers), the underlying inverse problem becomes very ill-posed and prior information is required to obtain a satisfactory solution. Sparsity-promoting inversion, coupled with fixed-basis sparsifying transforms, represent the go-to approach for many processing tasks due to its simplicity of implementation and proven successful application in a variety of acquisition scenarios. Nevertheless, such transforms rely on the assumption that seismic data can be represented as a linear combination of a finite number of basis functions. Such an assumption may not always be fulfilled, thus producing sub-optimal solutions. Leveraging the ability of deep neural networks to find compact representations of complex, multi-dimensional vector spaces, we propose to train an AutoEncoder network to learn a nonlinear mapping between the input seismic data and a representative latent manifold. The trained decoder is subsequently used as a nonlinear preconditioner for the solution of the physics-driven inverse problem at hand. Through synthetic and field data examples, the proposed nonlinear, learned transformations are shown to outperform fixed-basis transforms and converge faster to the sought solution for a variety of seismic processing tasks, ranging from deghosting to wavefield separation with both regularly and irregularly subsampled data.

Intelligent Seismic Deblending Through Deep Preconditioner

Seismic deblending is an ill-posed inverse problem that involves counteracting the effect of a blending matrix derived from the shots position and firing time. In this letter, we propose a seismic deblending method based on so-called deep preconditioners. A convolutional Autoencoder (AE) is first trained in a patch-wise fashion to learn an effective sparse representation of the common receiver gathers (CRGs) we aim to reconstruct. Then, the decoder branch of the trained AE is used as a nonlinear preconditioner for the deblending problem. Particularly, to avoid the explicit creation of a training dataset, we suggest to use the common shot gathers (CSGs) of the blended dataset itself to train the AE network, as they are not affected by incoherent blending noise. Numerical examples on synthetic and field datasets demonstrate the effectiveness of the proposed method in comparison with significantly comparable techniques: a dictionary-learning based deblending method; an end-to-end deblending convolution neutral network (CNN).

A Nonlinearly Preconditioned Forward-Backward Splitting Method and Applications

In this paper, we prove the weak convergence of the iterates generated by the nonlinearly preconditioned forward-backward splitting method for the sum of a maximally hypermonotone operator A and a hypercocoercive operator B under several suitable conditions. We provide several choices of the nonlinear preconditioners for solving nonlinearly composed inclusions. In particular, the backward-forward splitting method is recovered by the nonlinearly preconditioned forward-backward splitting method with a special choice of the nonlinear preconditioner.

A numerical study of the additive Schwarz preconditioned exact Newton method (ASPEN) as a nonlinear preconditioner for immiscible and compositional porous media flow

Domain decomposition methods are widely used as preconditioners for Krylov subspace linear solvers. In the simulation of porous media flow there has recently been a growing interest in nonlinear preconditioning methods for Newton’s method. In this work, we perform a numerical study of a spatial additive Schwarz preconditioned exact Newton (ASPEN) method as a nonlinear preconditioner for Newton’s method applied to both fully implicit or sequential implicit schemes for simulating immiscible and compositional multiphase flow. We first review the ASPEN method and discuss how the resulting linearized global equations can be recast so that one can use standard preconditioners developed for the underlying model equations. We observe that the local fully implicit or sequential implicit updates efficiently handle the local nonlinearities, whereas long-range interactions are resolved by the global ASPEN update. The combination of the two updates leads to a very competitive algorithm. We illustrate the behavior of the algorithm for conceptual one and two-dimensional cases, as well as realistic three dimensional models. A complexity analysis demonstrates that Newton’s method with a fully implicit scheme preconditioned by ASPEN is a very robust and scalable alternative to the well-established Newton’s method for fully implicit schemes.

A 6M digital twin for modeling and simulation in subsurface reservoirs

A 6M digital twin for modeling and simulation in subsurface reservoirs

A Multilayer Nonlinear Elimination Preconditioned Inexact Newton Method for Steady-State Incompressible Flow Problems in Three Dimensions

We develop a multilayer nonlinear elimination preconditioned inexact Newton method for a nonlinear algebraic system of equations, and a target application is the three-dimensional steady-state incompressible Navier--Stokes equations at high Reynolds numbers. Nonlinear steady-state problems are often more difficult to solve than time-dependent problems because the Jacobian matrix is less diagonally dominant, and a good initial guess from the previous time step is not available. For such problems, Newton-like methods may suffer from slow convergence or stagnation even with globalization techniques such as line search. In this paper, we introduce a cascadic multilayer nonlinear elimination approach based on feedback from intermediate solutions to improve the convergence of Newton iteration. Numerical experiments show that the proposed algorithm is superior to the classical inexact Newton method and other single layer nonlinear elimination approaches in terms of the robustness and efficiency. Using the proposed nonlinear preconditioner with a highly parallel domain decomposition framework, we demonstrate that steady solutions of the Navier--Stokes equations with Reynolds numbers as large as 7,500 can be obtained for the lid-driven cavity flow problem in three dimensions without the use of any continuation methods.

Nonlinearly preconditioned L‐BFGS as an acceleration mechanism for alternating least squares with application to tensor decomposition

SummaryWe derive nonlinear acceleration methods based on the limited‐memory Broyden–Fletcher–Goldfarb–Shanno (L‐BFGS) update formula for accelerating iterative optimization methods of alternating least squares (ALS) type applied to canonical polyadic and Tucker tensor decompositions. Our approach starts from linear preconditioning ideas that use linear transformations encoded by matrix multiplications and extends these ideas to the case of genuinely nonlinear preconditioning, where the preconditioning operation involves fully nonlinear transformations. As such, the ALS‐type iterations are used as fully nonlinear preconditioners for L‐BFGS, or equivalently, L‐BFGS is used as a nonlinear accelerator for ALS. Numerical results show that the resulting methods perform much better than either stand‐alone L‐BFGS or stand‐alone ALS, offering substantial improvements in terms of time to solution and robustness over state‐of‐the‐art methods for large and noisy tensor problems, including previously described acceleration methods based on nonlinear conjugate gradients and the nonlinear generalized minimal residual method. Our approach provides a general L‐BFGS‐based acceleration mechanism for nonlinear optimization.

An adaptive nonlinear elimination preconditioned inexact Newton algorithm for highly local nonlinear multicomponent PDE systems

This work aims to develop an adaptive nonlinear elimination preconditioned inexact Newton method as the numerical solution of large sparse multi-component partial differential equation systems with highly local nonlinearity. A nonlinear elimination algorithm used as a nonlinear preconditioner has been shown to be a practical technique for enhancing the robustness and improving the efficiency of an inexact Newton method for some challenging problems, such as the transonic full potential problems. The basic idea of our method is to remove some components causing troubles in order to decrease the impact of local nonlinearity on the global system. The two key elements of the method are the valid identification of the to-be-eliminated components and the choice of subspace correction systems, respectively. In the method, we employ the point-wise residual component of nonlinear systems as an indicator for selecting these to-be-eliminated components adaptively and build a subspace nonlinear system consisting of the components corresponding to the bad region and an auxiliary linearized subsystem to reduce the interfacial jump pollution. The numerical results demonstrate that the new approach significantly improves performance for incompressible fluid flow and heat transfer problems with highly local nonlinearity when compared to the classical inexact Newton method.

A Note on Adaptive Nonlinear Preconditioning Techniques

Nonlinear preconditioning is a globalization technique for Newton's method applied to systems of equations with unbalanced nonlinearities, in which nonlinear residual norm reduction stagnates due to slowly evolving subsets of the degrees of freedom. Even though the Newton corrections may effectively be sparse, a standard Newton method still requires large ill-conditioned linear systems resulting from global linearizations of the nonlinear residual to be solved at each step. Nonlinear preconditioners may enable faster global convergence by shifting work to where it is most strategic, on subsets of the original system. They require additional computation per outer iteration while aiming for many fewer outer iterations and correspondingly fewer global synchronizations. In this work, we improve upon previous nonlinear preconditioning implementations by introducing parameters that allow turning off nonlinear preconditioning during outer Newton iterations where it is not needed. Numerical experiments show that the adaptive nonlinear preconditioning algorithm has performance similar to monolithically applied nonlinear preconditioning, preserving robustness for some challenging problems representative of several PDE-based applications while saving work on nonlinear subproblems.

Nonlinearly preconditioned semismooth Newton methods for variational inequality solution of two-phase flow in porous media

Most existing methods for solving two-phase flow problems in porous media do not take the physically feasible saturation fractions between 0 and 1 into account, which often destroys the numerical accuracy and physical interpretability of the simulation. To calculate the solution without the loss of this basic requirement, we introduce a variational inequality formulation of the saturation equilibrium with a box inequality constraint, and use a conservative finite element method for the spatial discretization and a backward differentiation formula with adaptive time stepping for the temporal integration. The resulting variational inequality system at each time step is solved by using a semismooth Newton algorithm. To accelerate the Newton convergence and improve the robustness, we employ a family of adaptive nonlinear elimination methods as a nonlinear preconditioner. Some numerical results are presented to demonstrate the robustness and efficiency of the proposed algorithm. A comparison is also included to show the superiority of the proposed fully implicit approach over the classical IMplicit Pressure-Explicit Saturation (IMPES) method in terms of the time step size and the total execution time measured on a parallel computer.

Nonlinearly Preconditioned Optimization on Grassmann Manifolds for Computing Approximate Tucker Tensor Decompositions

Two accelerated optimization algorithms are presented for computing approximate Tucker tensor decompositions, formulated using orthonormal factor matrices, by minimizing error as measured by the Frobenius norm. The first is a nonlinearly preconditioned conjugate gradient (NPCG) algorithm, wherein a nonlinear preconditioner is used to generate a direction which replaces the gradient in the nonlinear conjugate gradient iteration. The second is a nonlinear GMRES (N-GMRES) algorithm, in which a linear combination of past iterates and a tentative new iterate, generated by a nonlinear preconditioner, is minimized to produce an improved search direction. The Euclidean versions of these methods are extended to the manifold setting, where optimization on Grassmann manifolds is used to handle orthonormality constraints and to allow isolated minimizers. Several modifications are required for use on manifolds: logarithmic maps are used to determine required tangent vectors, retraction mappings are used in the line se...

Pipelined, Flexible Krylov Subspace Methods

We present variants of the Conjugate Gradient (CG), Conjugate Residual (CR), and Generalized Minimal Residual (GMRES) methods which are both pipelined and flexible. These allow computation of inner products and norms to be overlapped with operator and nonlinear or nondeterministic preconditioner application.The methods are hence aimed at hiding network latencies and synchronizations which can become computational bottlenecks in Krylov methods on extreme-scale systems or in the strong-scaling limit. The new variants are not arithmetically equivalent to their base flexible Krylov methods, but are chosen to be similarly performant in a realistic use case, the application of strong nonlinear preconditioners to large problems which require many Krylov iterations. We provide scalable implementations of our methods as contributions to the PETSc package and demonstrate their effectiveness with practical examples derived from models of mantle convection and lithospheric dynamics with heterogeneous viscosity structure. These represent challenging problems where multiscale nonlinear preconditioners are required for the current state-of-the-art algorithms, and are hence amenable to acceleration with our new techniques. Large-scale tests are performed in the strong-scaling regime on a contemporary leadership supercomputer, where speedups approaching, and even exceeding $2\times$ can be observed. We conclude by analyzing our new methods with a performance model targeted at future exascale machines.

Nonlinear Preconditioning Techniques for Full-Space Lagrange--Newton Solution of PDE-Constrained Optimization Problems

The full-space Lagrange--Newton algorithm is one of the numerical algorithms for solving problems arising from optimization problems constrained by nonlinear partial differential equations. Newton-type methods enjoy fast convergence when the nonlinearity in the system is well-balanced; however, for some problems, such as the control of incompressible flows, even linear convergence is difficult to achieve and a long stagnation period often appears in the iteration history. In this work, we introduce a nonlinearly preconditioned inexact Newton algorithm for the boundary control of incompressible flows. The system has nine field variables, and each field variable plays a different role in the nonlinearity of the system. The nonlinear preconditioner approximately removes some of the field variables, and as a result, the nonlinearity is balanced and inexact Newton converges much faster when compared to the unpreconditioned inexact Newton method or its two-grid version. Some numerical results are presented to d...

Nonlinear Preconditioning: How to Use a Nonlinear Schwarz Method to Precondition Newton's Method

For linear problems, domain decomposition methods can be used directly as iterative solvers, but also as preconditioners for Krylov methods. In practice, Krylov acceleration is almost always used, since the Krylov method finds a much better residual polynomial than the stationary iteration, and thus converges much faster. We show in this paper that also for non-linear problems, domain decomposition methods can either be used directly as iterative solvers, or one can use them as preconditioners for Newton's method. For the concrete case of the parallel Schwarz method, we show that we obtain a preconditioner we call RASPEN (Restricted Additive Schwarz Preconditioned Exact Newton) which is similar to ASPIN (Additive Schwarz Preconditioned Inexact Newton), but with all components directly defined by the iterative method. This has the advantage that RASPEN already converges when used as an iterative solver, in contrast to ASPIN, and we thus get a substantially better preconditioner for Newton's method. The iterative construction also allows us to naturally define a coarse correction using the multigrid full approximation scheme, which leads to a convergent two level non-linear iterative domain decomposition method and a two level RASPEN non-linear preconditioner. We illustrate our findings with numerical results on the Forchheimer equation and a non-linear diffusion problem.

EPlace-MS: Electrostatics-Based Placement for Mixed-Size Circuits

We propose an electrostatics-based placement algorithm for large-scale mixed-size circuits (ePlace-MS). ePlace-MS is generalized, flat, analytic and nonlinear. The density modeling method eDensity is extended to handle the mixed-size placement. We conduct detailed analysis on the correctness of the gradient formulation and the numerical solution, as well as the rationale of dc removal and the advantages over prior density functions. Nesterov’s method is used as the nonlinear solver, which shows high yet stable performance over mixed-size circuits. The steplength is set as the inverse of Lipschitz constant of the gradient function, while we develop a backtracking method to prevent overestimation. An approximated nonlinear preconditioner is developed to minimize the topological and physical differences between large macros and standard cells. Besides, we devise a simulated annealer to legalize the layout of macros and use a second-phase global placement to reoptimize the standard cell layout. All the above innovations are integrated into our mixed-size placement prototype ePlace-MS, which outperforms all the related works in literature with better quality and efficiency. Compared to the leading-edge mixed-size placer NTUplace3, ePlace-MS produces up to 22.98% and on average 8.22% shorter wirelength over all the 16 modern mixed-size benchmark circuits with the same runtime.

A high-resolution computational model of the deforming human heart.

Modeling of the heart ventricles is one of the most challenging tasks in soft tissue mechanics because cardiac tissue is a strongly anisotropic incompressible material with an active component of stress. In most current approaches with active force, the number of degrees of freedom (DOF) is limited by the direct method of solution of linear systems of equations. We develop a new approach for high-resolution heart models with large numbers of DOF by: (1) developing a hex-dominant finite element mixed formulation and (2) developing a Krylov subspace iterative method that is able to solve the system of linearized equations for saddle-point problems with active stress. In our approach, passive cardiac tissue is modeled as a hyperelastic, incompressible material with orthotropic properties, and mixed pressure-displacement finite elements are used to enforce incompressibility. Active stress is generated by a model with force dependence on length and velocity of muscle shortening. The ventricles are coupled to a lumped circulatory model. For efficient solution of linear systems, we use Flexible GMRES with a nonlinear preconditioner based on block matrix decomposition involving the Schur complement. Three methods for approximating the inverse of the Schur complement are evaluated: inverse of the pressure mass matrix; least squares commutators; and sparse approximate inverse. The sub-matrix corresponding to the displacement variables is preconditioned by a V-cycle of hybrid geometric-algebraic multigrid followed by correction with several iterations of GMRES preconditioned by sparse approximate inverse. The overall solver is demonstrated on a high-resolution two ventricle mesh based on a human anatomy with roughly 130K elements and 1.7M displacement DOF. Effectiveness of the numerical method for active contraction is shown. To the best of our knowledge, this solver is the first to efficiently model ventricular contraction using an iterative linear solver for the mesh size demonstrated and therefore opens the possibility for future very high-resolution models. In addition, several relatively simple benchmark problems are designed for a verification exercise to show that the solver is functioning properly and correctly solves the underlying mathematical model. Here, the output of the newly designed solver is compared to that of the mechanics component of Chaste ('Cancer, Heart and Soft Tissue Environment'). These benchmark tests may be used by other researchers to verify their newly developed methods and codes.

A model reduction approach to numerical inversion for a parabolic partial differential equation

We propose a novel numerical inversion algorithm for the coefficients of parabolic partial differential equations, based on model reduction. The study is motivated by the application of controlled source electromagnetic exploration, where the unknown is the subsurface electrical resistivity and the data are time resolved surface measurements of the magnetic field. The algorithm presented in this paper considers inversion in one and two dimensions. The reduced model is obtained with rational interpolation in the frequency (Laplace) domain and a rational Krylov subspace projection method. It amounts to a nonlinear mapping from the function space of the unknown resistivity to the small dimensional space of the parameters of the reduced model. We use this mapping as a nonlinear preconditioner for the Gauss–Newton iterative solution of the inverse problem. The advantage of the inversion algorithm is twofold. First, the nonlinear preconditioner resolves most of the nonlinearity of the problem. Thus the iterations are less likely to get stuck in local minima and the convergence is fast. Second, the inversion is computationally efficient because it avoids repeated accurate simulations of the time-domain response. We study the stability of the inversion algorithm for various rational Krylov subspaces, and assess its performance with numerical experiments.

A nonlinearly preconditioned conjugate gradient algorithm for rank‐R canonical tensor approximation

SummaryAlternating least squares (ALS) is often considered the workhorse algorithm for computing the rank‐R canonical tensor approximation, but for certain problems, its convergence can be very slow. The nonlinear conjugate gradient (NCG) method was recently proposed as an alternative to ALS, but the results indicated that NCG is usually not faster than ALS. To improve the convergence speed of NCG, we consider a nonlinearly preconditioned NCG (PNCG) algorithm for computing the rank‐R canonical tensor decomposition. Our approach uses ALS as a nonlinear preconditioner in the NCG algorithm. Alternatively, NCG can be viewed as an acceleration process for ALS. We demonstrate numerically that the convergence acceleration mechanism in PNCG often leads to important pay‐offs for difficult tensor decomposition problems, with convergence that is significantly faster and more robust than for the stand‐alone NCG or ALS algorithms. We consider several approaches for incorporating the nonlinear preconditioner into the NCG algorithm that have been described in the literature previously and have met with success in certain application areas. However, it appears that the nonlinearly PNCG approach has received relatively little attention in the broader community and remains underexplored both theoretically and experimentally. Thus, this paper serves several additional functions, by providing in one place a concise overview of several PNCG variants and their properties that have only been described in a few places scattered throughout the literature, by systematically comparing the performance of these PNCG variants for the tensor decomposition problem, and by drawing further attention to the usefulness of nonlinearly PNCG as a general tool. In addition, we briefly discuss the convergence of the PNCG algorithm. In particular, we obtain a new convergence result for one of the PNCG variants under suitable conditions, building on known convergence results for non‐preconditioned NCG. Copyright © 2014 John Wiley & Sons, Ltd.

Optimal scaling parameters for sparse grid discretizations

SUMMARY We apply iterative subspace correction methods to elliptic PDE problems discretized by generalized sparse grid systems. The involved subspace solvers are based on the combination of all anisotropic full grid spaces that are contained in the sparse grid space. Their relative scaling is at our disposal and has significant influence on the performance of the iterative solver. In this paper, we follow three approaches to obtain close-to-optimal or even optimal scaling parameters of the subspace solvers and thus of the overall subspace correction method. We employ a Linear Program that we derive from the theory of additive subspace splittings, an algebraic transformation that produces partially negative scaling parameters that result in improved asymptotic convergence properties, and finally, we use the OptiCom method as a variable nonlinear preconditioner. Copyright © 2014 John Wiley & Sons, Ltd.