Saddle Point Systems Research Articles

On preconditioner updates for sequences of saddle-point linear systems

Abstract Updating preconditioners for the solution of sequences of large and sparse saddle- point linear systems via Krylov methods has received increasing attention in the last few years, because it allows to reduce the cost of preconditioning while keeping the efficiency of the overall solution process. This paper provides a short survey of the two approaches proposed in the literature for this problem: updating the factors of a preconditioner available in a block LDLT form, and updating a preconditioner via a limited-memory technique inspired by quasi-Newton methods.

A low-rank approach to the solution of weak constraint variational data assimilation problems

Weak constraint four-dimensional variational data assimilation is an important method for incorporating data (typically observations) into a model. The linearised system arising within the minimisation process can be formulated as a saddle point problem. A disadvantage of this formulation is the large storage requirements involved in the linear system. In this paper, we present a low-rank approach which exploits the structure of the saddle point system using techniques and theory from solving large scale matrix equations. Numerical experiments with the linear advection–diffusion equation, and the non-linear Lorenz-95 model demonstrate the effectiveness of a low-rank Krylov subspace solver when compared to a traditional solver.

Refining the Lower Bound on the Positive Eigenvalues of Saddle Point Matrices with Insights on the Interactions between the Blocks

Efficiently solving saddle point systems like Karush--Kuhn--Tucker (KKT) systems is crucial for many algorithms in constrained nonlinear continuous optimization. Such systems can be very ill conditioned, in particular when the (1,1) block has few very small eigenvalues (see Rusten and Winther [SIAM J. Matrix Anal. Appl., 13 (1992), pp. 887--904]). However, it is commonly observed that despite these small eigenvalues, some sort of interaction between this (1,1) block and the (1,2) block actually occurs that may influence strongly the convergence of Krylov subspace methods like Minres. In this paper, we highlight some aspects of this interaction. We illustrate in particular, with some examples, how and in which circumstances the convergence of Minres might be affected by these few very small eigenvalues in the (1,1) block. We further derive theoretically a tighter lower bound on the positive eigenvalues of saddle point matrices of the KKT form.

Iterative Methods for Double Saddle Point Systems

We consider the iterative solution of a class of linear systems with double saddle point structure. Several block preconditioners for Krylov subspace methods are described and analyzed. We derive some bounds for the eigenvalues of preconditioned matrices and present results of numerical experiments using test problems from two different applications: the potential fluid flow problem and the modeling of liquid crystals directors.

Combining the Augmented Lagrangian Preconditioner with the Simple Schur Complement Approximation

The augmented Lagrangian (AL) preconditioner and its variants have been successfully applied to solve saddle point systems arising from the incompressible Navier--Stokes equations discretized by th...

SPMR: A Family of Saddle-Point Minimum Residual Solvers

We introduce a new family of saddle-point minimum residual methods for iteratively solving saddle-point systems using a minimum or quasi-minimum residual approach. No symmetry assumptions are made. The basic mechanism underlying the method is a novel simultaneous bidiagonalization procedure that yields a simplified saddle-point matrix on a projected Krylov-like subspace and allows for a monotonic short-recurrence iterative scheme. We develop a few variants, demonstrate the advantages of our approach, derive optimality conditions, and discuss connections to existing methods. Numerical experiments illustrate the merits of this new family of methods.

A New Lagrange-Newton-Krylov Solver for PDE-constrained Nonlinear Model Predictive Control

Abstract Real-time optimization of systems governed by partial differential equations (PDEs) presents significant computational challenges to nonlinear model predictive control (NMPC). The large-scale nature of PDEs often limits the use of standard nested black-box optimizers that require repeated forward simulations and expensive gradient computations. Hence, to ensure online solutions at relevant time-scales, large-scale NMPC algorithms typically require powerful, customized PDE-constrained optimization solvers. To this end, this paper proposes a new Lagrange-Newton-Krylov (LNK) method that targets the class of time-dependent nonlinear diffusion-reaction systems arising from chemical processes. The LNK solver combines a high-order spectral Petrov-Galerkin (SPG) method with a new, parallel preconditioner tailored for the large-scale saddle-point systems that form subproblems of Sequential Quadratic Programming (SQP) methods. To establish proof-of-concept, a case study uses a simple parallel MATLAB implementation of the preconditioner with 10 cores. As a step towards real-time control, the results demonstrate that large-scale diffusion-reaction optimization problems with more than 106 unknowns can be solved efficiently in less than a minute.

Prestructuring sparse matrices with dense rows and columns via null space methods

SummarySeveral applied problems may produce large sparse matrices with a small number of dense rows and/or columns, which can adversely affect the performance of commonly used direct solvers. By posing the problem as a saddle point system, an unconventional application of a null space method can be employed to eliminate dense rows and columns. The choice of null space basis is critical in retaining the overall sparse structure of the matrix. A new one‐sided application of the null space method is also presented to eliminate either dense rows or columns. These methods can be considered techniques that modify the nonzero structure of the matrix before employing a direct solver and may result in improved direct solver performance.

Fast auxiliary space preconditioners for linear elasticity in mixed form

A block-diagonal preconditioner with the minimal residual method and an approximate block-factorization preconditioner with the generalized minimal residual method are developed for Hu-Zhang mixed finite element methods for linear elasticity. They are based on a new stability result for the saddle point system in mesh-dependent norms. The mesh-dependent norm for the stress corresponds to the mass matrix which is easy to invert while for the displacement it is spectral equivalent to the Schur complement. A fast auxiliary space preconditioner based on the H 1 H^1 -conforming linear element of the linear elasticity problem is then designed for solving the Schur complement. For both diagonal and triangular preconditioners, it is proved that the conditioning numbers of the preconditioned systems are bounded above by a constant independent of both the crucial Lamé constant and the mesh size. Numerical examples are presented to support theoretical results. As byproducts, a new stabilized low order mixed finite element method is proposed and analyzed and superconvergence results for the Hu-Zhang element are obtained.

Analysis on block diagonal and triangular preconditioners for a PML system of an electromagnetic scattering problem

We shall propose several block triangular preconditioners for a PML system of an electromagnetic wave scattering problem and analyze the spectral behavior of the preconditioned systems. When the PML system is discretized by edge element methods, it results in a discrete system with its stiffness matrix being complex, symmetric but indefinite, which can be formulated into a real symmetric but indefinite saddle-point system. In order to preserve the symmetry of the coefficient matrix, we present block triangular preconditioners with two-sided preconditioning for the discrete PML system. We will estimate the lower and upper bounds of positive and negative eigenvalues of the preconditioned matrices, respectively. On the other hand, one may also like to apply some iteration methods for nonsymmetric linear systems in applications although the discrete systems are symmetric. To this end, we propose a block triangular preconditioner to precondition the systems only from one side and analyze the spectrum of the preconditioned systems. In addition, we have also established a spectral estimate of the preconditioned system by an effective preconditioner that was recently developed in literature. Numerical experiments are presented to demonstrate the effectiveness and robustness of these new preconditioners and our theoretical predictions on the spectral bounds of the preconditioned systems.

A block-diagonal preconditioner for a four-field mixed finite element method for Biot's equations

In this paper, we explore an efficient preconditioning method for the saddle point system resulting from a four-field mixed finite element method applied to Biot's consolidation model. The proposed preconditioner is a block diagonal preconditioner based on the Schur complement. We obtain bounds on the eigenvalues of the preconditioned matrix that are clustered away from 0. To reduce the computational expense, this preconditioner is inverted approximately. Some numerical results are provided to show the efficiency of our preconditioning strategy when applied to a poroelasticity problem in a layered medium.

Semi-convergence analysis of the GSS iteration methods for singular saddle point problems

In this paper, we use the generalized shift-splitting (GSS) iteration method to solve nonsymmetric singular saddle point systems with the symmetric positive (semi-)definite (1,1) sub-block and symmetric positive semi-definite (2,2) sub-block, and prove the semi-convergence of the GSS iteration method. Numerical experiments are presented to investigate the correctness of the theoretical analyses and the feasibility of the GSS iteration method.

Multigrid Methods for A Mixed Finite Element Method of The Darcy-Forchheimer Model.

An efficient nonlinear multigrid method for a mixed finite element method of the Darcy-Forchheimer model is constructed in this paper. A Peaceman-Rachford type iteration is used as a smoother to decouple the nonlinearity from the divergence constraint. The nonlinear equation can be solved element-wise with a closed formulae. The linear saddle point system for the constraint is reduced into a symmetric positive definite system of Poisson type. Furthermore an empirical choice of the parameter used in the splitting is proposed and the resulting multigrid method is robust to the so-called Forchheimer number which controls the strength of the nonlinearity. By comparing the number of iterations and CPU time of different solvers in several numerical experiments, our multigrid method is shown to convergent with a rate independent of the mesh size and the Forchheimer number and with a nearly linear computational cost.

A comparative study of null‐space factorizations for sparse symmetric saddle point systems

SummaryNull‐space methods for solving saddle point systems of equations have long been used to transform an indefinite system into a symmetric positive definite one of smaller dimension. A number of independent works in the literature have identified that we can interpret a null‐space method as a matrix factorization. We review these findings, highlight links between them, and bring them into a unified framework. We also investigate the suitability of using null‐space factorizations to derive sparse direct methods and present numerical results for both practical and academic problems.

Intrusive uncertainty quantification for hyperbolic-elliptic systems governing two-phase flow in heterogeneous porous media

We model random locations of spatial interfaces of heterogeneities in porous media by means of the hybrid stochastic Galerkin (HSG) approach. This approach extends the concept of the generalized polynomial chaos (PC) expansion for a multi-element decomposition of the multidimensional stochastic space. In this way, the physically two-dimensional hyperbolic-elliptic fractional flow formulation for two-phase flow in heterogeneous porous media is transformed from a random partial differential equation into a deterministic system for the coefficients of the PC expansion of the primary unknown saturation, total velocity, and global pressure. The hyperbolic part is discretized based on a central-upwind finite volume scheme along with a mixed finite element method for the elliptic part. The latter partly uses the tensor product structure of the saddle point system together with appropriate preconditioning to speed up the computations. Since we use the sequential implicit pressure explicit saturation (IMPES) approach, the elliptic part is solved in every time step. The proposed method is particularly well-suited for parallel computations and allows for the consideration of a huge variety of complex flow problems. We illustrate the power of the method by means of several striking numerical examples of different complexities including their numerical convergence analysis.

New Perturbation Bounds Analysis of a Kind of Generalized Saddle Point Systems

AbstractIn this paper we consider new perturbation bounds analysis of a kind of generalized saddle point systems. We provide perturbation upper bounds for the solutions of generalized saddle point systems, which extend the corresponding results in [W.-W. Xu, W. Li, New perturbation analysis for generalized saddle point systems, Calcolo., 46(2009), pp. 25-36] to more general cases.

New local generalized shift-splitting preconditioners for saddle point problems

Based on a local generalized shift-splitting of the saddle point matrix with symmetric positive definite (1, 1)-block and symmetric positive semidefinite (2, 2)-block, a new local generalized shift-splitting preconditioner with two shift parameters for solving saddle point problems is proposed. The preconditioner is extracted from a new local generalized shift-splitting iteration and can lead to the unconditional convergence of the iteration. In addition, we consider solving the saddle point systems by preconditioned Krylov subspace methods and discuss some properties of the preconditioned saddle point matrix with a deteriorated preconditioner, such as eigenvalues, eigenvectors, and degree of the minimal polynomial. Numerical experiments arising from a finite element discretization model of the Stokes problem are given to validate the effectiveness of the proposed preconditioner.

A Modified Implementation of MINRES to Monitor Residual Subvector Norms for Block Systems

Saddle-point systems, i.e., structured linear systems with symmetric matrices are considered. A modified implementation of (preconditioned) MINRES is derived which allows subvectors of the residual to be monitored individually. Compared to the implementation from the textbook of [H. C. Elman, D. J. Silvester, and A. J. Wathen, Finite Elements and Fast Iterative Solvers: With Applications in Incompressible Fluid Dynamics, 2nd ed., Oxford University Press, Oxford, 2014], our method requires one extra vector of storage and no additional applications of the preconditioner. Numerical experiments are included.

Low rank updates in preconditioning the saddle point systems arising from data assimilation problems

The numerical solution of saddle point systems has received a lot of attention over the past few years in a wide variety of applications such as constrained optimization, computational fluid dynamics and optimal control, to name a few. In this paper, we focus on the saddle point formulation of a large-scale variational data assimilation problem, where the computations involving the constraint blocks are supposed to be much more expensive than those related to the (1, 1) block of the saddle point matrix. New low-rank limited memory preconditioners exploiting the particular structure of the problem are proposed and analysed theoretically. Numerical experiments performed within the Object-Oriented Prediction System are presented to highlight the relevance of the proposed preconditioners.

Perturbation analysis of saddle-point problems

The saddle-point systemis considered, where A is an matrix with full column rank, B is an symmetric and positive definite matrix, and b is an m-vector. When the perturbed systems are still saddle-point systems, perturbation theory is presented under three cases, such as the case of A and b perturbed, the case of B and b perturbed, and the case of A, B and b perturbed. We find that the bounds for the errors of the solution depend on A and B. To remove their influence, some conditions are disposed. To improve the accuracy of the solution, a scaling is given. When the perturbed systems are generalized saddle-point systems, the block factorization is applied. Thus the sensitivity of the block factorization should be discussed, then perturbation theorems for this case are presented.