Good Preconditioner Research Articles

A parallel preconditioning Schur complement approach for large scale industrial problems

The purpose of this paper is to introduce a new strategy to improve the convergence and efficiency of the class of domain decomposition known as Schur complement techniques related to interface variables for the simulation of mechanical, electrical and thermal problems in presence of cross points. More precisely, we are interested not only in domain decomposition with two equal parts having the same physical properties but rather in more general splitting components. It is known that in the first case, the optimal convergence with good pre-conditioner is obtained in two iterations and the problem is still challenging in the second case. The primary goal then is to fill part of the gap in such problem domain decomposition techniques and to contribute to solve extremely difficult industrial problems of large scale by using parallel sparse direct solver of the multi-core environment of the whole system and handling each part of the system independently of the change of the mesh or the shifting of the mathematical method of resolution and subsequently, we treat the interface as boundary conditions. The numerical experiments of our algorithm are performed on few models arising from discretization of partial differential equations using Finite Element Method (FEM).

Isogeometric simulation of acoustic radiation

In this paper we discuss the numerical solution of the Helmholtz equation with mixed boundary conditions on a 2D physical domain Ω. The so called radiation problem depends on the constant wavenumber k, that in some medical applications can be of order of thousands. For these values of k the classical Finite Element Method (FEM) faces up several numerical difficulties. To mitigate these limitations we apply the Isogeometric Analysis (IgA) to compute the approximated solution uh. Main steps of IgA are discussed and specific proposals for their fulfillment are addressed, with focus on some aspects not covered in available publications. In particular, we introduce a low distortion quadratic NURBS parametrization of Ω that represents exactly its boundary and contributes to the accuracy of uh. Our approach is non-isoparametric since uh is a bicubic tensor product polynomial B-spline function on Ω. This allows to improve the numerical solution refining the approximation space and keeping the coarser parametrization of the domain. Moreover, we discuss the role of the number of degrees of freedom in the directions perpendicular and longitudinal to wave front and its relationship with the noise and the shift in amplitude and phase of uh. The linear system derived from IgA discretization of the radiation problem is solved using GMRES and we show through experiments that the incomplete factorization of the Complex Shifted Laplacian provides a very good preconditioner. To solve the radiation problem, we have implemented IgA approach using the open source package GeoPDEs. A comparison with FEM is included, to provide evidence that IgA approach is superior since it is able to reduce significantly the pollution error, especially for high values of k, producing additionally smoother solutions which depend on less degrees of freedom.

Direct solution of the normal equations in interior point methods for convex transportation problems

Recent research has shown that very large convex transportation problems can be solved efficiently with interior point methods by finding a good pre-conditioner for an iterative linear-equation solver. In this article, it is demonstrated that the specific structure of the constraints allows use of a direct method for solving those linear equations with the same order of worst-case time complexity.

Benchmark Computations of the Phase Field Crystal and Functionalized Cahn-Hilliard Equations via Fully Implicit, Nesterov Accelerated Schemes

We introduce a fast solver for the phase field crystal (PFC) and functionalized Cahn-Hilliard (FCH) equations with periodic boundary conditions on a rectangular domain that features the preconditioned Nesterov accelerated gradient descent (PAGD) method. We discretize these problems with a Fourier collocation method in space, and employ various second-order schemes in time. We observe a significant speedup with this solver when compared to the preconditioned gradient descent (PGD) method. With the PAGD solver, fully implicit, second-order-in-time schemes are not only feasible to solve the PFC and FCH equations, but also do so more efficiently than some semi-implicit schemes in some cases where accuracy issues are taken into account. Benchmark computations of five different schemes for the PFC and FCH equations are conducted and the results indicate that, for the FCH experiments, the fully implicit schemes (midpoint rule and BDF2 equipped with the PAGD as a nonlinear time marching solver) perform better than their IMEX versions in terms of computational cost needed to achieve a certain precision. For the PFC, the results are not as conclusive as in the FCH experiments, which, we believe, is due to the fact that the nonlinearity in the PFC is milder nature compared to the FCH equation. We also discuss some practical matters in applying the PAGD. We introduce an averaged Newton preconditioner and a sweeping-friction strategy as heuristic ways to choose good preconditioner parameters. The sweeping-friction strategy exhibits almost as good a performance as the case of the best manually tuned parameters.

Convergence of parallel overlapping domain decomposition methods for the Helmholtz equation

We analyse parallel overlapping Schwarz domain decomposition methods for the Helmholtz equation, where the exchange of information between subdomains is achieved using first-order absorbing (impedance) transmission conditions, together with a partition of unity. We provide a novel analysis of this method at the PDE level (without discretization). First, we formulate the method as a fixed point iteration, and show (in dimensions 1, 2, 3) that it is well-defined in a tensor product of appropriate local function spaces, each with L^2 impedance boundary data. We then obtain a bound on the norm of the fixed point operator in terms of the local norms of certain impedance-to-impedance maps arising from local interactions between subdomains. These bounds provide conditions under which (some power of) the fixed point operator is a contraction. In 2-d, for rectangular domains and strip-wise domain decompositions (with each subdomain only overlapping its immediate neighbours), we present two techniques for verifying the assumptions on the impedance-to-impedance maps that ensure power contractivity of the fixed point operator. The first is through semiclassical analysis, which gives rigorous estimates valid as the frequency tends to infinity. At least for a model case with two subdomains, these results verify the required assumptions for sufficiently large overlap. For more realistic domain decompositions, we directly compute the norms of the impedance-to-impedance maps by solving certain canonical (local) eigenvalue problems. We give numerical experiments that illustrate the theory. These also show that the iterative method remains convergent and/or provides a good preconditioner in cases not covered by the theory, including for general domain decompositions, such as those obtained via automatic graph-partitioning software.

A Review on Acoustic Reconstruction of Temperature Profiles: From Time Measurement to Reconstruction Algorithm

Acoustic tomography is a technique widely used in non-intrusive temperature measurement. Times of flight (TOF) of acoustic waves can be used to estimate temperatures of a measurement area. After obtaining TOF of some transmitter-receiver pairs, the temperature profile of the probed area can be estimated by some reconstruction algorithms. Temperature profiles play a critical role in the safe operation and efficient control of thermal-related mechanical systems. This article aims to present a systematic review on the acoustic reconstruction of the temperature profile, including TOF measurements and reconstruction algorithms. First, time-domain, Fourier-domain, and hybrid domain methods for the TOF measurement are revisited and discussed. Second, algorithms for the solution of the reconstruction model are reviewed from three groups of regularization methods. We also discuss how to choose the regularization parameter and to form a good preconditioner for fast calculation. Third, some extensions in TOF measurements and reconstruction methods and potential errors that may affect the acoustic reconstruction are also discussed. Finally, some applications are presented. A framework for the whole reconstruction process can be a useful resource to help researchers select a proper method for specific applications.

Analysis of a Helmholtz preconditioning problem motivated by uncertainty quantification

This paper analyses the following question: let Aj, j = 1,2, be the Galerkin matrices corresponding to finite-element discretisations of the exterior Dirichlet problem for the heterogeneous Helmholtz equations ∇⋅ (Aj∇uj) + k2njuj = −f. How small must |A_{1} -A_{2}|_{L^{q}} and |{n_{1}} - {n_{2}}|_{L^{q}} be (in terms of k-dependence) for GMRES applied to either (mathbf {A}_1)^{-1}mathbf {A}_2 or A2(A1)− 1 to converge in a k-independent number of iterations for arbitrarily large k? (In other words, for A1 to be a good left or right preconditioner for A2?) We prove results answering this question, give theoretical evidence for their sharpness, and give numerical experiments supporting the estimates. Our motivation for tackling this question comes from calculating quantities of interest for the Helmholtz equation with random coefficients A and n. Such a calculation may require the solution of many deterministic Helmholtz problems, each with different A and n, and the answer to the question above dictates to what extent a previously calculated inverse of one of the Galerkin matrices can be used as a preconditioner for other Galerkin matrices.

Relationship between quality of basis for surface approximation and the effect of applying preconditioning strategies to their resulting linear systems

In this paper, we establish a relationship between preconditioning strategies for solving the linear systems behind a fitting surface problem using the Powell–Sabin finite element, and choosing appropriate basis of the spline vector space to which the fitting surface belongs. We study the problem of determining whether good (or effective) preconditioners lead to good basis and vice versa. A preconditioner is considered to be good if it either reduces the condition number of the preconditioned matrix or clusters its eigenvalues, and a basis is considered to be good if it has local support and constitutes a partition of unity. We present some illustrative numerical results which indicate that the basis associated to well-known good preconditioners do not have in general the expected good properties. Similarly, the preconditioners obtained from well-known good basis do not exhibit the expected good numerical properties. Nevertheless, taking advantage of the established relationship, we develop some adapted good preconditioning strategies which can be associated to good basis, and we also develop some adapted and not necessarily good basis that produce effective preconditioners.

An ADMM-based interior-point method for large-scale linear programming

ABSTRACT In this paper, we propose a new framework to implement interior point method (IPM) in order to solve some very large-scale linear programs (LPs). Traditional IPMs typically use Newton's method to approximately solve a subproblem that aims to minimize a log-barrier penalty function at each iteration. Due its connection to Newton's method, IPM is often classified as second-order method – a genre that is attached with stability and accuracy at the expense of scalability. Indeed, computing a Newton step amounts to solving a large system of linear equations, which can be efficiently implemented if the input data are reasonably sized and/or sparse and/or well-structured. However, in case the above premises fail, then the challenge still stands on the way for a traditional IPM. To deal with this challenge, one approach is to apply the iterative procedure, such as preconditioned conjugate gradient method, to solve the system of linear equations. Since the linear system is different in each iteration, it is difficult to find good pre-conditioner to achieve the overall solution efficiency. In this paper, an alternative approach is proposed. Instead of applying Newton's method, we resort to the alternating direction method of multipliers (ADMM) to approximately minimize the log-barrier penalty function at each iteration, under the framework of primal–dual path-following for a homogeneous self-dual embedded LP model. The resulting algorithm is an ADMM-Based Interior Point Method, abbreviated as ABIP in this paper. The new method inherits stability from IPM and scalability from ADMM. Because of its self-dual embedding structure, ABIP is set to solve any LP without requiring prior knowledge about its feasibility. We conduct extensive numerical experiments testing ABIP with large-scale LPs from NETLIB and machine learning applications. The results demonstrate that ABIP compares favourably with other LP solvers including SDPT3, MOSEK, DSDP-CG and SCS.

A Markov chain-based multi-elimination preconditioner for elliptic PDE problems

Preconditioning is a simple but very powerful technique for accelerating the convergence of an iterative method for solving large, sparse linear systems. Traditionally, to design a good preconditioner implemented on a personal computer or a cluster of computer systems with CPUs, the preconditioner should meet the following three conditions. (a) the preconditioner is a good approximation of the inverse of the original linear coefficient matrix in some sense, (b) Setting up the components needed for the preconditioning operator is easy, and (c) the cost for the application of the preconditioner with a given vector is low. However, as new computer systems are rapidly invented, such designing strategies need to be reinvestigated. Motivated by the random walk technique, which is a computationally intensive task, we proposed a different preconditioner based on a multi-elimination algorithm, where the inversion operator is approximated explicitly by a partial product of the Neumann series of the iterative matrix for the original system. The computation of such an approximation takes advantages of the strengths of the GPU system. We report some numerical results implemented by CUDA PETSc. In our implementation, we adopt the hybrid GPU–CPU approach that preconditioning setup and application are performed on GPU, while Krylov subspace iteration, which requires a certain degree of communications is done on CPU. This method minimizes data communication between GPU–CPU, which is a bottleneck for many existing algorithms.

Performance of relaxed iterative methods for image deblurring problems

In this paper, we consider performance of relaxation iterative methods for four types of image deblurring problems with different regularization terms. We first study how to apply relaxation iterative methods efficiently to the Tikhonov regularization problems, and then we propose how to find good preconditioners and near optimal relaxation parameters which are essential factors for fast convergence rate and computational efficiency of relaxation iterative methods. We next study efficient applications of relaxation iterative methods to Split Bregman method and the fixed point method for solving the L1-norm or total variation regularization problems. Lastly, we provide numerical experiments for four types of image deblurring problems to evaluate the efficiency of relaxation iterative methods by comparing their performances with those of Krylov subspace iterative methods. Numerical experiments show that the proposed techniques for finding preconditioners and near optimal relaxation parameters of relaxation iterative methods work well for image deblurring problems. For the L1-norm and total variation regularization problems, Split Bregman and fixed point methods using relaxation iterative methods perform quite well in terms of both peak signal to noise ratio values and execution time as compared with those using Krylov subspace methods.

Preconditioning for the Geometric Transportation Problem

$\newcommand{\eps}{\varepsilon}$In the geometric transportation problem, we are given a collection of points $P$ in $d$-dimensional Euclidean space, and each point is given a (positive or negative integer) supply. The goal is to find a transportation map that satisfies the supplies, while minimizing the total distance traveled. This problem has been widely studied in many fields of computer science: from computational geometry, to computer vision, graphics, and machine learning. In this work we study approximation algorithms for the geometric transportation problem. We give an algorithm which, for any fixed dimension $d$, finds a $(1+\eps)$-approximate transportation map in time nearly-linear in $n$, and polynomial in $\eps^{-1}$ and in the logarithm of the total positive supply. This is the first approximation scheme for the problem whose running time depends on $n$ as $n\cdot \mathrm{polylog}(n)$. Our techniques combine the generalized preconditioning framework of Sherman, which is grounded in continuous optimization, with simple geometric arguments to first reduce the problem to a minimum cost flow problem on a sparse graph, and then to design a good preconditioner for this latter problem.

An inexact dual logarithmic barrier method for solving sparse semidefinite programs

A dual logarithmic barrier method for solving large, sparse semidefinite programs is proposed in this paper. The method avoids any explicit use of the primal variable X and therefore is well-suited to problems with a sparse dual matrix S. It relies on inexact Newton steps in dual space which are computed by the conjugate gradient method applied to the Schur complement of the reduced KKT system. The method may take advantage of low-rank representations of matrices $$A_i$$ to perform implicit matrix-vector products with the Schur complement matrix and to compute only specific parts of this matrix. This allows the construction of the partial Cholesky factorization of the Schur complement matrix which serves as a good preconditioner for it and permits the method to be run in a matrix-free scheme. Convergence properties of the method are studied and a polynomial complexity result is extended to the case when inexact Newton steps are employed. A Matlab-based implementation is developed and preliminary computational results of applying the method to maximum cut and matrix completion problems are reported.

MultiGrid Preconditioners for Mixed Finite Element Methods of the Vector Laplacian

Due to the indefiniteness and poor spectral properties, the discretized linear algebraic system of the vector Laplacian by mixed finite element methods is hard to solve. A block diagonal preconditioner has been developed and shown to be an effective preconditioner by Arnold et al. (Acta Numer 15:1–155, 2006). The purpose of this paper is to propose alternative and effective block diagonal and approximate block factorization preconditioners for solving these saddle point systems. A variable V-cycle multigrid method with the standard point-wise Gauss–Seidel smoother is proved to be a good preconditioner for the discrete vector Laplacian operator. The major benefit of our approach is that the point-wise Gauss–Seidel smoother is more algebraic and can be easily implemented as a black-box smoother. This multigrid solver will be further used to build preconditioners for the saddle point systems of the vector Laplacian. Furthermore it is shown that Maxwell’s equations with the divergent free constraint can be decoupled into one vector Laplacian and one scalar Laplacian equation.

A Block Preconditioned Harmonic Projection Method for Large-Scale Nonlinear Eigenvalue Problems

We propose a block preconditioned harmonic projection (BPHP) method for solving large-scale nonlinear eigenproblems of the form $T(\lambda)v = 0$. Similar to classical preconditioned eigensolvers such as the locally optimal block preconditioned conjugate gradient method and preconditioned Lanczos, BPHP aims at computing a few eigenvalues of the nonlinear problem close to a specified shift, using preconditioners that enhance the local spectrum, without the need for exact solution of large shifted linear systems. We explore the development of search subspaces, stabilized preconditioning, nonlinear harmonic Rayleigh--Ritz projections, thick restart, and soft deflation capable of resolving linearly dependent eigenvectors. Numerical experiments show that BPHP with a good preconditioner is storage efficient, and it exhibits robust convergence. A moving-window-style partial deflation enables BPHP to reliably compute a large number of eigenvalues.

Constraint-Preconditioned Inexact Newton Method for Hydraulic Simulation of Large-Scale Water Distribution Networks

Many sequential mathematical optimization methods and simulation-based heuristics for optimal control and design of water distribution networks rely on a large number of hydraulic simulations. In this paper, we propose an efficient inexact subspace Newton method for hydraulic analysis of water distribution networks. By using sparse and well-conditioned fundamental null space bases, we solve the nonlinear system of hydraulic equations in a lower-dimensional kernel space of the network incidence matrix. In the inexact framework, the Newton steps are determined by solving the Newton equations only approximately using an iterative linear solver. Since large water network models are inherently badly scaled, Jacobian regularization is employed to improve the condition number of these linear systems and guarantee positive definiteness. After presenting a convergence analysis of the regularized inexact Newton method, we use the conjugate-gradient (CG) method to solve the sparse reduced Newton linear systems. Since CG is not effective without good preconditioners, we propose tailored constraint preconditioners that are computationally cheap because they are based only on invariant properties of the null-space linear systems and do not change with flows and pressures. The preconditioners are shown to improve the distribution of eigenvalues of the linear systems and so enable a more efficient use of the CG solver. Since contiguous Newton iterates can have similar solutions, each CG call is warm-started with the solution for a previous Newton iterate to accelerate its convergence rate. Operational network models are used to show the efficacy of the proposed preconditioners and the warm-starting strategy in reducing computational effort.

A comparison of two preconditioner algorithms within the representer-based four-dimensional variational data assimilation system for the Navy coastal ocean model

ABSTRACTFour-dimensional variational (4D-Var) data assimilation for operational systems requires the solution of large linear systems that are poorly conditioned in general. In addition to efficient iterative solvers for linear systems, using a good preconditioner is required to guarantee an acceptable solution with a small number of iterations. We consider the assimilation of ocean observations using a weak constraints 4D-Var for the Navy Coastal Ocean Model (NCOM), based on the representer method. Two methods of preconditioning the linear system are implemented, namely the scaling of the linear system by the square root of the observations error variances, and the approximating stabilised representer matrix based on the computation of some representer functions. We evaluate their convergence using criteria such as the norm of the residuals and of the gradient of the cost function, the analysis error evaluated at the observation locations, and finally the convergence of the sequence of analyses. Results from all criteria show that the fastest convergence is achieved with the rescaling preconditioner when the conjugate gradient used to solve the linear system is equipped with a suitable inner product instead of the Euclidean.

On Geometrical Properties of Preconditioners in IPMs for Classes of Block-Angular Problems

One of the most efficient interior-point methods for some classes of block-angular structured problems solves the normal equations by a combination of Cholesky factorizations and preconditioned conjugate gradient for, respectively, the block and linking constraints. In this work we show that the choice of a good preconditioner depends on geometrical properties of the constraint structure. In particular, the principal angles between the subspaces generated by the diagonal blocks and the linking constraints can be used to estimate ex ante the efficiency of the preconditioner. Numerical validation is provided with some generated optimization problems. An application to the solution of multicommodity network flow problems with nodal capacities and equal flows of up to 64 million variables and up to 7.9 million constraints is also presented. These computational results also show that predictor-corrector directions combined with iterative system solves can be a competitive option for large instances.

Solving security constrained optimal power flow problems by a structure exploiting interior point method

In this paper we present a new approach to solve the DC (n − 1) security constrained optimal power flow (SCOPF) problem by a structure exploiting interior point solver. Our approach is based on a reformulation of the linearised SCOPF model, in which most matrices that need to be factorized are constant. Hence, most factorizations and a large number of back-solve operations only need to be performed once. However, assembling the Schur complement matrix remains expensive in this scheme. To reduce the effort, we suggest using a preconditioned iterative method to solve the corresponding linear system. We suggest two main schemes to pick a good and robust preconditioner based on combining different “active” contingency scenarios of the SCOPF model. These new schemes are implemented within the object-oriented parallel solver (OOPS), a structure-exploiting primal-dual interior-point implementation. We give results on several SCOPF test problems. The largest example contains 500 buses. We compare the results from the original interior point method (IPM) implementation in OOPS and our new reformulation.

New preconditioners for systems of linear equations with Toeplitz structure

In this paper, we consider applying the preconditioned conjugate gradient (PCG) method to solve system of linear equations $$T x = \mathbf b $$ where $$T$$ is a block Toeplitz matrix with Toeplitz blocks (BTTB). We first consider Level-2 circulant preconditioners based on generalized Jackson kernels. Then, BTTB preconditioners based on a splitting of BTTB matrices are proposed. We show that the BTTB preconditioners based on splitting are special cases of embedding-based BTTB preconditioners, which are also good BTTB preconditioners. As an application, we apply the proposed preconditioners to solve BTTB least squares problems. Our preconditioners work for BTTB systems with nonnegative generating functions. The implementations of the construction of the preconditioners and the relevant matrix-vector multiplications are also presented. Finally, Numerical examples, including image restoration problems, are presented to demonstrate the efficiency of our proposed preconditioners.