Number Of Conjugate Gradient Iterations Research Articles

An Interior Point–Inspired Algorithm for Linear Programs Arising in Discrete Optimal Transport

Discrete optimal transport problems give rise to very large linear programs (LPs) with a particular structure of the constraint matrix. In this paper, we present a hybrid algorithm that mixes an interior point method (IPM) and column generation, specialized for the LP originating from the Kantorovich optimal transport problem. Knowing that optimal solutions of such problems display a high degree of sparsity, we propose a column generation–like technique to force all intermediate iterates to be as sparse as possible. The algorithm is implemented nearly matrix-free. Indeed, most of the computations avoid forming the huge matrices involved and solve the Newton system using only a much smaller Schur complement of the normal equations. We prove theoretical results about the sparsity pattern of the optimal solution, exploiting the graph structure of the underlying problem. We use these results to mix iterative and direct linear solvers efficiently in a way that avoids producing preconditioners or factorizations with excessive fill-in and at the same time guaranteeing a low number of conjugate gradient iterations. We compare the proposed method with two state-of-the-art solvers and show that it can compete with the best network optimization tools in terms of computational time and memory use. We perform experiments with problems reaching more than four billion variables and demonstrate the robustness of the proposed method. History: Accepted by Antonio Frangioni, Area Editor for Design & Analysis of Algorithms–Continuous. Funding: F. Zanetti received funding from the University of Edinburgh, in the form of a PhD scholarship. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2022.0184 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2022.0184 ). The complete IJOC Software and Data Repository is available at https://informsjoc.github.io/ .

Improved parallel preconditioners for multidisciplinary topology optimisations

ABSTRACTTwo commonly used preconditioners were evaluated for parallel solution of linear systems of equations with high condition numbers. The test cases were derived from topology optimisation applications in multiple disciplines, where the material distribution finite element methods were used. Because in this optimisation method, the equations rapidly become ill-conditioned due to disappearance of large number of elements from the design space as the optimisations progresses, it is shown that the choice for a suitable preconditioner becomes very crucial. In an earlier work the conjugate gradient (CG) method with a Block-Jacobi preconditioner was used, in which the number of CG iterations increased rapidly with the increasing number processors. Consequently, the parallel scalability of the method deteriorated fast due to the increasing loss of interprocessor information among the increased number of processors. By replacing the Block-Jacobi preconditioner with a sparse approximate inverse preconditioner, it is shown that the number of iterations to converge became independent of the number of processors. Therefore, the parallel scalability is improved.

Conjugate gradient adaptive matched filter

We consider an adaptive reduced-rank detector, referred to as the CG-AMF detector, which is obtained by using the conjugate gradient (CG) algorithm to solve for the weight vector of the adaptive matched filter (AMF). The CG is a computationally efficient iterative algorithm, which finds the projection of the AMF weight vector to the Krylov subspace with a dimension growing with the CG iterations. This effectively leads to a family of reduced-rank detectors indexed by the number of CG iterations. The main purpose of this paper is to examine the output signal-to-interference-and-noise ratio (SINR) of the CG-AMF detector in the presence of strong clutter/interference. Specifically, by exploiting a connection between the CG algorithm and the Lanczos algorithm, we show the output SINR can be asymptotically expressed in a simple form involving a Ritz vector of the sample covariance matrix. The probability density function (pdf) and expected value of the output SINR are then obtained based on this approximation. Our theoretical analysis of the CG-AMF detector is verified by computer simulation. Numerical comparisons are also made with several popular reduced-rank detectors using either data-independent or data-dependent rank reduction approaches. Our results show that for a fixed training size, the CG-AMF detector often reaches its peak output SINR with a lower rank compared with the other reduced-rank detectors, which implies that the CG-AMF detector has lower computational complexity and less training requirement.

Multigrid solution of a distributed optimal control problem constrained by the Stokes equations

In this work we construct multigrid preconditioners to accelerate the solution process of a linear-quadratic optimal control problem constrained by the Stokes system. The first order optimality conditions of the control problem form a linear system (the KKT system) connecting the state, adjoint, and control variables. Our approach is to eliminate the state and adjoint variables by essentially solving two Stokes systems, and to construct efficient multigrid preconditioners for the Schur-complement of the block associated with the state and adjoint variables. These multigrid preconditioners are shown to be of optimal order with respect to the convergence properties of the discrete methods used to solve the Stokes system. In particular, the number of conjugate gradient iterations is shown to decrease as the resolution increases, a feature shared by similar multigrid preconditioners for elliptic constrained optimal control problems.

A trust region method for solving semidefinite programs

When using interior point methods for solving semidefinite programs (SDP), one needs to solve a system of linear equations at each iteration. For problems of large size, solving the system of linear equations can be very expensive. In this paper, we propose a trust region algorithm for solving SDP problems. At each iteration we perform a number of conjugate gradient iterations, but do not need to solve a system of linear equations. Under mild assumptions, the convergence of this algorithm is established. Numerical examples are given to illustrate the convergence results obtained.

A Globally Convergent Filter‐Type Trust Region Method for Semidefinite Programming

When using interior methods for solving semidefinite programming (SDP), one needs to solve a system of linear equations at each iteration. For problems of large size, solving the system of linear equations can be very expensive. In this paper, based on a semismooth equation reformulation using Fischer′s function, we propose a filter method with trust region for solving large‐scale SDP problems. At each iteration we perform a number of conjugate gradient iterations, but do not need to solve a system of linear equations. Under mild assumptions, the convergence of this algorithm is established. Numerical examples are given to illustrate the convergence results obtained.

On the Moore–Penrose inverse in solving saddle‐point systems with singular diagonal blocks

SUMMARYThis paper deals with the role of the generalized inverses in solving saddle‐point systems arising naturally in the solution of many scientific and engineering problems when finite‐element tearing and interconnecting based domain decomposition methods are used to the numerical solution. It was shown that the Moore–Penrose inverse may be obtained in this case at negligible cost by projecting an arbitrary generalized inverse using orthogonal projectors. Applying an eigenvalue analysis based on the Moore–Penrose inverse, we proved that for simple model problems, the number of conjugate gradient iterations required for the solution of associate dual systems does not depend on discretization norms. The theoretical results were confirmed by numerical experiments with linear elasticity problems. Copyright © 2011 John Wiley & Sons, Ltd.

An eigen-based high-order expansion basis for structured spectral elements

We present an eigen-based high-order expansion basis for the spectral element approach with structured elements. The new basis exhibits a numerical efficiency significantly superior, in terms of the conditioning of coefficient matrices and the number of iterations to convergence for the conjugate gradient solver, to the commonly-used Jacobi polynomial-based expansion basis. This basis results in extremely sparse mass matrices, and it is very amenable to the diagonal preconditioning. Ample numerical experiments demonstrate that with the new basis and a simple diagonal preconditioner the number of conjugate gradient iterations to convergence has essentially no dependence or only a very weak dependence on the element order. The expansion bases are constructed by a tensor product of a set of special one-dimensional (1D) basis functions. The 1D interior modes are constructed such that the interior mass and stiffness matrices are simultaneously diagonal and have identical condition numbers. The 1D vertex modes are constructed to be orthogonal to all the interior modes. The performance of the new basis has been investigated and compared with other expansion bases.

Fast spectral-domain method for acoustic scattering problems.

This paper presents the application of the conjugate-gradient (CG) fast Fourier transform (FFT) (CG-FFT) method and the CG nonuniform FFT (CG-NUFFT) method for the integral equation arising from acoustic scattering problems. In the conventional method of moments (MoM) for integral equations, the CPU and memory requirements are O(N3) and O(N2), respectively, where N is the number of unknowns in the problem. The CG-FFT method, which combines the iterative conjugate-gradient method with FFT, reduces these requirements to O(KN log2 N) and O(N), respectively, where K is the number of CG iterations. The CG-NUFFT method differs from the CG-FFT method in that it makes use of nonuniform FFT algorithms instead of FFT to allow a nonuniform discretization. Therefore, the CG-NUFFT method can solve the integral equation with both uniform and nonuniform grid while retaining the efficiency of the CG-FFT method. These two methods are applied to solve for two-dimensional constant density acoustic scattering problems. Numerical results demonstrate that they can solve much larger problems than the MoM.

Incomplete Cholesky Factorizations with Limited Memory

We propose an incomplete Cholesky factorization for the solution of large-scale trust region subproblems and positive definite systems of linear equations. This factorization depends on a parameter p that specifies the amount of additional memory (in multiples of n, the dimension of the problem) that is available; there is no need to specify a drop tolerance. Our numerical results show that the number of conjugate gradient iterations and the computing time are reduced dramatically for small values of p. We also show that in contrast with drop tolerance strategies, the new approach is more stable in terms of number of iterations and memory requirements.

Implementation of Symanzik's improvement program for simulations of dynamical Wilson fermions in lattice QCD

We discuss the implementation of a Sheikholeslami-Wohlert term for simulations of lattice QCD with dynamical Wilson fermions as required by Symanzik's improvement program. We show that for the Hybrid Monte Carlo or Kramers equation algorithm standard even-odd preconditioning can be maintained. We design tests of the implementation using analytically and numerically computed cumulant expansions. We find that, for situations where the average number of Conjugate Gradient iterations exceeds 200, the overhead is only about 20%.

Chronological inversion method for the Dirac matrix in hybrid Monte Carlo

In hybrid Monte Carlo simulations for full QCD, the gauge fields evolve smoothly as a function of molecular dynamics time. Here we investigate improved methods of estimating the trial or starting solutions for the Dirac matrix inversion as superpositions of a chronological sequence of solutions in the recent past. By taking as the trial solution the vector which minimizes the residual in the linear space spanned by the past solutions, the number of conjugate gradient iterations per unit MD time is decreased by at least a factor of 2. Extensions of this basic approach to precondition the conjugate gradient iterations are also discussed.

Extrapolation methods for the Dirac inverter in hybrid Monte Carlo

In Hybrid Monte Carlo(HMC) simulations for full QCD, the gauge fields evolve smoothly as a function of Molecular Dynamics (MD) time. Thus we investigate improved methods of estimating the trial solutions to the Dirac propagator as superpositions of the solutions in the recent past. So far our best extrapolation method reduces the number of Conjugate Gradient iterations per unit MD time by about a factor of 4. Further improvements should be forthcoming as we further exploit the information of past trajectories.

Additive polynomial preconditioners for parallel computers

We present a polynomial preconditioner that can be used with the conjugate gradient method to solve symmetric and positive definite systems of linear equations. Each step of the preconditioning is achieved by simultaneously taking an iteration of the SOR method and an iteration of the reverse SOR method (equations taken in reverse order) and averaging the results. This yields a symmetric preconditioner that can be implemented on parallel computers by performing the forward and reverse SOR iterations simultaneously. We give necessary and sufficient conditions for additive preconditioners to be positive definite. We find an optimal parameter, ω, for the SOR-Additive linear stationary iterative method applied to 2-cyclic matrices. We show this method is asymptotically twice as fast as SSOR when the optimal ω is used. We compare our preconditioner to the SSOR polynomial preconditioner for a model problem. With the optimal ω, our preconditioner was found to be as effective as the SSOR polynomial preconditioner in reducing the number of conjugate gradient iterations. Parallel implementations of both methods are discussed for vector and multiple processors. Results show that if the same number of processors are used for both preconditioners, the SSOR preconditioner is more effective. If twice as many processors are used for the SOR-Additive preconditioner, it becomes more efficient than the SSOR preconditioner when the number of equations assigned to a processor is small. These results are confirmed by the Blue Chip emulator at the University of Washington.