Symmetric Linear Systems Research Articles

Multicolor low‐rank preconditioner for general sparse linear systems

SummaryThis article presents a multilevel parallel preconditioning technique for solving general large sparse linear systems of equations. Subdomain coloring is invoked to reorder the coefficient matrix by multicoloring the adjacency graph of the subdomains, resulting in a two‐level block diagonal structure. A full binary tree structure is then built to facilitate the construction of the preconditioner. A key property that is exploited is the observation that the difference between the inverse of the original matrix and that of its block diagonal approximation is often well approximated by a low‐rank matrix. This property and the block diagonal structure of the reordered matrix lead to a multicolor low‐rank (MCLR) preconditioner. The construction procedure of the MCLR preconditioner follows a bottom‐up traversal of the tree . All irregular matrix computations, such as ILU factorizations and related triangular solves, are restricted to leaf nodes where these operations can be performed independently. Computations in nonleaf nodes only involve easy‐to‐optimize dense matrix operations. In order to further reduce the number of iteration of the Preconditioned Krylov subspace procedure, we combine MCLR with a few classical block‐relaxation techniques. Numerical experiments on various test problems are proposed to illustrate the robustness and efficiency of the proposed approach for solving large sparse symmetric and nonsymmetric linear systems.

Modified Hermitian-normal splitting iteration methods for a class of complex symmetric linear systems

In this paper, the modifications of the Hermitian-Normal splitting iteration methods for solving a class of complex symmetric linear systems are presented. Theoretical analysis shows that the modified iteration methods of Hermitian-normal splitting are unconditionally convergent; the coefficient matrices of the two linear systems solved in each iteration of the methods are real symmetric positive definite. Inexact version of the methods employs the Krylov subspace method as an internal iteration to accelerate. Numerical examples from two model problems are given to illustrate the effectiveness of the modified iteration methods.

Reducing the effect of global synchronization in delayed gradient methods for symmetric linear systems

Compared with arithmetic operation, communication cost is often the bottleneck on modern computers, and thus should be paid increasing attention when choosing algorithms. Lagged gradient methods are known for their error tolerance and fast convergence. However, it appears that their parallel behavior is not well understood. In this paper, we explore the cyclic formulations of lagged gradient methods and s-dimensional methods for reducing global synchronizations. We provide parallel implementations for these methods and propose some new variants. A comparison is then reported for different gradient iterative schemes. To illustrate the performance, we run a number of experiments, from which we conclude that our formulations perform better than traditional methods in view of both iteration count and computing time.

Fast gradient methods with alignment for symmetric linear systems without using Cauchy step

The performance of gradient methods has been considerably improved by the introduction of delayed parameters. Recently, the revealing of second-order information has given rise to the Cauchy-based methods with alignment, which are generally considered as the state of the art of gradient methods. This paper investigates the spectral properties of minimal gradient and asymptotically optimal steps, and then suggests three fast methods with alignment without using the Cauchy step. The convergence results are provided, and numerical experiments show that the new methods provide competitive alternatives to the classical Cauchy-based methods. In particular, alignment gradient methods present advantages over the Krylov subspace methods in some situations, which makes them attractive in practice.

Minimum residual modified HSS iteration method for a class of complex symmetric linear systems

For solving a class of complex symmetric system of linear equations, we apply the minimum residual technique to the modified Hermitian and skew-Hermitian splitting (MHSS) iteration scheme and propose an iteration method referred to as minimum residual MHSS (MRMHSS) iteration method. Compared with the classical MHSS method, the MRMHSS method has two more iteration parameters, which can be automatically and easily computed. Then, some properties of the MRMHSS iteration method are carefully studied. Finally, we use four examples to test the performance of the MRMHSS iteration method by comparing its numerical results with three other iteration methods.

A Survey of Low-Rank Updates of Preconditioners for Sequences of Symmetric Linear Systems

The aim of this survey is to review some recent developments in devising efficient preconditioners for sequences of symmetric positive definite (SPD) linear systems A k x k = b k , k = 1 , … arising in many scientific applications, such as discretization of transient Partial Differential Equations (PDEs), solution of eigenvalue problems, (Inexact) Newton methods applied to nonlinear systems, rational Krylov methods for computing a function of a matrix. In this paper, we will analyze a number of techniques of updating a given initial preconditioner by a low-rank matrix with the aim of improving the clustering of eigenvalues around 1, in order to speed-up the convergence of the Preconditioned Conjugate Gradient (PCG) method. We will also review some techniques to efficiently approximate the linearly independent vectors which constitute the low-rank corrections and whose choice is crucial for the effectiveness of the approach. Numerical results on real-life applications show that the performance of a given iterative solver can be very much enhanced by the use of low-rank updates.

Analyzing random permutations for cyclic coordinate descent

We consider coordinate descent methods on convex quadratic problems, in which exact line searches are performed at each iteration. (This algorithm is identical to Gauss-Seidel on the equivalent symmetric positive definite linear system.) We describe a class of convex quadratic problems for which the random-permutations version of cyclic coordinate descent (RPCD) outperforms the standard cyclic coordinate descent (CCD) approach, yielding convergence behavior similar to the fully-random variant (RCD). A convergence analysis is developed to explain the empirical observations.

Numerical aspects of two coupled harmonic oscillators

Abstract In this study an interesting symmetric linear system is considered. As a first step we obtain the Lagrangian of the system. Secondly, we derive the classical Euler- Lagrange equations of the system. Finally, numerical and analytic solution for these equations have been presented for some chosen initial conditions.

English

The adaptive s-step CG algorithm is a solver for sparse symmetric positive definite linear systems designed to reduce the synchronization cost per iteration while still achieving a user-specified accuracy requirement. In this work, we improve the adaptive s-step conjugate gradient algorithm by the use of iteratively updated estimates of the largest and smallest Ritz values, which give approximations of the largest and smallest eigenvalues of A, using a technique due to G.Meurant and P.Tichý (2018). The Ritz value estimates are used to dynamically update parameters for constructing Newton or Chebyshev polynomials so that the conditioning of the s-step bases can be continuously improved throughout the iterations. These estimates are also used to automatically set a variable related to the ratio of the sizes of the error and residual, which was previously treated as an input parameter. We show through numerical experiments that in many cases the new algorithm improves upon the previous adaptive s-step approach both in terms of numerical behavior and reduction in number of synchronizations.

Hopf bifurcation at infinity in 3D symmetric piecewise linear systems. Application to a Bonhoeffer–van der Pol oscillator

In this work, a Hopf bifurcation at infinity in three-dimensional symmetric continuous piecewise linear systems with three zones is analyzed. By adapting the so-called closing equations method, which constitutes a suitable technique to detect limit cycles bifurcation in piecewise linear systems, we give for the first time a complete characterization of the existence and stability of the limit cycle of large amplitude that bifurcates from the point at infinity. Analytical expressions for the period and amplitude of the bifurcating limit cycles are obtained. As an application of these results, we study the appearance of a large amplitude limit cycle in a Bonhoeffer–van der Pol oscillator.

Modified QHSS iteration methods for a class of complex symmetric linear systems

We introduce a modified QHSS (MQHSS) iteration method for solving a class of complex symmetric linear systems, and prove that this MQHSS iteration method is convergent under certain conditions. We also consider acceleration of the MQHSS iteration by Krylov subspace methods. Numerical experiments show the good performance of the MQHSS iteration method.

A Class of Approximate Inverse Preconditioners Based on Krylov-Subspace Methods for Large-Scale Nonconvex Optimization

We introduce a class of positive definite preconditioners for the solution of large symmetric indefinite linear systems or sequences of such systems, in optimization frameworks. The preconditioners are iteratively constructed by collecting information on a reduced eigenspace of the indefinite matrix by means of a Krylov-subspace solver. A spectral analysis of the preconditioned matrix shows the clustering of some eigenvalues and possibly the nonexpansion of its spectrum. Extensive numerical experimentation is carried out on standard difficult linear systems and by embedding the class of preconditioners within truncated Newton methods for large-scale unconstrained optimization (the issue of major interest). Although the Krylov-based method may provide modest information on matrix eigenspaces, the results obtained show that the proposed preconditioners lead to substantial improvements in terms of efficiency and robustness, particularly on very large nonconvex problems.

An Algebraic Sparsified Nested Dissection Algorithm Using Low-Rank Approximations

We propose a new algorithm for the fast solution of large, sparse, symmetric positive-definite linear systems, spaND (sparsified Nested Dissection). It is based on nested dissection, sparsification, and low-rank compression. After eliminating all interiors at a given level of the elimination tree, the algorithm sparsifies all separators corresponding to the interiors. This operation reduces the size of the separators by eliminating some degrees of freedom but without introducing any fill-in. This is done at the expense of a small and controllable approximation error. The result is an approximate factorization that can be used as an efficient preconditioner. We then perform several numerical experiments to evaluate this algorithm. We demonstrate that a version using orthogonal factorization and block-diagonal scaling takes fewer CG iterations to converge than previous similar algorithms on various kinds of problems. Furthermore, this algorithm is provably guaranteed to never break down and the matrix stays symmetric positive-definite throughout the process. We evaluate the algorithm on some large problems show it exhibits near-linear scaling. The factorization time is roughly $\mathcal{O}$(N), and the number of iterations grows slowly with N.

Certain efficient iterative methods for bipolar fuzzy system of linear equations

A bipolar fuzzy set model is an extension of fuzzy set model. We develop new iterative methods: generalized Jacobi, generalized Gauss-Seidel, refined Jacobi, refined Gauss-seidel, refined generalized Jacobi and refined generalized Gauss-seidel methods, for solving bipolar fuzzy system of linear equations(BFSLEs). We decompose n × n BFSLEs into 4n × 4n symmetric crisp linear system. We present some results that give the convergence of proposed iterative methods. We solve some BFSLEs to check the validity, efficiency and stability of our proposed iterative schemes. Further, we compute Hausdorff distance between the exact solutions and approximate solution of our proposed schemes. The numerical examples show that some proposed methods converge for the BFSLEs, but Jacobi and Gauss-seidel iterative methods diverge for BFSLEs. Finally, comparison tables show the performance, validity and efficiency of our proposed iterative methods for BFSLEs.

A two-sweep shift-splitting iterative method for complex symmetric linear systems

Recently, Chen and Ma [21] constructed the generalized shift-splitting (GSS) preconditioner, and gave the corresponding theoretical analysis and numerical experiments. In this paper, based on the generalized shift-splitting (GSS) preconditioner, we generalize their algorithms and further study the two-sweep shift-splitting (TSSS) preconditioner for complex symmetric linear systems. Moreover, by similar theoretical analysis, we obtain that the two-sweep shift-splitting iterative method is unconditionally convergent. In finally, one example is provided to confirm the effectiveness.

Batched transpose-free ADI-type preconditioners for a Poisson solver on GPGPUs

We investigate the iterative solution of a symmetric positive definite linear system involving the shifted Laplacian as the system matrix on General Purpose Graphics Processing Units (GPGPUs). We consider in particular the Chebyshev iteration for its reduced global communication. The ADI-type preconditioner involves solving multiple (batched) symmetric positive tridiagonal Toeplitz systems along each coordinate direction. We investigate several variants how to solve these tridiagonal systems, the Thomas algorithm, the Thomas combined with the SPIKE algorithm, and a polynomial approximation of the inverse. We test the various implementations numerically by means of two- and three-dimensional examples. It turns out that a combination of the Thomas algorithm and the approximate inverse leads to a solution that does not need either tiling or transpositions. As such none of the kernels uses an extensive amount of shared memory which yields a very high GPU utilization and more importantly optimal coalesced global memory access patterns.

Retraction notice to: The generalized double steps scale-SOR iteration method for solving complex symmetric linear systems [CAM 346 (2018) 284 – 306

Retraction notice to: The generalized double steps scale-SOR iteration method for solving complex symmetric linear systems [CAM 346 (2018) 284 – 306

Non-smooth Newton Methods for Deformable Multi-body Dynamics

We present a framework for the simulation of rigid and deformable bodies in the presence of contact and friction. Our method is based on a non-smooth Newton iteration that solves the underlying nonlinear complementarity problems (NCPs) directly. This approach allows us to support nonlinear dynamics models, including hyperelastic deformable bodies and articulated rigid mechanisms, coupled through a smooth isotropic friction model. The fixed-point nature of our method means it requires only the solution of a symmetric linear system as a building block. We propose a new complementarity preconditioner for NCP functions that improves convergence, and we develop an efficient GPU-based solver based on the conjugate residual (CR) method that is suitable for interactive simulations. We show how to improve robustness using a new geometric stiffness approximation and evaluate our method’s performance on a number of robotics simulation scenarios, including dexterous manipulation and training using reinforcement learning.

Uzawa methods for a class of block three‐by‐three saddle‐point problems

SummaryIn this work, we consider numerical methods for solving a class of block three‐by‐three saddle‐point problems, which arise from finite element methods for solving time‐dependent Maxwell equations and some other applications. The direct extension of the Uzawa method for solving this block three‐by‐three saddle‐point problem requires the exact solution of a symmetric indefinite system of linear equations at each step. To avoid heavy computations at each step, we propose an inexact Uzawa method, which solves the symmetric indefinite linear system in some inexact way. Under suitable assumptions, we show that the inexact Uzawa method converges to the unique solution of the saddle‐point problem within the approximation level. Two special algorithms are customized for the inexact Uzawa method combining the splitting iteration method and a preconditioning technique, respectively. Numerical experiments are presented, which demonstrated the usefulness of the inexact Uzawa method and the two customized algorithms.

On Euler preconditioned SHSS iterative method for a class of complex symmetric linear systems

In this paper, we propose an Euler preconditioned single-step HSS (EP-SHSS) iterative method for solving a broad class of complex symmetric linear systems. The proposed method can be applied not only to the non-singular complex symmetric linear systems but also to the singular ones. The convergence (semi-convergence) properties of the proposed method are carefully discussed under suitable restrictions. Furthermore, we consider the acceleration of the EP-SHSS method by preconditioned Krylov subspace method and discuss the spectral properties of the corresponding preconditioned matrix. Numerical experiments verify the effectiveness of the EP-SHSS method either as a solver or as a preconditioner for solving both non-singular and singular complex symmetric linear systems.