SSOR Research Articles

Parallel preconditioned conjugate gradient algorithm on GPU

We propose a parallel implementation of the Preconditioned Conjugate Gradient algorithm on a GPU platform. The preconditioning matrix is an approximate inverse derived from the SSOR preconditioner. Used through sparse matrix–vector multiplication, the proposed preconditioner is well suited for the massively parallel GPU architecture. As compared to CPU implementation of the conjugate gradient algorithm, our GPU preconditioned conjugate gradient implementation is up to 10 times faster (8 times faster at worst).

A modified symmetric successive overrelaxation method for augmented systems

In this paper, we establish a modified symmetric successive overrelaxation (MSSOR) method, to solve augmented systems of linear equations, which uses two relaxation parameters. This method is an extension of the symmetric SOR (SSOR) iterative method. The convergence of the MSSOR method for augmented systems is studied. Numerical examples show that the new method is an efficient method.

Performance of Zero-Level Fill-In Preconditioning Techniques for Iterative Solutions with Geotechnical Applications

AbstractBiot's symmetric indefinite linear systems of equations are commonly encountered in finite-element computations of geotechnical problems. The development of efficient solution methods for Biot's linear systems of equations is of practical importance to geotechnical software packages. In conjunction with the Krylov-subspace iterative method symmetric quasi-minimal residual (SQMR), some zero-level fill-in incomplete factorization preconditioning techniques including a symmetric successive overrelaxation (SSOR) type method and several zero-level incomplete LU [ILU(0)] methods are investigated and compared for Biot's symmetric indefinite linear systems of equations. Numerical experiments are carried out based on three practical geotechnical problems. Numerical results indicate that ILU(0) preconditioners are classical and generally efficient when adequately stabilized. However, the tunnel problem provides a counterexample demonstrating that ILU(0) preconditioners cannot be fully stabilized by prelimin...

Two class of synchronous matrix multisplitting schemes for solving linear complementarity problems

Many problems in the areas of scientific computing and engineering applications can lead to the solution of the linear complementarity problem LCP ( M , q ) . It is well known that the matrix multisplitting methods have been found very useful for solving LCP ( M , q ) . In this article, by applying the generalized accelerated overrelaxation (GAOR) and the symmetric successive overrelaxation (SSOR) techniques, we introduce two class of synchronous matrix multisplitting methods to solve LCP ( M , q ) . Convergence results for these two methods are presented when M is an H -matrix (and also an M -matrix). Also the monotone convergence of the new methods is established. Finally, the numerical results show that the introduced methods are effective for solving the large and sparse linear complementary problems.

Preconditioners for Indefinite Linear System from the Helmholtz Equation

Using the finite difference method to discretize the Helmholtz equation usually leads to a large spare linear system. Since the coefficient matrix of the linear system is frequently indefinite, it is difficult to solve iteratively. The approach taken in this paper is to precondition this linear system with SSOR and ILU preconditioners and then to solve it iteratively by using Krylov subspace method such as LSQR method. Numerical experiments are given in order to demonstrate the efficiency of the presented preconditioners. Key Words: Helmholtz equation; Krylov subspace method; Preconditioner

A Comparative Study on Iterative Progressive Numerical Methods for Boundary Element Analysis of Electromagnetic Multiple Scattering

This paper presents various types of iterative progressive numerical methods (IPNMs) for the computation of electromagnetic (EM) wave scattering from many objects and reports comparatively the performance of these methods. The original IPNM is similar to the Jacobi method which is one of the classical linear iterative solvers. Then the modified IPNMs are based on other classical solvers like the Gauss-Seidel (GS), the relaxed Jacobi, the successive overrelaxation (SOR), and the symmetric SOR (SSOR) methods. In the original and modified IPNMs, we repeatedly solve linear systems of equations by using a nonstationary iterative solver. An initial guess and a stopping criterion are discussed in order to realize a fast computation. We treat EM wave scattering from 27 perfectly electric conducting (PEC) spheres and evaluate the performance of the IPNMs. However, the SOR- and SSOR-type IPNMs are not subject to the above numerical test in this paper because an optimal relaxation parameter is not possible to determine in advance. The evaluation reveals that the IPNMs converge much faster than a standard BEM computation. The relaxed Jacobi-type IPNM is better than the other types in terms of the net computation time and the application range for the distance between objects.

A Comparative Analysis of Krylov Solvers for Three-Dimensional Simulations of Borehole Sensors

We perform a comparative analysis of three Krylov subspace methods, viz., the restarted generalized minimum residual (RGMRES), the conjugate gradient squared (CGS), and the stabilized biconjugate gradient (Bi-CGSTAB), for solving large non-Hermitian sparse linear systems arising from the 3-D finite-volume modeling of electromagnetic borehole sensors in complex earth formations. Incomplete LU factorization and symmetric successive overrelaxation preconditioning strategies are used to speed up the convergence rate. We compare these algorithms in terms of accuracy, convergence rate, and overall CPU time. Results show that CGS has a highly irregular convergence behavior, whereas RGMRES and Bi-CGSTAB provide similar numerical accuracy. However, the convergence rate and CPU time of the latter depend on the borehole sensor geometry and on the type of preconditioner adopted.

Preface

Preface

Block-Triangular Preconditioners for Systems Arising from Edge-Preserving Image Restoration

Signal and image restoration problems are often solved by minimizing a cost function consisting of an l(2) data-fidelity term and a regularization term. We consider a class of convex and edge-preserving regularization functions. In specific, half-quadratic regularization as a fixed-point iteration method is usually employed to solve this problem. The main aim of this paper is to solve the above-described signal and image restoration problems with the half-quadratic regularization technique by making use of the Newton method. At each iteration of the Newton method, the Newton equation is a structured system of linear equations of a symmetric positive definite coefficient matrix, and may be efficiently solved by the preconditioned conjugate gradient method accelerated with the modified block SSOR preconditioner. Our experimental results show that the modified block-SSOR preconditioned conjugate gradient method is feasible and effective for further improving the numerical performance of the half-quadratic regularization approach.

Characterizing sparse preconditioner performance for the support vector machine kernel

Abstract A two-class Support Vector Machine (SVM) classifier finds a hyperplane that separates two classes of data with the maximum margin. In a first learning phase, SVM involves the construction and solution of a primal-dual interiorpoint optimization problem. Each iteration of the interior-point method (IPM) requires a sparse linear system solution, which dominates the execution time of SVM learning. Solving this linear system can often be computationally expensive depending on the conditioning of the matrix. Consequently, preconditioned linear systems can lead to improved SVM performance while maintaining the classification accuracy. In this paper, we seek to characterize the role of preconditioning schemes for enhancing the SVM classifier performance. We compare and report on the solution time, convergence, and number of Newton iterations of the iterior-point method and classification accuracy of the SVM for 6 well-accepted preconditioning schemes and datasets chosen from well-known machine learning repositories. In particular, we introduce Δ -IPM that sparsifies the linear system at each iteration of the IPM. Our results indicate that on average the Jacobi and SSOR preconditioners perform 10.01 times better than other preconditioning schemes for IPM and 8.83 times better for Δ -IPM. Also, across all datasets Jacobi and SSOR perform between 2 to 30 times better than other schemes in both IPM and Δ -IPM. Moreover, Δ -IPM obtains a speedup over IPM performance on average by 1.25 and as much as 2 times speedup in the best case.

Asynchronous Multisplitting GAOR Method and Asynchronous Multisplitting SSOR Method for Systems of Weakly Nonlinear Equations

In this article, we introduce two new asynchronous multisplitting methods for solving the system of weakly nonlinear equations Ax = G(x) in which A is an n × n real matrix and G(x) = (g1(x), g2(x), . . . , gn(x))T is a P-bounded mapping. First, by generalized accelerated overrelaxation (GAOR) technique, we introduce the asynchronous parallel multisplitting GAOR method (including the synchronous parallel multisplitting AOR method as a special case) for solving the system of weakly nonlinear equations. Second, asynchronous parallel multisplitting method based on symmetric successive overrelaxation (SSOR) multisplitting is introduced, which is called asynchronous parallel multisplitting SSOR method. Then under suitable conditions, we establish the convergence of the two introduced methods. The given results contain synchronous multisplitting iterations as a special case.

GAUSS-SEIDEL AND SUCCESSIVE OVERRELAXATION METHODS FOR RADIATIVE TRANSFER WITH PARTIAL FREQUENCY REDISTRIBUTION

The linearly-polarized solar limb spectrum that is produced by scattering processes contains a wealth of information on the physical conditions and magnetic fields of the solar outer atmosphere, but the modeling of many of its strongest spectral lines requires solving an involved non-LTE radiative transfer problem accounting for partial redistribution (PRD) effects. Fast radiative transfer methods for the numerical solution of PRD problems are also needed for a proper treatment of hydrogen lines when aiming at realistic time-dependent magnetohydrodynamic simulations of the solar chromosphere. Here we show how the two-level atom PRD problem with and without polarization can be solved accurately and efficiently via the application of highly convergent iterative schemes based on the Gauss-Seidel (GS) and Successive Overrelaxation (SOR) radiative transfer methods that had been previously developed for the complete redistribution (CRD) case. Of particular interest is the Symmetric SOR method, which allows us to reach the fully converged solution with an order of magnitude of improvement in the total computational time with respect to the Jacobi-based local ALI (Accelerated Lambda Iteration) method.

Association of Collagen Type I α1 (COLIA1) Sp1 Polymorphism with Osteoporotic Fracture in Caucasian Post-menopausal Women: a Meta-analysis

This study was designed to summarize quantitatively the evidence for a relationship between collagen type I alpha1 (COLIA1) Sp1 polymorphism and osteoporotic fracture risk in Caucasian post-menopausal women. This meta-analysis included 16 studies, which analysed 2294 patients with fractures and 10 285 controls. The combined results showed that there was a significant difference in genotype distribution (SS odds ratio [OR] 0.72; Ss OR 1.18; ss OR 1.97) between patients with fractures and controls. When stratifying by the fracture site, it was found that: (i) patients with vertebral fractures had a significantly higher frequency of the Ss genotype and a lower frequency of the SS genotype than controls; and (ii) patients with non-vertebral fractures had a significantly higher frequency of the ss genotype and a lower frequency of the SS genotype than controls. This meta-analysis suggests that the COLIA1 Sp1 polymorphism may be associated with osteoporotic fracture in Caucasian post-menopausal women.

Parallel SSOR preconditioning implemented on dynamic SMP clusters with communication on the fly

The paper presents a comparative analysis of parallel implementation of the preconditioned conjugate gradient method with the symmetric-successive over-relaxation preconditioner. Two parallel implementations of the matrix solver are compared. The first one is a message-passing version executed on a cluster of workstations. The other one is an efficient version simulated on a novel architecture of dynamically reconfigurable shared memory clusters with a new paradigm of inter-processor communication called communication on the fly. The presented example shows high suitability of the proposed architecture for fine grain numerical computations. It can be very useful in the simulation of physical phenomena described as numerical problems suitable for fine grain parallel execution.

Shifted SSOR preconditioning technique for electromagnetic wave scattering problems

AbstractTo efficiently solve large dense complex linear system arising from electric field integral equations (EFIE) formulation of electromagnetic scattering problems, the multilevel fast multipole method (MLFMM) is used to accelerate the matrix‐vector product operations. The symmetric successive over‐relaxation (SSOR) preconditioner is constructed based on the near‐field matrix of the EFIE and employed to speed up the convergence rate of iterative methods. This technique can be greatly improved by shifting the near‐field matrix of the EFIE with the principle value term of the magnetic field integral equation (MFIE) operator. Numerical results demonstrate that this method can reduce both the number of iterations and the computational time significantly with low cost for construction and implementation of preconditioners. © 2009 Wiley Periodicals, Inc. Microwave Opt Technol Lett 51: 1035–1039, 2009; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/mop.24254

New parallel symmetric SOR preconditioners by multi-type partitioning

A new parallel symmetric successive over-relaxation (PSSOR) preconditioner is proposed in this paper by the multi-type partition techniques introduced in SIAM J. Scientific Computing 20, 2006, pp. 1513–1533. In a general matrix expression, it is proved to be symmetric and positive-definite (SPD) if the coefficient matrix of a linear system is SPD. It is also proved to be equivalent to the SSOR preconditioner using the multi-type ordering. Thus, it works for the preconditioned conjugate gradient method (PCG) and can be analysed by the classic SOR theory. Numerical tests on an anisotropic model problem show that the PSSOR preconditioner can make the PCG have a faster rate of convergence and better parallel performance than the red–black SSOR preconditioner. They also confirm that the PSSOR preconditioner can have a rate of convergence that is nearly the same as the classic sequential SSOR preconditioner when the problem has large anisotropy.

Accelerated convergence of the numerical simulation of incompressible flow in general curvilinear co-ordinates by discretizations on overset grids

The convergence rate of a methodology for solving incompressible flow in general curvilinear co-ordinates is analyzed. Overset grids (double-staggered grids type), each defined by the same boundaries as the physical domain are used for discretization. Both grids are Marker and Cell (MAC) quadrilateral meshes with scalar variables (pressure, temperature, etc.) arranged at the center and the Cartesian velocity components at the middle of the sides of the mesh. The problem was checked against benchmark solutions of natural convection in a squeezed cavity, heat transfer in concentric and eccentric horizontal cylindrical annuli and hot cylinder in a duct. Convergence properties of Poisson’s pressure equations which arise from application of the SIMPLE-like procedure are analyzed by several methods: successive overrelaxation, symmetric successive overrelaxation, modified incomplete factorization, and conjugate gradient. A genetic algorithm was developed to solve problems of numerical optimization of calculation time, in a space of iteration numbers and relaxation factors. The application provides a means of making an unbiased comparison between the double-staggered grids method and the standard interpolation method. Furthermore, the convergence rate was demonstrated with the well-known calculation of natural convection heat transfer in concentric horizontal cylindrical annuli. Calculation times when double staggered grids were used were 6–10 times shorter than those achieved by interpolation.

SSOR PRECONDITIONED INNER-OUTER FLEXIBLE GMRES METHOD FOR MLFMM ANALYSIS OF SCATTERING OF OPEN OBJECTS

To efficiently solve large dense complex linear system arising from electric field integral equations (EFIE) formulation of electromagnetic scattering problems, the multilevel fast multipole method (MLFMM) is used to accelerate the matrix-vector product operations. The inner-outer flexible generalized minimum residual method (FGMRES) is combined with the symmetric successive over- relaxation (SSOR) preconditioner based on the near-part matrix of the EFIE in the inner iteration of FGMRES to speed up the convergence rate of iterative methods. Numerical experiments with a few electromagnetic scattering problems for open structures are given to demonstrate the efficiency of the proposed method.

A modified SSOR iterative method for augmented systems

Golub, Wu and Yuan [G.H. Golub, X. Wu, J.Y. Yuan, SOR-like methods for augmented systems, BIT 41 (2001) 71–85] have presented the SOR-like algorithm to solve augmented systems. In this paper, we present the modified symmetric successive overrelaxation (MSSOR) method for solving augmented systems, which is based on Darvishi and Hessari’s work above. We derive its convergence under suitable restrictions on the iteration parameter, determine its optimal iteration parameter and the corresponding optimal convergence factor under certain conditions. Finally, we apply the MSSOR method to solve augmented systems.

A Performance Comparison of the Parallel Preconditioners for Iterative Methods for Large Sparse Linear Systems Arising from Partial Differential Equations on Structured Grids

In this paper we compare various parallel preconditioners such as Point-SSOR (Symmetric Successive OverRelaxation), ILU(0) (Incomplete LU) in the Wavefront ordering, ILU(0) in the Multi-color ordering, Multi-Color Block SOR (Successive OverRelaxation), SPAI (SParse Approximate Inverse) and pARMS (Parallel Algebraic Recursive Multilevel Solver) for solving large sparse linear systems arising from two-dimensional PDE (Partial Differential Equation)s on structured grids. Point-SSOR is well-known, and ILU(0) is one of the most popular preconditioner, but it is inherently serial. ILU(0) in the Wavefront ordering maximizes the parallelism in the natural order, but the lengths of the wavefronts are often nonuniform. ILU(0) in the Multi-color ordering is a simple way of achieving a parallelism of the order N, where N is the order of the matrix, but its convergence rate often deteriorates as compared to that of natural ordering. We have chosen the Multi-Color Block SOR preconditioner combined with direct sparse matrix solver, since for the Laplacian matrix the SOR method is known to have a nondeteriorating rate of convergence when used with the Multi-Color ordering. By using block version we expect to minimize the interprocessor communications. SPAI computes the sparse approximate inverse directly by least squares method. Finally, ARMS is a preconditioner recursively exploiting the concept of independent sets and pARMS is the parallel version of ARMS. Experiments were conducted for the Finite Difference and Finite Element discretizations of five two-dimensional PDEs with large meshsizes up to a million on an IBM p595 machine with distributed memory. Our matrices are real positive, i.e., their real parts of the eigenvalues are positive. We have used GMRES(m) as our outer iterative method, so that the convergence of GMRES(m) for our test matrices are mathematically guaranteed. Interprocessor communications were done using MPI (Message Passing Interface) primitives. The results show that in general ILU(0) in the Multi-Color ordering and ILU(0) in the Wavefront ordering outperform the other methods but for symmetric and nearly symmetric 5-point matrices Multi-Color Block SOR gives the best performance, except for a few cases with a small number of processors.