Block SOR Method Research Articles

Block SOR methods for rank-deficient least-squares problems

Many papers have discussed preconditioned block iterative methods for solving full rank least-squares problems. However very few papers studied iterative methods for solving rank-deficient least-squares problems. Miller and Neumann (1987) proposed the 4-block SOR method for solving the rank-deficient problem. Here a 2-block SOR method and a 3-block SOR method are proposed to solve such problem. The convergence of the block SOR methods is studied. The optimal parameters are determined. Comparison between the 2-block SOR method and the 3-block SOR method is given also.

Methodes iteratives de type sor pour resoudre les problemes des moindres carres*

In order to solve least squares problems the block SOR iterative methods are proposed and investigated by some authors. In this paper, we generalize the usual block SOR methods. Other five SOR type iterative methods are defined. Their convergence are discussed, the optimal parameters and optimal spectral radii about the SOR type methods are given.

On domains of superior convergence of the SSOR method to that of the SOR method

Let v denote the spectral radius of J A B , the block Jacobi iteration matrix. For the classes of (1) nonsingular M-matrices and (2) p-cyclic, p ⩾ 3, consistently ordered matrices, we study domains in the ( v, ω) plane when v < 1, where the block SSOR iteration method has at least as favorable an asymptotic rate of convergence as the block SOR method. Let L A ω and S A ω denote, respectively, the block SOR and SSOR iteration matrices. For the class of nonsingular M-matrices A, we determine conditions when the spectral radii satisfy ρ( S A ω) ⩽ ρ( L A ω) ∀0 < ω ⩽ 2 1 + v and ∀0 ⩽ v < 1 . Under these conditions we also show that the optimal SOR iteration parameter is ω b = 1. For the class of p-cyclic, p ⩾ 3, consistently ordered matrices A we determine for which ω's and v's, ρ( S A ω) < |ω − 1| [ ⩽ ρ( L A ω)]. Our investigations make use of the equality case in Wielandt's inequality between the spectral radii of a complex matrix and its nonnegative and irreducible majorizers and of Rouché's theorem for the location of zeros of complex functions.

Some recent results on the modified SOR theory

For the solution of the linear system Ax=( I− T) x= c (1), where T is a weakly cyclic of index p⩾2 matrix of a special structure, the block SOR method with different relaxation factors associated with the row blocks of A is considered. First, a well-known relationship connecting the eigenvalue spectra of the block Jacobi matrix T of A and its associated modified SOR (MSOR) matrix is proved for all p⩾3, via an approach due to Varga, Niethammer, and Cai. Next, it is shown that the matrix analogue of the eigenvalue relationship holds at least for p=3. This, together with the facts that the matrix analogue holds true for p=2 and also for any p⩾3, provided all relaxation factors coincide, suggests a more general validity of the aforementioned matrix identity. Then, based on the matrix relationship, an equivalence is established between the MSOR method and a stationary p-parameter p-step iterative one for the solution of (1). So one can study convergence properties and determine optimal parameters of both methods by studying the simpler of the two, that is, the p-step one. Finally, some applications of the theory developed when p=2 are presented, and a brief discussion concerning comparisons of the optimum MSOR and AOR methods in a specific case is given.

SOR for AX− XB= C

We consider a new approach to the block SOR method applied to linear systems of equations which can be written as a matrix equation AX− XB= C. Such systems arise, for example, from finite difference discretizations of seperable elliptic boundary value problems on rectangular domains. On one hand, this gives us an iterative method for the solution of such matrix equations (e.g., Lyapunov's matrix equation where B=− A T ), and on the other hand, the problem of choosing appropriate parameters for the block SOR method can be written in a more compact form which may be helpful, especially, for non-self-adjoint problems, i.e., if A and B are nonsymmetric. Using this technique, we determine—under more general assumptions than those of Chin and Manteuffel—the optimal SOR parameters for the model problem of a convection-diffusion equation.

Comparisons of nonnegative splittings of matrices

The comparison of the asymptotic rates of convergence of two iteration matrices induced by two splitting of the same matrix has arisen in the work of many authors. In this article, some new comparison theorems for two nonnegative splittings (a splitting A= M− N is nonnegative if M −1 exist and M −1 N is nonnegative) are derived. They extend the known results in the literature. In addition, we also point out three incorrect conditions in a paper by Beauwens. Furthermore, we give some reasonable conditions ensuring the strict inequality between the asymptotic convergence rates. This also answers the open question which additional and appropriate conditions should be imposed on Miller-Neumann splittings to obtain strict inequality. Finally, some applications to a class of generalized AOR, SOR, and JOR iterative methods whose special cases imply block (also point) AOR, SOR, and JOR iterative methods for solving linear systems are discussed.

Block cyclic SOR for Markov chains with p-cyclic infinitesimal generator

We introduce a new application of p-cyclic iterations, for arbitrary p⩾2. The block SOR method for the computation of the steady state-distribution of finite Markov chains that possess p-cyclic infinitesimal generators is considered. It is shown that convergence, in a sense more general than the usual, may be obtained even if the SOR iteration violates the usual conditions for semiconvergence. Necessary and sufficient conditions for convergence in this extended sense are derived. They are then applied in the case where the pth power of the associated Jacobi matrix of the system to be solved possesses only nonnegative eigenvalues. Exact convergence intervals and the optimal ω-values are derived for this case. In addition to the “usual” optimal ω in the interval (1, p (p−1) ) , other ω-values that yield convergence in the extended sense are found to achieve the same optimal convergence rate. Numerical tests indicate that small perturbations of ω around the optimal value affect the convergence factor much less if these newly introduced optimal ω-values are used.

A historical overview of iterative methods

The object of this paper is to present a historical overview of the development of iterative methods for the solution of large sparse systems of linear equations. The emphasis is on methods which are applicable to linear systems arising in the numerical solution of partial differential equations. Aspects to be covered including the methods of L.F. Richardson and of Liebmann as well as relaxation methods used by Southwell and others; the SOR method and extensions such as block SOR methods, and p-cyclic matrices; Chebyshev polynomial methods; alternating direction implicit methods; the SSOR method; approximate matrix factorization methods including the strongly implicit method (SIP) and the incomplete Cholesky method (ICC); fast direct methods; conjugate gradient methods; adaptive methods for the automatic determination of iteration parameters; multigrid methods; methods for nonsymmetric systems; and iterative software. Future developments will be discussed with emphasis on the use of vector and parallel processors.

SOR methods for coupled elliptic partial differential equations

The biharmonic or Navier-Stokes problems in the form of a coupled pair of Dirichlet problems (J. Smith, SIAM J. Numer. Anal. 5, 323 (1968) are numerically solved by using a two parameter point SOR method. We emphasize the dependance of the convergence domain on the discrete boundary formulae. Optimization of this SOR method is heuristic but can be foreseen with a satisfactory precision. The optimal region is rather large and although using imprecisely optimal parameters, we can greatly improve the classical block SOR method (L. W. Ehrlich, SIAM J. Numer. Anal. 8, 278 (1971); L. W. Ehrlich and M. M. Gupta, SIAM J. Numer. Anal. 12, 773 (1975); M. M. Gupta and R. P. Manohar, J. Comput. Phys. 31, 265 (1979); M. Khalil, Thesis, Université Paul Sabatier, Toulouse 1983 (unpublished)).

A note on two block-SOR methods for sparse least squares problems

We compare two recently proposed block-SOR methods for the solution of large least squares problems with the conjugate gradient algorithm for solving the specially preconditioned normal equations. By proving that all three methods are based on the same Krylov sequence, we show that the conjugate gradient algorithm is generally preferable to the two SOR approaches.

Convergence of a direct-iterative method for large-scale least-squares problems

In 1975 Chen and Gentleman suggested a 3-block SOR method for solving least-squares problems, based on a partitioning scheme for the observation matrix A into A= A 1 A 2 where A 1 is square and nonsingular. In many cases A 1 is obvious from the nature of the problem. This combined direct-iterative method was discussed further and applied to angle adjustment problems in geodesy, where A 1 is easily formed and is large and sparse, by Plemmons in 1979. Recently, Niethammer, de Pillis, and Varga have rekindled interest in this method by correcting and extending the SOR convergence interval. The purpose of our paper is to discuss an alternative formulation of the problem leading to a 2-block SOR method. For this formulation it is shown that the resulting direct-iterative method always converges for sufficiently small SOR parameter, in contrast to the 3-block formulation. Formulas for the optimum SOR parameter and the resulting asymptotic convergence factor, based upon ‖ A 2 A -1 1‖ 2, are given. Furthermore, it is shown that this 2-cyclic block SOR method always gives better convergence results than the 3-cyclic one for the same amount of work per iteration. The direct part of the algorithm requires only a sparse-matrix factorization of A 1. Our purpose here is to establish theoretical convergence results, in line with the purpose of the recent paper by Niethammer, de Pillis, and Varga. Practical considerations of choosing A 1 in certain applications and of estimating the resulting ‖ A 2 A -1 1‖ 2 will be addressed elsewhere.

The Solution of Nonlinear Elliptic Dirichlet Problems on Rectangles by Almost Globally Convergent Interval Methods

The use of interval arithmetic offers the possibility to develop methods for the solution of systems of nonlinear equations that converge to the solution under relatively weak conditions, provided an initial inclusion is known. In the present paper we describe methods for discretizations of nonlinear elliptic equations, in particular for Dirichlet problems of the type $(au_x )_x + (bu_y )_y = f(x,y,u)$ where $a \equiv a(x)$ and $b = b(y)$ or $a \equiv a(x,y)$, $b \equiv b(x,y)$. The interval arithmetic enables us to combine a Newton-like interval method with the concept of fast direct solvers. The convergence to the solution can be ensured by auxiliary methods or by monotonicity arguments. We compare some of the possible variants with the generalized CG method and the nonlinear block SOR method.

Generalized iterative methods for semidefinite linear systems

We consider iterative methods for semidefinite systems Ax = b based on splittings A = B − C, where B is not necessarily nonsingular. Necessary and sufficient conditions for convergence are obtained. These are then used to obtain convergence results for block SOR, block SSOR, and block JOR methods for matrices with semidefinite block diagonal.

Coupled Harmonic Equations, SOR, and Chebyshev Acceleration

A coupled pair of harmonic equations is solved by the application of Chebyshev acceleration to the Jacobi, Gauss-Seidel, and related iterative methods, where the Jacobi iteration matrix has purely imaginary (or zero) eigenvalues. Comparison is made with a block SOR method used to solve the same problem.

Coupled harmonic equations, SOR, and Chebyshev acceleration

A coupled pair of harmonic equations is solved by the application of Chebyshev acceleration to the Jacobi, Gauss-Seidel, and related iterative methods, where the Jacobi iteration matrix has purely imaginary (or zero) eigenvalues. Comparison is made with a block SOR method used to solve the same problem.

Solving the Biharmonic Equation as Coupled Finite Difference Equations

A technique is proposed for solving the finite difference biharmonic equation as a coupled pair of harmonic difference equations. Essentially, the method is a general block SOR method with convergence rate $O(h^{{1 / 2}} )$ on a square, where h is mesh size.