Iterative Solution Of Linear Systems Research Articles

Algorithmic bombardment for the iterative solution of linear systems: A poly-iterative approach

Many algorithms employing short recurrences have been developed for iteratively solving linear systems. Yet when the matrix is nonsymmetric or indefinite, or both, it is difficult to predict which method will perform best, or indeed, converge at all. Attempts have been made to classify the matrix properties for which a particular method will yield a satisfactory solution, but “luck” still plays large role. This report describes the implementation of a poly-iterative solver. Here we apply three algorithms simultaneously to the system, in the hope that at least one will converge to the solution. While this approach has merit in a sequential computing environment, it is even more valuable in a parallel environment. By combining global communications, the cost of three methods can be reduced to that of a single method.

Extension and Completion of Wynn's Theory on Convergence of Columns of the Epsilon Table

Let {Sn}∞n=0be such thatSn∽S+∑∞j=1ajλnjasn→∞, with 1>|λ1|>|λ2|>…, such that limj→∞λj=0. A well-known result by Wynn states that when the Shanks transformation or its equivalentε-algorithm is applied to {Sn}∞n=0, thenε(n)2k−S∽ak+1[∏ki=1(λk+1−λi)/(1−λi)]2λnk+1asn→∞. In the present work we extend this result (i) by allowing some of theλjto have the same modulus and (ii) by replacing the constantsajby some polynomialsPj(n) inn. Sequences {Sn}∞n=0with these characteristics arise frequently, e.g., in fixed point iterative solution of linear systems and in trapezoidal rule approximation of finite range integrals with logarithmic endpoint singularities and their multidimensional analogues. The results of this work are obtained by exploiting the connection between the Shanks transformation and Padé approximants and by using some recent results of the author on Padé approximants for meromorphic functions.

Algorithm-based error-detection schemes for iterative solution of partial differential equations

Algorithm-based fault tolerance is an inexpensive method of achieving fault tolerance without requiring any hardware modifications. For numerical applications involving the iterative solution of linear systems arising from discretization of various PDEs, there exist almost no fault-tolerant algorithms in the literature. We describe an error-detecting version of a parallel algorithm for iteratively solving the Laplace equation over a rectangular grid. This error-detecting algorithm is based on the popular successive overrelaxation scheme with red-black ordering. We use the Laplace equation merely as a vehicle for discussion; we show how to modify the algorithm to devise error-detecting iterative schemes for solving linear systems arising from discretizations of other PDEs, such as the Poisson equation and a variant of the Laplace equation with a mixed derivative term. We also discuss a modification of the basic scheme to handle situations where the underlying solution domain is not rectangular. We then discuss a somewhat different error-detecting algorithm for iterative solution of PDEs which can be expected to yield better error coverage. We also present a new way of dealing with the roundoff errors which complicate the check phase of algorithm-based schemes. Our approach is based on error analysis incorporating some simplifications and gives high fault coverage and no false alarms for a large variety of data sets. We report experimental results on the error coverage and performance overhead of our algorithm-based error-detection schemes on an Intel iPSC/2 hypercube multiprocessor.

ICIAM/GAMM 95 Numerical Analysis, Scientific computing Computer ScienceICIAM/GAMM 95 Numerical Analysis, Scientific computing Computer Science

ZAMM - Journal of Applied Mathematics and Mechanics / Zeitschrift für Angewandte Mathematik und MechanikVolume 76, Issue S1 p. 311-362 Article ICIAM/GAMM 95 Numerical Analysis, Scientific computing Computer ScienceICIAM/GAMM 95 Numerical Analysis, Scientific computing Computer Science First published: 1996 https://doi.org/10.1002/zamm.19960761109AboutPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinked InRedditWechat Volume76, IssueS1Supplement: ICIAM/GAMM 95 Numerical Analysis, Scientific Computing, Computer Science. Hamburg July 3‐7, 19951996Pages 311-362 RelatedInformation

The Design of Data-Structure-Neutral Libraries for the Iterative Solution of Sparse Linear Systems

Over the past few years several proposals have been made for the standardization of sparse matrix storage formats in order to allow for the development of portable matrix libraries for the iterative solution of linear systems. We believe that this is the wrong approach. Rather than define one standard (or a small number of standards) for matrix storage, the community should define an interface (i.e., the calling sequences) for the functions that act on the data. In addition, we cannot ignore the interface to the vector operations because, in many applications, vectors may not be stored as consecutive elements in memory. With the acceptance of shared memory, distributed memory, and cluster memory parallel machines, the flexibility of the distribution of the elements of vectors is also extremely important. This issue is ignored in most proposed standards. In this article we demonstrate how such libraries may be written using data encapsulation techniques.

The convergence rate of the Chebyshev sim under a perturbation of a complex line-segment spectrum

The Chebyshev semiiterative method ( chsim) is probably the best known and most often used method for the iterative solution of linear system x = Tx + c, where the spectrum of T is located in a complex line segment [α, β] excluding 1. The asymptotic convergence factor (ACF) of the chsim, under a perturbation of [α, β], is considered. Several formulae for the approximation to the ACFs, up to the second order of a perturbation, are derived. This generalizes the results about the sensitivity of the asymptotic rate of convergence to the estimated eigenvalues by Hageman and Young in the case that both α and β are real. Two numerical examples are given to illustrate the theoretical results.

Iterative solution of linear systems arising from the Nyström method for the double‐layer potential equation over curves with corners

AbstractIn this paper we consider a quadrature method for the solution of the double‐layer potential equation corresponding to Laplace's equation in a polygonal domain. We prove the stability for our method in case of special triangulations over the boundary of the polygon. For the solution of the corresponding system of linear equations, we consider a two‐grid iteration and establish the rates of convergence and complexity. Finally, we discuss the effect of mesh refinement near the corners of the polygon.

Iterative Solution of Linear Systems Arising from the Boundary Integral Method

In this paper the behavior of some standard two-grid iterative schemes for the solution of linear systems arising from discretizations of second-kind boundary integral equations of potential theory is studied. When the boundary of the domain has corners, these schemes converge slowly, and in some cases even diverge. New iterative schemes are derived which always converge, and estimates for the accuracy of the solution to which they converge are given. Extensive numerical experiments are reported and discussed.

The modified preconditioned jacobi method for iterative solution of linear systems of equations

In this paper the authors develop explicit forms of preconditioning iterative methods which are suitable for parallel computers. The convergence of the method is analysed and the existence of an optimum preconditioning parameter established which is confirmed from the numerical experiments carried out.

Iterative solution of linear systems

Recent advances in the field of iterative methods for solving large linear systems are reviewed. The main focus is on developments in the area of conjugate gradient-type algorithms and Krylov subspace methods for nonHermitian matrices.

Hydrologic Applications of the Connection Machine CM‐2

Massively parallel computers will play an increasingly dominant role in hydrological computing. One such computer is the Connection Machine model CM‐2, a single‐instruction stream, multiple‐data stream computer with up to 65,536 processors, as much as 8 gigabytes (Gbyte) of random access memory distributed among the processors, and a FORTRAN compiler based on the proposed FORTRAN‐90 standard. One‐, two‐, and three‐dimensional examples from hydrology are used in this paper to present a tutorial on programming for the CM‐2. The problem of saturated, steady flow in a randomly heterogeneous three‐dimensional porous medium is explored here in some detail. A diagonally preconditioned conjugate gradient (DPCG( iterative solver is applied to this problem for up to 1283 nodes. Less than l min of CM‐2 time is required to reduce the error by a factor of 10−6 for a 128 × 128 × 128 grid with heterogeneous hydraulic conductivity. Measured CPU times for the DPCG method are significantly smaller than those reported in the literature for a polynomial PCG solver applied to the same domain with different boundary conditions and executed on a Cray X‐MP/48 and an Alliant FX/8. The measured performance is also much greater than that reported in the literature for a strongly implicit procedure solver applied to a similar problem on a Cray 2. The need for continued development of massively parallel algorithms, including effective iterative solution of linear systems of equations and problems with irregular domains, is indicated.

On the theory of equivalent operators and application to the numerical solution of uniformly elliptic partial differential equations

This work is motivated by the preconditioned iterative solution of linear systems that arise from the discretization of uniformly elliptic partial differential equations. Iterative methods with bounds independent of the discretization are possible only if the preconditioning strategy is based upon equivalent operators. The operators A, B: W → V are said to be V norm equivalent if ∥Au∥ v ∥Bu∥ v is bounded above and below by positive constants for u ϵ D, where D is “sufficiently dense.” If A is V norm equivalent to B, then for certain discretization strategies one can use B to construct a preconditioned iterative scheme for the approximate solution of the problem Au = F. The iteration will require an amount of work that is at most a constant times the work required to approximately solve the problem B u ̂ = \\ ̂ tf to reduce the V norm of the error by a fixed factor. This paper develops the theory of equivalent operators on Hubert spaces. Then, the theory is applied to uniformly elliptic operators. Both the strong and weak forms are considered. Finally, finite element and finite difference discretizations are examined.

Use of preconditioned bi-conjugate gradient method in modeling two-phase fluid flow in porous media

We have found the preconditioned bi-conjugate gradient method superior to the standard conjugate gradient method for iterative solution of linear systems occurring in solving the finite difference form of partial differential equations describing multi-dimensional two-phase flow in porous media. The matrices involved in these systems are typically non-symmetric and not positive definite. Fully vectorizable preconditioning was effective in our solutions using the bi-conjugate gradient method, whereas more extensive, non-vectorizable preconditioning is often necessary when using the conjugate gradient method.

A survey of the iterative methods for the solution of linear systems by extrapolation, relaxation and other techniques

For the iterative solution of linear systems of algebraic equations Ax = b(1), with A ∈ C n,n, x, b ∈ C n and det( A) ≠ 0, numerous methods exist. Although a classification of them seems not to be possible one may note that the first step for the construction of an iterative method usually begins with a splitting of A in (1). Thus A is written as A = M − N, where det( M) ≠ 0 and M is easily inverted, so that (1) is equivalent to x = Tx + c (2), T ≔ M −1 N, c ≔ M −1. The discussion will be restricted to the so-called linear (stationary) iterative methods, although some nonstationary ones will be mentioned, and emphasis will be given to those which fall into the categories of extrapolation, relaxation and similar ones. For this it will be assumed that the spectrum σ( T) of T in (2) is contained in a well-defined compact region R, whose complement with respect to the complex plane is simply connected, and 1 ∉ R. Under these assumptions and for each specific class of methods described each time an attempt will be made to present the ‘optimum’ one. The optimum is the one out of the class of methods for which the sequence of vectors yielded converges asymptotically to the unique solution of (1) as fast as possible.

On the application of orthogonal polynomials to the iterative solution of linear systems of equations with indefinite or non-Hermitian matrices

Let Ax= b be a large linear system of equations, and let the eigenvalues of the matrix A lie in one or several known simply connected regions S j , j=1(1) l, in the complex plane. We consider the iterative solution of such linear systems by methods that make use of polynomials { p n ( z)} ∞ n=0 orthogonal on the boundary of ∪ l j=1 S j . We show that for boundary curves such that the p n ( z) satisfy a three-term recurrence relation, iterative methods based on this recurrence relation yield an optimal asymptotic rate of convergence. For boundary curves for which the p n ( z) do not satisfy a three-term recurrence relation, we show that n-cyclic Richardson iteration methods with relaxation parameters chosen as the reciprocal roots of p n ( z) give nearly asymptotic optimal convergence for n sufficiently large. Our analysis suggests that in many cases the iterative methods should be based on residual polynomials p n(z) p n(0) , instead of on the corresponding kernel polynomials q z= ∑ n k=0 p k(0) p k(z) ∑ n k=0|p k(0)| 2 . Numerical examples are presented.

VLSI implementation of iterative methods for the solution of linear systems

Area-time upper bounds for the iterative solution of linear systems and for the inversion of matrices in VLSI models are obtained and compared to the known upper bounds obtained by direct methods. Error analysis gives the number of iterations for which an acceptable error bound is obtained in the result. The use of the Jacobi method to solve partial differential equations is discussed.

Generation of permutations by adjacent transposition

1. RICHARD S. VARGA, Matrix Analysis, Prentice-Hall, Inc., 1962. 2. E. G. D'YAKONOV, On a Method of Solving the Poisson Equation, Dokl. Akad. Nauk SSSR 143 (1962), 21-24, the same paper appears in English in Soviet Math.-Doklady 3, 1962, p. 320-323. 3. R. B. KELLOGG, Another Alternating-Direction-Implicit Method, to appear in J. Soc. Ind. Appi. Math. 4. A. M. OsTRowshI Iterative Solution of Linear Systems of Functional Equations, J. Math. Anal. Apple , 2, 1961, p. 351-369. 5. L. COLLATZ, Fehlerabschatzung fur das Iterationsverfahren zur Aufl6sung linearer Gleichungssysteme, Z. Angew. Math. Mech., 22, 1942, p. 357-361. 6. DAVID G. FEINGOLD, & R. S. VARGA, Block diagonally dominant matrices and generalizations of the Gerschgorin circle theorem, to appear in the Pacific J. Math. 7. J. DOUGLAS, JR., and H. H. RACHFORD, JR., On the numerical solution of heat conduction problems in two or three space variables, Trans. Amer. Math. Soc., 82, 1956, p. 421-439. 8. J. DOUGLAS, JR., Alternating direction methods for three space variables, Numer. Math., 4, 1962, p. 41-63.

Iterative solution of linear systems of functional equations

Iterative solution of linear systems of functional equations