Efficient approximate solution of sparse linear systems

Abstract

We consider the problem of approximate solution x ̄ of of a linear system Ax = b over the reals, such that ∥A x ̄ − b∥ ≤ ε ∥b∥ , for a given ε, 0 < ε < 1. This is one of the most fundamental of all computational problems. Let κ( A) = ∥ A∥ ∥ A −1∥ be the condition number of the n × n input matrix A. Sparse, Diagonally Dominant (DD) linear systems appear very frequently in the solution of linear systems associated with PDEs and stochastic systems, and generally have polynomial condition number. While there is a vast literature on methods for approximate solution of sparse DD linear systems, most of the results are empirical, and to date there are no known proven linear bounds on the complexity of this problem. Using iterative algorithms, and building on the work of Vaidya [1] and Gremban et al. [2–4] we provide the best known sequential work bounds for the solution of a number of major classes of DD sparse linear systems. Let π = log( κ(A) ε ) . The sparsity graph of A is a graph whose nodes are the indices and whose edges represent pairs of indices of A with nonzero entries. The following results hold for a DD matrix A with nonzero off-diagonal entries of bounded magnitude: 1. (1) if A has a sparsity graph which is a regular d-dimensional grid for constant d, then our work is O( nπ 2), 2. (2) if A is a stochastic matrix with fixed s( n)-separable graph as its sparsity graph, then our work is O(( n + s( n) 2) π). The following results hold for a DD matrix A with entries of unbounded magnitude: 1. (3) if A is sparse (i.e., O( n) nonzeros), our work is less than O( n( π + log n)) 1.5, 2. (4) if A has a sparsity graph in a family of graphs with constant size forbidden graph minors (e.g., planar graphs), then our work is bounded by O( n( π + log n) 1+ o(1) ) in the case log n = o(log π) and O( n( π + log n)) 1+ o(1) in the case log π = o(log n). We use approximate preconditioned iterations to compute a sequence of iterative improvements to the preconditioned linear system. For class (1) of matrices (and class (2) with s( n) = O(√ n)), we construct in n) work preconditioners, which reduce the condition number of the resulting preconditioned linear system condition number to O(1); and our resulting total work bounds to approximately solve the linear systems are O( n), if π = O(1). For class (4), we are within a small increasing factor of these bounds.