A doubly stochastic block Gauss–Seidel algorithm for solving linear equations

Abstract

We propose a doubly stochastic block Gauss–Seidel algorithm for solving linear systems of equations. By varying the row partition parameter and the column partition parameter for the coefficient matrix, we recover the Landweber algorithm, the randomized Kaczmarz algorithm, the randomized coordinate descent algorithm, and the doubly stochastic Gauss–Seidel algorithm. For arbitrary (consistent or inconsistent, full column rank or rank-deficient) linear systems, we prove the exponential convergence of the norm of the expected error via exact formulas. We also prove the exponential convergence of the expected norm of the error for consistent linear systems, and the exponential convergence of the expected norm of the residual for arbitrary linear systems. Numerical experiments for linear systems with synthetic and real-world coefficient matrices are given to demonstrate the efficiency of our algorithm.