On convergence rates of Kaczmarz-type methods with different selection rules of working rows

Abstract

The Kaczmarz method is a classical while effective iteration method for solving very large-scale consistent systems of linear equations, and the randomized Kaczmarz method is an important and valuable variant of the Kaczmarz method. By theoretically analyzing and numerically experimenting several criteria typically adopted in the non-randomized and the randomized Kaczmarz method for selecting the working row, we derive sharper upper bounds for the convergence rates of some of the correspondingly induced Kaczmarz-type methods including those with respect to the maximal residual, maximal distance, and distance selection rules of the working row, and, for this whole suite of iteration methods consisting of the Kaczmarz methods with respect to the uniform, non-uniform, residual, distance, maximal residual, and maximal distance selection rules of the working row, we reveal their comparable relationships in terms of both mean-squared distance and mean-squared error, and show their computational effectiveness and numerical robustness based upon implementing a large number of test examples. Here the mean-squared distance is defined as the mean-value of the squared Euclidean norm of the current update increment of the iteration, and the mean-squared error is defined as the mean-value of the squared Euclidean norm of the current error that is the difference between the current iterate and the true solution of the target linear system.