Three Versions of Sequential Projection Algorithm and Condition Numbers of Large Random Matrices

Abstract

In this paper, we discuss three modifications of Kaczmarz's algorithm (sequential projection algorithm) for solving linear systems. It is well known that for square or underdetermined systems of full rank, the usual (classic) Kaczmarz's algorithm converges to a solution, while in the case of overdetermined systems iterations converge to a sequence of vertices of a limit polygon. Algorithm looping in overdetermined case can be eliminated by introducing a relaxation factor into the projection process. In this version, the limit point will be a weighted pseudo-solution with weights equal to the reciprocal of the norms of the rows of the system matrix (geometric pseudo-solution). This modification of the algorithm has been discussed many times in the literature. On the other hand, there are two rather lesser-known versions of the sequential projection algorithm in which the iterations converges to the usual pseudo-solution: "double" and "columnar" versions. We describe these modifications and explain the relation between "double" and "columnar" versions. In connection with numerical experiments, the behavior of the condition numbers of random matrices with increasing sizes is also discussed.