The Smallest Singular Value Anomaly and the Condition Number Anomaly

Abstract

Let A be an arbitrary matrix in which the number of rows, m, is considerably larger than the number of columns, n. Let the submatrix Ai,i=1,…,m, be composed of the first i rows of A. Let βi denote the smallest singular value of Ai, and let ki denote the condition number of Ai. In this paper, we examine the behavior of the sequences β1,…,βm, and k1,…,km. The behavior of the smallest singular values sequence is somewhat surprising. The first part of this sequence, β1,…,βn, is descending, while the second part, βn,…,βm, is ascending. This phenomenon is called “the smallest singular value anomaly”. The sequence of the condition numbers has a different character. The first part of this sequence, k1,…,kn, always ascends toward kn, which can be very large. The condition number anomaly occurs when the second part, kn,…,km, descends toward a value of km, which is considerably smaller than kn. It is shown that this is likely to happen whenever the rows of A satisfy two conditions: all the rows are about the same size, and the directions of the rows scatter in some random way. These conditions hold in a wide range of random matrices, as well as other types of matrices. The practical importance of these phenomena lies in the use of iterative methods for solving large linear systems, since several iterative solvers have the property that a large condition number results in a slow rate of convergence, while a small condition number yields fast convergence. Consequently, a condition number anomaly leads to a similar anomaly in the number of iterations. The paper ends with numerical experiments that illustrate the above phenomena.