Approximating minimum norm solutions of rank‐deficient least squares problems

Abstract

In this paper we consider the solution of linear least squares problems minx∥Ax - b∥22 where the matrix A ∈ Rm × n is rank deficient. Put p = min{m, n}, let σi, i = 1, 2,…, p, denote the singular values of A, and let ui and vi denote the corresponding left and right singular vectors. Then the minimum norm solution of the least squares problem has the form x* = ∫ri = 1(uTib/σi)vi, where r ≤ p is the rank of A. The Riley–Golub iteration, xk + 1 = arg minx{∥Ax - b∥22 + λ∥x − xk∥22} converges to the minimum norm solution if x0 is chosen equal to zero. The iteration is implemented so that it takes advantage of a bidiagonal decomposition of A. Thus modified, the iteration requires only O(p) flops (floating point operations). A further gain of using the bidiagonalization of A is that both the singular values σi and the scalar products uTib can be computed at marginal extra cost. Moreover, we determine the regularization parameter, λ, and the number of iterations, k, in a way that minimizes the difference x* − xk with respect to a certain norm. Explicit rules are derived for calculating these parameters. One advantage of our approach is that the numerical rank can be easily determined by using the singular values. Furthermore, by the iterative procedure, x* is approximated without computing the singular vectors of A. This gives a fast and reliable method for approximating minimum norm solutions of well-conditioned rank-deficient least squares problems. Numerical experiments illustrate the viability of our ideas, and demonstrate that the new method gives more accurate approximations than an approach based on a QR decomposition with column pivoting. © 1998 John Wiley & Sons, Ltd.