Stability of Two Direct Methods for Bidiagonalization and Partial Least Squares

Abstract

The partial least squares (PLS) method computes a sequence of approximate solutions $x_k \in {\cal K}_k(A^TA,A^Tb)$, $k = 1,2,\ldots\,$, to the least squares problem $\min_x\|Ax - b\|_2$. If carried out to completion, the method always terminates with the pseudoinverse solution $x^\dagger = A^\dagger b$. Two direct PLS algorithms are analyzed. The first uses the Golub--Kahan Householder algorithm for reducing $A$ to upper bidiagonal form. The second is the NIPALS PLS algorithm, due to Wold et al., which is based on rank-reducing orthogonal projections. The Householder algorithm is known to be mixed forward-backward stable. Numerical results are given, that support the conjecture that the NIPALS PLS algorithm shares this stability property. We draw attention to a flaw in some descriptions and implementations of this algorithm, related to a similar problem in Gram--Schmidt orthogonalization, that spoils its otherwise excellent stability. For large-scale sparse or structured problems, the iterative algorithm...