Partial least-squares vs. Lanczos bidiagonalization—I: analysis of a projection method for multiple regression

Abstract

Multiple linear regression is considered and the partial least-squares method (PLS) for computing a projection onto a lower-dimensional subspace is analyzed. The equivalence of PLS to Lanczos bidiagonalization is a basic part of the analysis. Singular value analysis, Krylov subspaces, and shrinkage factors are used to explain why, in many cases, PLS gives a faster reduction of the residual than standard principal components regression. It is also shown why in some cases the dimension of the subspace, given by PLS, is not as small as desired.