Basis sets for multivariate regression

Abstract

Estimates of regression coefficients for a multivariate linear model have been the subject of considerable discussion in the literature. A purpose of this paper is to discuss biased estimators using common basis sets. Estimators of focus are least squares, principal component regression, partial least squares, ridge regression, generalized ridge regression, continuum regression, and cyclic subspace regression. Variations of these methods are also proposed. It is shown that it is not the common basis set used to span the calibration space or the number of vectors from the common basis set used to form respective calibration models that are important, i.e. a parsimony emphasis. Instead, it is suggested that the size and direction of the calibration subspace used to form the models is essential, i.e. a harmony consideration. The approach of the paper is based on representing estimated regression vectors as weighted sums of basis vectors.