On the difference between low-rank and subspace approximation: improved model for multi-linear PLS regression

Abstract

While both Tucker3 and PARAFAC models can be viewed as latent variable models extending principal component analysis (PCA) to multi-way data, most fundamental properties of PCA do not extend to both models. This has practical importance, which will be explained in this paper. The fundamental difference between the PARAFAC and the Tucker3 model can be viewed as the difference between so-called low-rank and subspace approximation of the data. This insight is used to pose a modification of the multi-linear partial least squares regression (N-PLS) model. The modification is found by exploiting the basic properties of PLS and of multi-way models. Compared to the current prevalent implementation of N-PLS, the new model provides a more reasonable fit to the independent data and exactly the same predictions of the dependent variables. Thus, the reason for introducing this improved model is not to obtain better predictions, but rather the aim is to improve the secondary aspect of PLS: the modeling of the independent variables. The original version of N-PLS has some built-in problems that are easily circumvented with the modification suggested here. This is of importance, for example, in process monitoring, outlier detection and also, implicitly, for jackknifing of model parameters. Some examples are provided to illustrate some of these points.