Random error bias in principal component analysis. Part I. derivation of theoretical predictions

Abstract

Principal component analysis (PCA) or singular value decomposition (SVD) are multivariate techniques that are often used to compress large data matrices to a relevant size. Subsequent data analysis then proceeds with the model representation of the data. In this first paper expressions are derived for the prediction of the bias in the eigenvalues of PCA and singular values of SVD that results from random measurement errors in the data. Theoretical expressions for the prediction of this “random error bias” have been given in the statistics literature. These results are, however, restricted to the case that only one principal component (PC) is significant. The first objective of this paper is to extend these results to an arbitrary number of significant PCs. For the generalization Malinowski's error functions are used. A signal-to-noise ratio is defined that describes the error situation for each individual PC. This definition enhances the interpretability of the derived expressions. The adequacy of the derived expressions is tested by a limited Monte Carlo study. This finally leads to the second objective of this paper. Simulation results are always restricted to the class of data that is well represented in the study. Thus rather than giving extensive simulation results it is outlined how the validation and evaluation of theoretical predictions can proceed for a specific application in practice.