Least squares regression principal component analysis: A supervised dimensionality reduction method

Abstract

AbstractDimensionality reduction is an important technique in surrogate modeling and machine learning. In this article, we propose a supervised dimensionality reduction method, “least squares regression principal component analysis” (LSR‐PCA), applicable to both classification and regression problems. To show the efficacy of this method, we present different examples in visualization, classification, and regression problems, comparing it with several state‐of‐the‐art dimensionality reduction methods. Finally, we present a kernel version of LSR‐PCA for problems where the inputs are correlated nonlinearly. The examples demonstrate that LSR‐PCA can be a competitive dimensionality reduction method.