Principal Component Analysis

Abstract

Principal component analysis (PCA) tries to find an orthogonal linear projection that projects the data into a novel coordinate system, in which the greatest variance by some scalar projection of the data lies on the first coordinate (called the first principal component), the second greatest variance on the second coordinate, and so on. PCA is widely used as a tool for dimensionality reduction and data visualization. In this chapter, we will first give a brief review of the theoretical derivation of PCA from two commonly used definitions: maximizing variance and minimizing residuals. And then, we show how PCA arises naturally as the maximum likelihood solution to a particular form of a linear-Gaussian latent variable model, which is called probabilistic principal component analysis. Finally, through case studies, we show how PCA can be used in different applications. The first simple case shows how to use PCA in dimension reduction and data visualization. The second case shows PCA-based feature extraction in fault detection for process monitoring.