Principal Component Analysis Versus Factor Analysis

Abstract

In this chapter, we refer to exploratory factor analysis simply as factor analysis and consider the principal component analysis formulated as reduced rank approximation as in Chap. 5. Principal component analysis (PCA) and factor analysis (FA) can be performed for identical data sets, with the purpose of dimension reduction. This reduction means that p observed variables, i.e., the p-dimensional scores, are reduced to lower-dimensional scores. The lower dimensions correspond to the m principal components in PCA and the m common factors in FA, with m < p. A major purpose of this chapter is to introduce mathematical facts that contrast PCA and FA solutions for an identical data set. The facts elucidate crucial differences between PCA and FA, which can suggest whether PCA or FA should be used for a particular data set.