On Closeness Between Factor Analysis and Principal Component Analysis Under High-Dimensional Conditions

Abstract

This article studies the relationship between loadings from factor analysis (FA) and principal component analysis (PCA) when the number of variables p is large. Using the average squared canonical correlation between two matrices as a measure of closeness, results indicate that the average squared canonical correlation between the sample loading matrix from FA and that from PCA approaches 1 as p increases, while the ratio of p/N does not need to approach zero. Thus, the two methods still yield similar results with high-dimensional data. The Fisher-z transformed average canonical correlation between the two loading matrices and the logarithm of p is almost perfectly linearly related.