Vector $l_0$ Sparse Variable PCA

Abstract

Principal component analysis (PCA) achieves dimension reduction by replacing the original measured variables with a smaller set of derived variables called the principal components. Sparse PCA improves this with sparsity. There are two kinds of sparse PCA; sparse loading PCA (slPCA) which keeps all the measured variables but zeroes out some of their loadings; and sparse variable PCA (svPCA) which removes some measured variables completely by simultaneously zeroing out all their loadings. Because it zeroes out some measured variables completely svPCA is capable of huge additional dimension reduction beyond PCA; while slPCA keeps all measured variables and does not have this capability. Here we consider a vector l0 penalized likelihood approach to svPCA and develop a penalized expectation-maximization (pEM) algorithm which remarkably, in an l0 setting, leads to a closed form M-step and we provide a convergence analysis.