An unified EM algorithm for PCA and KPCA

Abstract

In this note, from another point of view and in a more general situation, we formulate an EM algorithm for finding the leading eigen-system of any positive semi-definite matrix in a very simple derivation. The proposed EM approach can directly compute not only the eigen-system of sample covariance matrix in data space but also that of kernel matrix. Thus, the proposed algorithm provides an unified framework for EM-based principal component analysis (PCA) and kernel PCA (KPCA). Particularly, when it is applied to KPCA, it is a dual form of the commonly used constrained EM algorithm for performing KPCA. And thus it is a beneficial complementarity or dual description of the constrained EM method.