Iterative Kernel Principal Component for Large-Scale Data Set

Abstract

Abstract Kernel principal component analysis (KPCA) is a popular nonlinear feature extraction method that uses eigendecomposition techniques to extract the principal components in the feature space. Most of the existing approaches are not feasible for analyzing large-scale data sets because of extensive storage needs and computation costs. To overcome these disadvantages, an efficient iterative method for computing kernel principal components is proposed. First, the power iteration is used to compute the first eigenvalue and the corresponding eigenvector. Then Schur-Weilandt deflation is repeatedly applied to obtain other higher order eigenvectors. No computation and storage of the kernel matrix is involved in this procedure. Instead, each row of the kernel matrix is calculated sequentially through the iterations. Thus, the kernel principal components can be computed without relying on the traditional eigendecomposition. The space complexity of the proposed method is O(m), and the time complexity is also greatly reduced. We illustrate the effectiveness of our approach through a series of real data experiments.