Class mean vector component and discriminant analysis

Abstract

• We propose a component analysis method for kernel-based subspace learning. • We give extensive analysis of the method and its connection with KPCA and KDA. • A discriminant analysis method is proposed based on properties of KDA projections. The kernel matrix used in kernel methods encodes all the information required for solving complex nonlinear problems defined on data representations in the input space using simple, but implicitly defined, solutions. Spectral analysis on the kernel matrix defines an explicit nonlinear mapping of the input data representations to a subspace of the kernel space, which can be used for directly applying linear methods. However, the selection of the kernel subspace is crucial for the performance of the proceeding processing steps. We propose a new optimization criterion , leading to a new component analysis method for kernel-based dimensionality reduction that optimally preserves the pair-wise distances of the class means in the feature space. This leads to efficient kernel subspace learning, which is crucial for kernel-based machine learning solutions. We provide extensive analysis on the connections and differences between the proposed criterion and the criteria used in kernel Principal Component Analysis, kernel Entropy Component Analysis and Kernel Discriminant Analysis, leading to a discriminant analysis version of the proposed method. Our theoretical analysis also provides more insights on the properties of the feature spaces obtained by applying these methods. Results on a variety of visual classification problems illustrate the properties of the proposed methods.