Nonlinear discriminant analysis based on vanishing component analysis

Abstract

Most kernel-based nonlinear discriminant analysis methods need to compute the kernel distance between test samples and all of the training samples, but this approach consumes large volumes of time and memory, and it may be impractical when there is a large number of training samples. In this study, we propose a vanishing component analysis (VCA) based nonlinear discriminant analysis (VNDA) method. First, VNDA learns nonlinear mapping functions explicitly using the modified VCA method, before employing these functions to map the input feature onto a high-dimensional polynomial feature space, where the linear discriminant analysis (LDA) method is then applied. We prove that principal components analysis plus LDA is a special case of VNDA and that the set of mapping functions learned by VNDA is the best solution to the ratio trace problem in the degree bounded polynomial feature space. Unlike kernel-based methods, VNDA only stores these mapping functions instead of all the training samples in the test step. Experimental results obtained based on four simulated data sets and 15 real data sets demonstrate that the proposed method yields highly competitive test recognition results compared to the state-of-the-art methods, while consuming less memory and time resources.