Individualized learning for improving kernel Fisher discriminant analysis

Abstract

Kernel Fisher discriminant analysis (KFDA) is a very popular learning method for the purpose of classification. In this paper, we propose a novel learning algorithm to improve KFDA and make it very suitable for dealing with the large-scale and high-dimensional data sets. The proposed algorithm is termed individualized KFDA (IKFDA). IKFDA is based on individualized learning, i.e., a strategy to learn and classify the individual test samples one by one. Our approach seeks to find the appropriate training subset, referred to as learning area, for each individual test sample, and then employ the learning area to construct the KFDA model for the test sample. For each individual test sample, IKFDA exploits some types of similarity measures to determine a learning area that consists of the training samples that are most similar to the test sample. Compared with the traditional learning algorithms that often exploit the whole training set to construct the learning models without considering the distribution property of the test samples, IKFDA can adaptively learn the individual test samples. It is a powerful tool to deal with the real-world complicated data sets that are often very large-scale and high-dimensional, and are usually drawn from the different distributions. Extensive experiments show that the proposed algorithm can obtain good classification results.