An Improvement of Incremental Recursive Fisher Linear Discriminant for Online Feature Extraction

Abstract

This paper proposes a new online feature extraction method called Incremental Recursive Fisher Linear Discriminant (IRFLD) whose batch learning algorithm called RFLD has been proposed by Xiang et al. In the conventional Linear Discriminant Analysis (LDA), the number of discriminant vectors is limited to the number of classes minus one due to the rank of the between-class covariance matrix. However, RFLD and the proposed IRFLD can break this limit; that is, an arbitrary number of discriminant vectors can be obtained. In the proposed IRFLD, the Pang et al.'s Incremental Linear Discriminant Analysis (ILDA) is extended such that effective discriminant vectors are recursively searched for the complementary space of a conventional discriminant subspace. In addition, to estimate a suitable number of effective discriminant vectors, the classification accuracy is evaluated with a cross-validation method in an online manner. For this purpose, validation data are obtained by performing the k-means clustering against incoming training data and previous validation data. The performance of IRFLD is evaluated for 16 benchmark data sets. The experimental results show that the final classification accuracies of IRFLD are always better than those of ILDA. We also reveal that this performance improvement is attained by adding discriminant vectors in a complementary LDA subspace.