Online discriminative component analysis feature extraction from stream data with domain knowledge

Abstract

In this paper, we introduce an incremental version of recently proposed constrained Linear Discriminant Analysis (LDA). In addition of application in constrained LDA problems, our algorithm which we call Online Discriminative Component Analysis (ODCA) is usable in standard incremental LDA problems. ODCA incrementally computes the solution of LDA with the time complexity lower than most incremental algorithm for LDA while keeps the accuracy of final result as close as possible to offline version. This is done using a special formulation for the scatter matrix updating along with Eigen-space calculation. By exploiting such formulation, the proposed algorithm made capable of updating the solution where a data point added or removed from the problem. It is also usable in problems where its data points have concept drift property. To show efficiency of proposed algorithm, its speed is compared to other existing incremental algorithms as order of complexity. In addition, the classification accuracy of our approach is experimentally compared to other algorithms.