Fast Fisher discriminant analysis with randomized algorithms

Abstract

Fisher discriminant analysis is a classical method for classification and dimension reduction jointly. Regularized FDA (RFDA) and kernel FAD (KFDA) are two important variants. However, RFDA will get stuck in computational burden due to either the high dimension of data or the big number of data and KFDA has similar computational burden due to kernel operations. We propose fast FDA algorithms based on random projection and random feature map to accelerate FDA and kernel FDA. We give theoretical guarantee that the fast FDA algorithms using random projection have good generalization ability in comparison with the conventional regularized FDA. We also give a theoretical guarantee that the pseudoinverse FDA based on random feature map can share similar generalization ability with the conventional kernel FDA. Experimental results further validate that our methods are powerful.