FPCANet: Fisher discrimination for Principal Component Analysis Network

Abstract

With the development of Deep Learning (DL) in recent years, integrating traditional machine learning methods with DL has received a lot of attention. One of such representative work is the Principal Component Analysis Network (PCANet), which adopts Principal Component Analysis (PCA) to learn convolutional kernels (or filters) for image classification. Nevertheless, PCANet does not use the discriminative information during learning filters. In this paper, based on PCA in the PCANet, we propose a new model called Fisher PCA (FPCA) which combines Fisher Linear Discriminant Analysis (LDA) with PCA. To facilitate the practical calculation, a approximate model of FPCA is given by introducing a intermediate variable. Theoretically, we analyze the relationship between the original FPCA model and its approximate model, and give a convergence analysis of the approximate model. Additionally, stacking the approximate model of FPCA, we also construct a deep network named FPCA Network (FPCANet). Extensive experiments are conducted to compare FPCANet with other state-of-the-art models for classification problems. The results show that the proposed FPCANet can learn features with more discriminative information, and thus demonstrating its competitive performances on classification tasks.