Subspace Regularized Linear Discriminant Analysis for Small Sample Size Problems

Abstract

Linear discriminant analysis (LDA) can extract features that preserve class separability. For small sample size (SSS) problems, the number of data samples is smaller than the dimension of data space, and the within-class scatter matrix of data samples is singular. LDA cannot be directly applied to SSS problems, since LDA requires the within-class scatter matrix to be non-singular. Regularized linear discriminant analysis (RLDA) is a common way to deal with singularity problems. In this paper, a subspace RLDA (SRLDA) algorithm is presented for SSS problems, which is performed in a subspace containing the range space of the total scatter matrix. The use of different parameter values in the regularization of the within-class matrix connects SRLDA to several extensions of LDA including discriminative common vectors (DCV), complete LDA (CLDA) and pseudo-inverse LDA (PLDA). An efficient algorithm to perform SRLDA based on the QR decomposition is developed, which makes the algorithm feasible and efficient for SSS problems. Face recognition is a well-known SSS problem and extensive experiments on various datasets of face images are conducted to evaluate the proposed algorithm and compare SRLDA with other extended LDA algorithms. Experimental results show the proposed algorithm can fuse discriminative information in the range and the null space of the within-class scatter matrix to find optimal discriminant vectors.