Optimal discriminant plane for a small number of samples and design method of classifier on the plane

Abstract

In a previous work (Zi-Quan Hong and Jing-Yu Yang, Minimum distance classifier on the optimal discriminant plane), we suggested and derived a method for constructing classifier on the optimal discriminant plane using minimum distance criteria. In this paper, the problem of solving optimal discriminant plane for a small number of samples is discussed, which is based on the above paper by the same authors. In the case of a small number of samples, generalized eigenequation AX = λBX established for a large number of samples usually has no solution because within-class scatter matrix is singular. To obtain the solution of the generalized eigenequation, a new method is suggested, in which the Singular Value Perturbation is added to the within-class scatter matrix such that the matrix becomes a nonsingular matrix. Therefore, the generalized eigenequation can be solved using existing algorithms. We proved that the generalized eigenequations on the optimal discriminant plane are stable in respect of eigenvalues and the generalized eigenvectors are indeed the optimal discriminant directions, if the perturbation is subject to some certain conditions. The experimental results have shown that our method works well and the minimum distance classifier on the optimal discriminant plane can be constructed with high performance even in the case of a small number of samples.