DROP-D: Dimension reduction by orthogonal projection for discrimination

Abstract

The objective of this paper is two-fold. First, some theoretical aspects of dimension reduction in the context of supervised classification or discrimination are given. The emphasis is put on the different subspaces that can be defined in this context and what information is contained in each of them. Then, based on these theoretical aspects, we propose a novel method for supervised dimension reduction that is dedicated to discrimination purposes. The method, called Dimension Reduction by Orthogonal Projection for Discrimination (DROP-D) is particularly well suited to the high dimensionality and high intercorrelation of spectral variables. As with Fisher discriminant analysis, DROP-D aims at finding a lower dimensional subspace in which the classes are well separated. To do so, DROP-D cleans the observation matrix of variability sources that do not help with the classification task. For this purpose, the matrix is projected orthogonally to the within-class axes which prevent a good class separability. In cases where some between-class axes are collinear with the within-class axes, DROP-D can preserve these axes in order not to destroy the class separability. DROP-D discriminant axes are orthogonal to one another and thus offer a simplified interpretability. The main advantage of DROP-D is that because it is based on removing unnecessary information, there is no need of a validation set to tune the model parameters. In contrast to modeling techniques, DROP-D thus cannot find class separability when there is none. In terms of results, DROP-D offers similar performances to the usual linear classification methods.