A New SVM + NDA Model for Improved Classification and Recognition

Abstract

Support Vector Machine (SVM) is a powerful classification methodology where the Support Vectors (SVs) fully describe the decision surface by incorporating local information. On the other hand, Nonparametric Discriminant Analysis (NDA) is an improvement over the more general Linear Discriminant Analysis (LDA) where the normality assumption from LDA is relaxed. NDA is also based on detecting the dominant normal directions to the decision surface. This paper introduces a novel SVM + NDA model which combines these two methods. This can be viewed as an extension to the SVM by incorporating some partially global information about the data, especially, discriminatory information in the normal direction to the decision boundary. This can also be considered as an extension to the NDA where the support vectors improve the choice of κ-nearest neighbors (κ−NN's) on the decision boundary by incorporating local information. Since our model is an extension to both SVM and NDA, it can deal with heteroscedastic and non-normal data. It also avoids the small sample size problem. Moreover, this model can be reduced to the classical SVM model so that the existing SVM programs can be used for easy implementation. An extensive comparison of the SVM + NDA to the LDA, SVM, NDA and the combined SVM and LDA, performed on artificial and real data sets, has shown the advantages and superiority of our proposed model. In particular, the experiments on face recognition have clearly shown a significant improvement of SVM + NDA over the other methods, especially, SVM and NDA.