Multiple nearest neighbor classifiers system based on feature perturbation by mutual information

Abstract

In this paper, a feature perturbation by mutual information is proposed and multiple nearest neighbor classifiers are combined according to this way. Multiple nearest neighbor classifiers system based on feature perturbation can improve the performance of single nearest neighbor classifier biased with the curse of dimensionality. However, there are two problems in multiple nearest neighbor classifiers system based on feature perturbation: (i) how to determine the number of component classifiers, and (ii) how to select features for each component classifier. In this paper, the proposed method by mutual information is able to solve the two problems. (i) The number of component classifiers is set to be the number of classes, and (ii) the selected features for each component classifier are automatically determined by mutual information. In order to evaluate the effectiveness of the proposed method, three UCI datasets are selected. And the proposed method is compared with: (i) NNC, (ii) NNC after feature selection by mutual information, (iii) multiple nearest neighbor classifiers system based on feature perturbation by attribute bagging. The experimental results show that multiple classifiers system is superior to single classifier. And multiple nearest neighbor classifiers system based on feature perturbation by mutual information is better than any other combination methods. In addition, the number of component classifiers in the proposed method is less than any other methods.