A novel feature selection algorithm based on LVQ hypothesis margin

Abstract

Feature selection has been widely discussed as an important preprocessing step in machine learning and data mining. In this paper, a new feature selection evaluation criterion based on low-loss learning vector quantization (LVQ) classification is proposed. Based on the evaluation criterion, a feature selection algorithm that optimizes the hypothesis margin of LVQ classification through minimizing its loss function is presented. Some experiments that are compared with well-known SVM-RFE and Relief are carried out on 4 UCI data sets using Naive Bayes and RBF Network classifier. Experimental results show that new algorithm achieves similar or even higher performance than Relief on all training data and has better or comparable performance than SVM-RFE.