A Robust Feature Selection based on Bayes Theorem, Self Information and Symmetric Uncertainty for Improving the Performance of Naive Bayesian Classifier

Abstract

Feature selection (FS) is used to determine the minimal subset of features that are more efficient and could render high accuracy when compared with the whole set of features for the classifiers. Many FS algorithms have been proposed in the literature and it is found that the FS methods which incorporate both relevance and redundant analysis give high accuracy for the classifiers. This paper also provides an alternate novel hybrid-filter-based FS approach for enhancing the predictive accuracy of Naive Bayesian Classifier (NBC) by combining both relevance and redundant analysis. In the relevance analysis, the feature weights relevant to the target are obtained using Bayes Theorem (BT) and Self Information (SI) and the relevant features are selected from the weighted features using Sequential Forward Selection (SFS). But there is a possibility of redundant attributes from the relevant features picked from the relevance analysis and the proposed work removes it by using the information theoretic correlation measure called Symmetric Uncertainty (SU). The efficiency and effectiveness of the proposed algorithm are evaluated by comparing it with the existing methods on eight real-world data sets from the University of California at Irvine UCI machine-learning repository. From the experiments, it has been proved that the proposed work not only reduces the number of features but also enhances the accuracy of NBC.