Applying a Mutual Information Theory Based Feature Selection Method to a Classifier

Abstract

OBJECTIVE: The purpose of this study was to explore the usability of a feature selection method based on the mutual information theory to increase predictive performance of a classifier in data mining. METHODS: The HIV Cost and Services Utilization Study(HCSUS) dataset was used to apply the feature selection method to a classifier. Its contribution to increasing the predictive performance of the classifier was evaluated by comparing the Naive Bayes(NB) and the Logistic Regression(LG) models using different variables. The infrequent office visit representing limited health service utilization was selected as an outcome variable. HUGIN Researcher(TM) 6.3 was used to train and test the NB models and SAS(R) 8.0 was used for the LG modeling. RESULTS: Higher AUC in the NB model was obtained using the variables selected by the mutual information based feature selection method(AUC=.639, CI=.611, .660); lower AUC using the variables defined by a previous study(AUC=.599, CI=.570, .620). There was no difference between the LG models with different variables. CONCLUSION: This study demonstrated the mutual information method may be useful in identifying relevant predictors as the feature selection method, which can contribute to an increase in the predictive performance of a classifier.