Conditional Mutual Information based Attribute Weighting on General Bayesian Network Classifier

Abstract

Bayesian networks are used for building effective classifiers in the field of machine learning. Naive Bayes (NB) classifier which has the simplest structure among Bayesian Network (BN) classifiers assume conditional independence of attributes, given the class. This assumption often fails in real world context. Attribute weighting is one of the major approaches used to alleviate this weakness and is proved to be effective. Unlike the BN classifiers in which class node is treated as a parent to all the attribute nodes, such as NB, BN Augmented Naive Bayes (BAN), Tree Augmented Naive Bayes (TAN), Averaged One Dependence Estimators (AODE), the class node is treated as an ordinary node in General Bayesian Network (GBN). No study had been done on whether attribute weighting will improve GBN classifier.Here, we evaluate the effectiveness of Conditional Mutual Information based attribute weighting on GBN. We experimentally test ICS algorithm for GBN provided by WEKA tool, using 20 UCI datasets and compare it with the attribute weighted approach with regard to ACC which denotes classification accuracy and AUC which measures Area Under ROC Curve. Results of our experiments show that conditional mutual information based attribute weighted GBN significantly outperforms unweighted GBN.