Selective Bayesian Classifiers Research Articles

Beyond accuracy: Learning selective Bayesian classifiers with minimal test cost

Some existing test-cost sensitive learning algorithms are about balancing act of the misclassification cost and the total test cost, and the others focus on the balance between the classification accuracy and the total test cost. By far, however, few works reduce the total test cost, yet at the same time maintain the high classification accuracy. In order to achieve this goal, this paper modifies the backward greedy search strategy employed in selective Bayesian classifiers (SBC), which is a state-of-the-art improved naive Bayes algorithm pursuing the high classification accuracy but ignoring the total test cost. We call the resulting model test-cost sensitive naive Bayes (TCSNB). TCSNB conducts a modified backward greedy search strategy to select an optimal attribute subset with the minimal total test cost, yet at the same time maintains the high classification accuracy that characterizes SBC. Extensive empirical study validates its effectiveness and efficiency.

A Novel Bayes Model: Hidden Naive Bayes

Because learning an optimal Bayesian network classifier is an NP-hard problem, learning-improved naive Bayes has attracted much attention from researchers. In this paper, we summarize the existing improved algorithms and propose a novel Bayes model: hidden naive Bayes (HNB). In HNB, a hidden parent is created for each attribute which combines the influences from all other attributes. We experimentally test HNB in terms of classification accuracy, using the 36 UCI data sets selected by Weka, and compare it to naive Bayes (NB), selective Bayesian classifiers (SBC), naive Bayes tree (NBTree), tree-augmented naive Bayes (TAN), and averaged one-dependence estimators (AODE). The experimental results show that HNB significantly outperforms NB, SBC, NBTree, TAN, and AODE. In many data mining applications, an accurate class probability estimation and ranking are also desirable. We study the class probability estimation and ranking performance, measured by conditional log likelihood (CLL) and the area under the ROC curve (AUC), respectively, of naive Bayes and its improved models, such as SBC, NBTree, TAN, and AODE, and then compare HNB to them in terms of CLL and AUC. Our experiments show that HNB also significantly outperforms all of them.