A New Singly Connected Network Classifier based on Mutual Information

Abstract

For reasoning under uncertainty the Bayesian Network has become the representation of choice. However, except were models are considered 'simple' the tasks of construction and inference are provably NP hard. For modelling larger real-world problems this computational complexity has been addressed by methods that approximate the model. The Naive Bayes (NB) Classifier which has strong assumptions of independence among features is a common approach whilst the class of trees another less extreme example. The aim of this paper is to investigate the use of an information theory based technique as a mechanism for inference in Singly Connected Networks (SCN) or 'polytrees'. We call this variant a Mutual Information Measure (MIM) Classifier. We experimentally evaluate this new approach and compare the resulting classification performance of the MIM Classifier against (a) a Naive Bayes Classifier, (b) a General Bayesian Network (GBN) Classifier and (c) a Singly Connected Network, using benchmark problems taken from the UCI repository. With respect to (a) we show that the MIM Classifier generally performs better than the NB Classifier. For (b) and (c) we show that the MIM Classifier is comparable with both the GBN and SCN Classifiers and in most data sets used performs marginally better.