Structure Learning of Bayesian Network Based on Adaptive Thresholding.

Yang Zhang,Minghui Sun,Zhiyi Duan,Limin Wang

doi:10.3390/e21070665

Abstract

Direct dependencies and conditional dependencies in restricted Bayesian network classifiers (BNCs) are two basic kinds of dependencies. Traditional approaches, such as filter and wrapper, have proved to be beneficial to identify non-significant dependencies one by one, whereas the high computational overheads make them inefficient especially for those BNCs with high structural complexity. Study of the distributions of information-theoretic measures provides a feasible approach to identifying non-significant dependencies in batch that may help increase the structure reliability and avoid overfitting. In this paper, we investigate two extensions to the k-dependence Bayesian classifier, MI-based feature selection, and CMI-based dependence selection. These two techniques apply a novel adaptive thresholding method to filter out redundancy and can work jointly. Experimental results on 30 datasets from the UCI machine learning repository demonstrate that adaptive thresholds can help distinguish between dependencies and independencies and the proposed algorithm achieves competitive classification performance compared to several state-of-the-art BNCs in terms of 0–1 loss, root mean squared error, bias, and variance.

Highlights

Classification is one of the most important tasks in machine learning
We investigate two extensions to k-dependence Bayesian classifier (KDB), mutual information (MI)-based feature selection and conditional mutual information (CMI)-based dependence selection based on a novel adaptive thresholding method
Considering that Adaptive KDB (AKDB) has significantly lower 0–1 loss and root mean squared error (RMSE) in comparison to other algorithms, we argue that the FS technique in tandem with the DS technique used in the proposed algorithm is powerful to improve classification accuracy

Summary

Introduction

The basic problem of supervised classification is the induction of a model with feature set X = { X1 , · · · , Xn } that classifies testing instance (example) x = { x1 , · · · , xn } into one of the several class labels {c1 , · · · , cm } of class variable C. Bayesian network classifiers (BNCs) have many desirable properties over other numerous classification models, such as model interpretability, the ease of implementation, the ability to deal with multi-class classification problems and the comparable classification performance [1]. Unrestricted BNCs are the least biased, the search-space that is needed to train such a model increases exponentially with the number of features [2]. The arising complexity issues limit the study of unrestricted BNCs and it has led to the study of restricted BNCs, from 0-dependence naive

Methods

Findings

Discussion

Conclusion