From undirected dependence to directed causality: A novel Bayesian learning approach

Abstract

Bayesian network (BN) is one of the most powerful probabilistic models in the field of uncertain knowledge representation and reasoning. During the past decade, numerous approaches have been proposed to build directed acyclic graph (DAG) as the structural specification of BN. However, for most Bayesian network classifiers (BNCs) the directed edges in DAG substantially represent assertions of conditional independence rather than causal relationships although the learned joint probability distributions may fit data well, thus they cannot be applied to causal reasoning. In this paper, conditional entropy is introduced to measure causal uncertainty due to its asymmetry characteristic, and heuristic search strategy is applied to build Bayesian causal tree (BCT) by identifying significant causalities. The resulting highly scalable topology can represent causal relationship in terms of causal science, and corresponding joint probability can fit training data in terms of data science. Then ensemble learning strategy is applied to build Bayesian causal forest (BCF) with a set of BCTs, each taking different attribute as the root node to represent root cause for causality analysis. Extensive experiments performed on 32 public datasets from the UCI machine learning repository show that BCF achieves outstanding classification performance compared to state-of-the-art single-model BNCs (e.g., CFWNB), ensemble BNCs (e.g., WATAN, IWAODE, WAODE-MI and TAODE) and non-Bayesian learners (e.g., SVM, k-NN, LR).