Learning Hybrid Bayesian Networks from Data

Abstract

The main goal of this research is to generalize current techniques for learning Bayesian networks (BNs) from data to domains containing mixtures of continuous and discrete variables—learning hybrid Bayesian networks for short. To this end, a distinction is made between two learning tasks: (i) the causal discovery task, whereby the main goal is the discovery of interesting structural patterns in the data; and (ii) the classification task, whereby the prediction accuracy with respect to the class variable is the main goal. This distinction of tasks leads in turn to the definition of different techniques for their accomplishment. With regard to the causal discovery task, our approach is based on the discretization of the data, and on the subsequent parameterization of the discovered structure (or structures) based on the original non-discretized data. We propose new methods for the multivariate discretization of continuous variables, whereby variable interaction is taken into account. The defined discretization methods are incorporated into a procedure for learning hybrid BNs. With regard to the classification task, we use restricted BN models, whereby the structural form of the BN used is fixed or strictly constrained and the search focuses on its parameterization. We propose a new BN classifier that builds upon the naive-Bayes classifier and the finite-mixture classifier; its structure is defined so as to relax the modeling assumptions on which its component models are based. The hypothesis to be tested is that (i) for the causal discovery task, structure search based on discretized data will yield more accurate BN structures than structure search based on nondiscretized data; and (ii) for the classification task, the proposed BN classifier will achieve better classification performance than its component models, and it will allow for the direct handling of continuous variables. The experimental results provide evidence that the multivariate discretization of the continuous attributes allows for the induction of BN models more structurally accurate than the ones induced based on non-discretized data, and that the proposed BN classifier outperforms its component models both in terms of classification accuracy and of calibration.