Discrete Bayesian Networks Research Articles

Discrete Bayesian Network Classifiers

We have had to wait over 30 years since the naive Bayes model was first introduced in 1960 for the so-called Bayesian network classifiers to resurge. Based on Bayesian networks, these classifiers have many strengths, like model interpretability, accommodation to complex data and classification problem settings, existence of efficient algorithms for learning and classification tasks, and successful applicability in real-world problems. In this article, we survey the whole set of discrete Bayesian network classifiers devised to date, organized in increasing order of structure complexity: naive Bayes, selective naive Bayes, seminaive Bayes, one-dependence Bayesian classifiers, k -dependence Bayesian classifiers, Bayesian network-augmented naive Bayes, Markov blanket-based Bayesian classifier, unrestricted Bayesian classifiers, and Bayesian multinets. Issues of feature subset selection and generative and discriminative structure and parameter learning are also covered.

Discrete exponential Bayesian networks: Definition, learning and application for density estimation

Our work aims at developing or expliciting bridges between Bayesian networks (BNs) and Natural Exponential Families, by proposing discrete exponential Bayesian networks as a generalization of usual discrete ones. We introduce a family of prior distributions which generalizes the Dirichlet prior applied on discrete Bayesian networks, and then we determine the overall posterior distribution. Subsequently, we develop the Bayesian estimators of the parameters, and a new score function that extends the Bayesian Dirichlet score for BN structure learning. Our goal is to determine empirically in which contexts some of our discrete exponential BNs (Poisson deBNs) can be an effective alternative to usual BNs for density estimation.

A Compound Poisson Model for Learning Discrete Bayesian Networks

We introduce here the concept of Bayesian networks, in compound Poisson model, which provides a graphical modeling framework that encodes the joint probability distribution for a set of random variables within a directed acyclic graph. We suggest an approach proposal which offers a new mixed implicit estimator. We show that the implicit approach applied in compound Poisson model is very attractive for its ability to understand data and does not require any prior information. A comparative study between learned estimates given by implicit and by standard Bayesian approaches is established. Under some conditions and based on minimal squared error calculations, we show that the mixed implicit estimator is better than the standard Bayesian and the maximum likelihood estimators. We illustrate our approach by considering a simulation study in the context of mobile communication networks.

A new multivariate imputation method based on Bayesian networks

Dealing with incomplete data is a pervasive problem in statistical surveys. Bayesian networks have been recently used in missing data imputation. In this research, we propose a new methodology for the multivariate imputation of missing data using discrete Bayesian networks and conditional Gaussian Bayesian networks. Results from imputing missing values in coronary artery disease data set and milk composition data set as well as a simulation study from cancer-neapolitan network are presented to demonstrate and compare the performance of three Bayesian network-based imputation methods with those of multivariate imputation by chained equations (MICE) and the classical hot-deck imputation method. To assess the effect of the structure learning algorithm on the performance of the Bayesian network-based methods, two methods called Peter-Clark algorithm and greedy search-and-score have been applied. Bayesian network-based methods are: first, the method introduced by Di Zio et al. [Bayesian networks for imputation, J. R. Stat. Soc. Ser. A 167 (2004), 309–322] in which, each missing item of a variable is imputed using the information given in the parents of that variable; second, the method of Di Zio et al. [Multivariate techniques for imputation based on Bayesian networks, Neural Netw. World 15 (2005), 303–310] which uses the information in the Markov blanket set of the variable to be imputed and finally, our new proposed method which applies the whole available knowledge of all variables of interest, consisting the Markov blanket and so the parent set, to impute a missing item. Results indicate the high quality of our new proposed method especially in the presence of high missingness percentages and more connected networks. Also the new method have shown to be more efficient than the MICE method for small sample sizes with high missing rates.

Estimating expected value of information using Bayesian belief networks: a case study in fish consumption advisory

A recent international collaborative effort was directed at quantifying the risks and benefits of fish consumption. A nonparametric continuous–discrete Bayesian belief network was constructed to support these calculations. The same Bayesian belief network has enabled calculation of the expected benefits of further research directed at shrinking the uncertainties and prioritization of possible research efforts.

Programming algorithm for reference of discrete dynamic Bayesian network

The discrete dynamic Bayesian network is a useful tool for modeling and inferring the time series progress,and it has wide modeling application value.But its inference algorithm needs improving.Concerning the shortcomings of the discrete dynamic Bayesian network,such as its inference algorithm is hard to understand,hard to program and running slowly,this paper proposed the most suitable storage data structure of the discrete dynamic Bayesian network,deduced the fast inference algorithm for the discrete dynamic Bayesian network,and verified the inference algorithm for the discrete dynamic Bayesian networks through a sample.

Gene Ontology consistent protein function prediction: the FALCON algorithm applied to six eukaryotic genomes.

Gene Ontology (GO) is a hierarchical vocabulary for the description of biological functions and locations, often employed by computational methods for protein function prediction. Due to the structure of GO, function predictions can be self- contradictory. For example, a protein may be predicted to belong to a detailed functional class, but not in a broader class that, due to the vocabulary structure, includes the predicted one.We present a novel discrete optimization algorithm called Functional Annotation with Labeling CONsistency (FALCON) that resolves such contradictions. The GO is modeled as a discrete Bayesian Network. For any given input of GO term membership probabilities, the algorithm returns the most probable GO term assignments that are in accordance with the Gene Ontology structure. The optimization is done using the Differential Evolution algorithm. Performance is evaluated on simulated and also real data from Arabidopsis thaliana showing improvement compared to related approaches. We finally applied the FALCON algorithm to obtain genome-wide function predictions for six eukaryotic species based on data provided by the CAFA (Critical Assessment of Function Annotation) project.

A continuous Bayesian network for earth dams' risk assessment: methodology and quantification

Dams' safety is highly important for authorities around the world. The impacts of a dam failure can be enormous. Models for investigating dam safety are required for helping decision-makers to mitigate the possible adverse consequences of flooding. A model for earth dam safety must specify clearly possible contributing factors, failure modes and potential consequences of dam failure. Probabilistic relations between variables should also be specified. Bayesian networks (BNs) have been identified as tools that would assist dam engineers on assessing risks. BNs are graphical models that facilitate the construction of a joint probability distribution. Most of the time, the variables included in a model for earth dam risk assessment involve continuous quantities. The presence of continuous random variables makes the implementation of discrete BNs difficult. An alternative to discrete BNs is the use of non-parametric continuous BNs, which will be briefly described in this article. As an example, a model for earth dams' safety in the State of Mexico will be discussed. Results regarding the quantification of conditional rank correlations through ratios of unconditional rank correlations have not been presented before and are introduced herein. While the complete application of the model for the State of Mexico is presented in an accompanying paper, here some results regarding model use are shown for demonstration purposes. The methods presented in this article can be applied for investigating risks of failure of civil infrastructures other than earth dams.

Consistent model selection of discrete Bayesian networks from incomplete data

A maximum likelihood based model selection of discrete Bayesian networks is considered. The structure learning is performed by employing a scoring function $S$, which, for a given network $G$ and $n$-sample $D_{n}$, is defined as the maximum marginal log-likelihood $l$ minus a penalization term $\lambda_{n}h$ proportional to network complexity $h(G)$, $$S(G|D_{n})=l(G|D_{n})-\lambda_{n}h(G).$$ An available case analysis is developed with the standard log-likelihood replaced by the sum of sample average node log-likelihoods. The approach utilizes partially missing data records and allows for comparison of models fitted to different samples. In missing completely at random settings the estimation is shown to be consistent if and only if the sequence $\lambda_{n}$ converges to zero at a slower than $n^{-{1/2}}$ rate. In particular, the BIC model selection ($\lambda_{n}=0.5\log(n)/n$) applied to the node-average log-likelihood is shown to be inconsistent in general. This is in contrast to the complete data case when BIC is known to be consistent. The conclusions are confirmed by numerical experiments.

Signal propagation in Bayesian networks and its relationship with intrinsically multivariate predictive variables

A set of predictor variables is said to be intrinsically multivariate predictive (IMP) for a target variable if all properly contained subsets of the predictor set are poor predictors of the target but the full set predicts the target with great accuracy. In a previous article, the main properties of IMP Boolean variables have been analytically described, including the introduction of the IMP score, a metric based on the coefficient of determination (CoD) as a measure of predictiveness with respect to the target variable. It was shown that the IMP score depends on four main properties: logic of connection, predictive power, covariance between predictors and marginal predictor probabilities (biases). This paper extends that work to a broader context, in an attempt to characterize properties of discrete Bayesian networks that contribute to the presence of variables (network nodes) with high IMP scores. We have found that there is a relationship between the IMP score of a node and its territory size, i.e., its position along a pathway with one source: nodes far from the source display larger IMP scores than those closer to the source, and longer pathways display larger maximum IMP scores. This appears to be a consequence of the fact that nodes with small territory have larger probability of having highly covariate predictors, which leads to smaller IMP scores. In addition, a larger number of XOR and NXOR predictive logic relationships has positive influence over the maximum IMP score found in the pathway. This work presents analytical results based on a simple structure network and an analysis involving random networks constructed by computational simulations. Finally, results from a real Bayesian network application are provided.

Non-impeding noisy-AND tree causal models over multi-valued variables

To specify a Bayesian network (BN), a conditional probability table (CPT), often of an effect conditioned on its n causes, must be assessed for each node. Its complexity is generally exponential in n. Noisy-OR and a number of extensions reduce the complexity to linear, but can only represent reinforcing causal interactions. Non-impeding noisy-AND (NIN-AND) trees are the first causal models that explicitly express reinforcement, undermining, and their mixture. Their acquisition has a linear complexity, in terms of both the number of parameters and the size of the tree topology. As originally proposed, however, they allow only binary effects and causes. This work generalizes binary NIN-AND tree models to multi-valued effects and causes. It is shown that the generalized NIN-AND tree models express reinforcement, undermining, and their mixture at multiple levels, relative to each active value of the effect. The model acquisition is still efficient. For binary variables, they degenerate into binary NIN-AND tree models. Hence, this contribution enables CPTs of discrete BNs of arbitrary variables (binary or multi-valued) to be specified efficiently through the intuitive concepts of reinforcement and undermining.

SOFC modelling based on discrete Bayesian network For system diagnosis use

Abstract We propose in this paper a diagnosis method that is aimed to detect and isolate SOFC system fault by using the FC stack as a sensor. A discrete Bayesian network (BN) was established to illustrate the input-output causal relations of the stack. In order to examine the generalizability of the network structure, the BN was parameterized to fit the experimental data from two different SOFC systems. The models showed reasonable accuracy of state estimation for 6 operating variables. Finally, the BN model was experimented for diagnosing a specified system fault.

Developing an effective algorithm for dynamic UAV path planning with incomplete threat information

In this paper the problem of online dynamic UAV path planning is studied for the situation where a manoeuvring pop-up threat exists and information about this threat is incomplete. The problem is addressed by integrating a structure-varying discrete dynamic Bayesian network (SVDDBN) into the model prediction control (MPC) algorithm. The SVDDBN is used to construct online a dynamic threat assessment model by estimating and predicting the states of the manoeuvring pop-up threat. The output of this model is then fed into the MPC algorithm for effective path planning. The results obtained in the performed simulation studies demonstrate the excellent performance of the proposed algorithm.

Extended Shenoy–Shafer architecture for inference in hybrid bayesian networks with deterministic conditionals

The main goal of this paper is to describe an architecture for solving large general hybrid Bayesian networks (BNs) with deterministic conditionals for continuous variables using local computation. In the presence of deterministic conditionals for continuous variables, we have to deal with the non-existence of the joint density function for the continuous variables. We represent deterministic conditional distributions for continuous variables using Dirac delta functions. Using the properties of Dirac delta functions, we can deal with a large class of deterministic functions. The architecture we develop is an extension of the Shenoy–Shafer architecture for discrete BNs. We extend the definitions of potentials to include conditional probability density functions and deterministic conditionals for continuous variables. We keep track of the units of continuous potentials. Inference in hybrid BNs is then done in the same way as in discrete BNs but by using discrete and continuous potentials and the extended definitions of combination and marginalization. We describe several small examples to illustrate our architecture. In addition, we solve exactly an extended version of the crop problem that includes non-conditional linear Gaussian distributions and non-linear deterministic functions.

Implicit inequality constraints in a binary tree model

In this paper we investigate the geometry of a discrete Bayesian network whose graph is a tree all of whose variables are binary and the only observed variables are those labeling its leaves. We provide the full geometric description of these models which is given by a set of polynomial equations together with a set of complementary implied inequalities induced by the positivity of probabilities on hidden variables. The phylogenetic invariants given by the equations can be useful in the construction of simple diagnostic tests. However, in this paper we point out the importance of also incorporating the associated inequalities into any statistical analysis. The full characterization of these inequality constraints derived in this paper helps us determine how and why routine statistical methods can break down for this model class.

A Gaussian mixed model for learning discrete Bayesian networks

In this paper, we address the problem of learning discrete Bayesian networks from noisy data. A graphical model based on a mixture of Gaussian distributions with categorical mixing structure coming from a discrete Bayesian network is considered. The network learning is formulated as a maximum likelihood estimation problem and performed by employing an EM algorithm. The proposed approach is relevant to a variety of statistical problems for which Bayesian network models are suitable—from simple regression analysis to learning gene/protein regulatory networks from microarray data.

Database-support for continuous prediction queries over streaming data

Prediction is emerging as an essential ingredient for real-time monitoring, planning and decision support applications such as intrusion detection, e-commerce pricing and automated resource management. This paper presents a system that efficiently supports continuous prediction queries (CPQs) over streaming data using seamlessly-integrated probabilistic models. Specifically, we describe how to execute and optimize CPQs using discrete (Dynamic) Bayesian Networks as the underlying predictive model. Our primary contribution is a novel cost-based optimization framework that employs materialization, sharing, and model-specific optimization techniques to enable highly-efficient point- and range-based CPQ execution. Furthermore, we support efficient execution of top-k and threshold-based high probability queries. We characterize the behavior of our system and demonstrate significant performance gains using a prototype implementation operating on real-world network intrusion data and deployed as part of a real-time software-performance monitoring system.

Estimation and application of a Bayesian network model for discrete travel choice analysis

This paper describes a Bayesian network approach for modeling discrete travel choice problems. Bayesian networks are a marriage between probabilistic theory and graph theory. In a Bayesian network, the graphical network topology specifies the model structure while conditional probability distributions provide a mechanism to represent the probabilistic causal relationships between variables. Though this modeling tool is extensively used in many scientific areas for data causality or correlation analyses, no formal statement has been made on how to specify and estimate a Bayesian network for generic discrete choice modeling problems. In this paper, our particular interest is on the structure estimation of a discrete Bayesian network model in the context of travel choice prediction. The feature of a Bayesian network model, which allows its network structure to be estimated by combining both cause-effect hypotheses and observational information, enhances its capability in modeling complex travel choice patterns that existing models with a linear structure may not capture. Through a mode choice example for work trips, we demonstrate the model's advantages and disadvantages in this kind of applications.

Message Passing for Hybrid Bayesian Networks: Representation, Propagation, and Integration

The traditional message passing algorithm was originally developed by Pearl in the 1980s for computing exact inference solutions for discrete polytree Bayesian networks (BN). When a loop is present in the network, propagating messages are not exact, but the loopy algorithm usually converges and provides good approximate solutions. However, in general hybrid BNs, the message representation and manipulation for arbitrary continuous variable and message propagation between different types of variables are still open problems. The novelty of the work presented here is to propose a framework to compute, propagate, and integrate the messages for hybrid models. First, we combine unscented transformation and Pearl's message passing algorithm to deal with the arbitrary functional relationships between continuous variables in the network. For the general hybrid model, we partition the network into separate parts by introducing the concept of interface node. We then apply different algorithms for each subnetwork. Finally we integrate the information through the channel of interface nodes and then estimate the posterior distributions for all hidden variables. The numerical experiments show that the algorithm works well for nonlinear hybrid BNs.

Join tree propagation with prioritized messages

Current join tree propagation algorithms treat all propagated messages as being of equal importance. On the contrary, it is often the case in real-world Bayesian networks that only some of the messages propagated from one join tree node to another are relevant to subsequent message construction at the receiving node. In this article, we propose the first join tree propagation algorithm that identifies and constructs the relevant messages first. Our approach assigns lower priority to the irrelevant messages as they only need to be constructed so that posterior probabilities can be computed when propagation terminates. Experimental results, involving the processing of evidence in four real-world Bayesian networks, empirically demonstrate an improvement over the state-of-the-art method for exact inference in discrete Bayesian networks. © 2009 Wiley Periodicals, Inc. NETWORKS, 2010