Bayesian Network Structure Learning Research Articles

Bayesian network hybrid learning using an elite-guided genetic algorithm

Bayesian networks (BNs) constitute a powerful framework for probabilistic reasoning and have been extensively used in different research domains. This paper presents an improved hybrid learning strategy that features parameterized genetic algorithms (GAs) to learn the structure of BNs underlying a set of data samples. The performance of GAs is influenced by the choice of multiple initial parameters. This work is concerned with designing a series of parameter-less hybrid methods on a build-up basis: first the standard implementation is refined with the previously-developed data-informed evolutionary strategies. Then, two novel knowledge-driven parent controlling enhancements are presented. The first improvement works upon the parent limitation setting. BN structure learning algorithms typically set a bound for the maximum number of parents a BN node can possess to comply with the computational feasibility of the learning process. Our proposed method carefully selects the parents to rule out based on a knowledge-driven strategy. The second enhancement aims at reducing the sensitivity of the parent control setting by dynamically adjusting the maximum number of parents each node can hold. In the experimental section, it is shown how the adopted baseline outperforms the competitor algorithms included in the benchmark: thanks to its global search capabilities, the genetic methodology can efficiently prevail over other state-of-the-art structural learners on large networks. Presented experiments also prove how the proposed methods enhance the algorithmic efficiency and sensitivity to parameter setting, and address the problem of data fragmentation with respect to the baseline, with the advantage of higher performances in some cases.

PQISEM: BN's structure learning based on partial qualitative influences and SEM algorithm from missing data

The structure learning of Bayesian Network (BN) is an important issue for probabilistic inference of BN. In this paper, for missing data, we have proposed a BN's structure learning algorithm by making full use of some partial qualitative influences and SEM algorithm. Specifically, firstly, we address the problem that how to modify BN's parameters based on partial qualitative influence knowledge, which makes these parameters meet the given qualitative constraint relationship. Then, based on qualitative influences, we give random search operators in hill climbing method, and then analyse the selection rule of the initial network and selection strategy of candidate networks. Further, the PQISEM algorithm is proposed based on partial qualitative influences and SEM algorithm. Its complexity and convergence are analysed. Finally, the experiment illustrates PQISEM's performance by comparing with other algorithms on standard networks and discussing on different sample sizes and different missing value proportion.

Empirical hardness of finding optimal Bayesian network structures: algorithm selection and runtime prediction

Various algorithms have been proposed for finding a Bayesian network structure that is guaranteed to maximize a given scoring function. Implementations of state-of-the-art algorithms, solvers, for this Bayesian network structure learning problem rely on adaptive search strategies, such as branch-and-bound and integer linear programming techniques. Thus, the time requirements of the solvers are not well characterized by simple functions of the instance size. Furthermore, no single solver dominates the others in speed. Given a problem instance, it is thus a priori unclear which solver will perform best and how fast it will solve the instance. We show that for a given solver the hardness of a problem instance can be efficiently predicted based on a collection of non-trivial features which go beyond the basic parameters of instance size. Specifically, we train and test statistical models on empirical data, based on the largest evaluation of state-of-the-art exact solvers to date. We demonstrate that we can predict the runtimes to a reasonable degree of accuracy. These predictions enable effective selection of solvers that perform well in terms of runtimes on a particular instance. Thus, this work contributes a highly efficient portfolio solver that makes use of several individual solvers.

Cervid Exclusion Alters Boreal Forest Properties with Little Cascading Impacts on Soils

Large herbivores are capable of modifying entire ecosystems with a combination of direct (for example browsing/grazing, trampling, defecation) and indirect (for example affecting plant species composition that then alters soil properties) effects. With many ungulate populations increasing across the northern hemisphere it is important to develop a general theory for how these animals can be expected to impact their habitats. Here we present the results of an 8-year experimental exclusion of moose (Alces alces) from 15 recent boreal forest clear-cut sites in Central Norway. We used standard univariate techniques to describe the treatment effect on multiple forest and soil properties and combined this with a multivariate Bayesian network structure learning approach to objectively assess the potential mechanistic pathways for indirect effects on soils and soil fertility. We found that excluding moose had predictable direct effects, such as increasing the ratio of deciduous to coniferous tree biomass and the canopy cover and decreasing soil bulk density and temperature. However, we found no treatment effects on any measures of soil processes or quality (decomposition, nitrogen availability, C/N ratio, pH, nutrient stocks), and furthermore, we found only limited evidence that the direct effects had cascading (indirect) effects on soils. These findings oppose the commonly held belief that moose exclusion will increase soil fertility, but still highlights the strong ability of moose to directly modify forested ecosystems.

Order priors for Bayesian network discovery with an application to malware phylogeny

Bayesian networks have been used extensively to model and discover dependency relationships among sets of random variables. We learn Bayesian network structure with a combination of human knowledge about the partial ordering of variables and statistical inference of conditional dependencies from observed data. Our approach leverages complementary information from human knowledge and inference from observed data to produce networks that reflect human beliefs about the system as well as to fit the observed data. Applying prior beliefs about partial orderings of variables is an approach distinctly different from existing methods that incorporate prior beliefs about direct dependencies (or edges) in a Bayesian network. We provide an efficient implementation of the partial‐order prior in a Bayesian structure discovery learning algorithm, as well as an edge prior, showing that both priors meet the local modularity requirement necessary for an efficient Bayesian discovery algorithm. In benchmark studies, the partial‐order prior improves the accuracy of Bayesian network structure learning as well as the edge prior, even though order priors are more general. Our primary motivation is in characterizing the evolution of families of malware to aid cyber security analysts. For the problem of malware phylogeny discovery, we find that our algorithm, compared to existing malware phylogeny algorithms, more accurately discovers true dependencies that are missed by other algorithms.

Hybrid Parrallel Bayesian Network Structure Learning from Massive Data Using MapReduce

Bayesian Network (BN) is the popular and important data-mining model for representing uncertain knowledge. Much work has been done on migrating the BN structure learning algorithms, such as constraint-based (CB) and score-and-search-based (SSB) ones, to the MapReduce framework. But this approach is not suitable for hybrid algorithms, which have to conduct the Map and Reduce operation for all the data to get the scores, but not just the scores of the data in the pruned structures as in the traditional centralized version of hybrid algithm. This means the most time-comsuming part of the algorithm, the Map operations, will be run twice, once in CB and once in SSB. So in the MapReduce framework, when facing massive data, the simple migration of the traditional hybrid algorithm is almost equivalent to executing the CB and SSB sequentially, with little advantage. In this paper, we introduce a distributed hybrid BN structure learning algorithm. By using constraints and search methods that require the same data basis, the algorithm only needs to conduct the Map operation only once, in the CB stage, to prepare the data for the calculation of constraints and scores. Then it reuses intermediate results of constraints calculation in the SSB stage without Mapping the whole data again, thus greatly simplified the computing work. Experiment results show that the efficiency of the algorithm is more than doubled compared to the SSB, and the accuracy is improved by about 36% compared to the CB.

Learning Discrete Bayesian Networks from Continuous Data

Learning Bayesian networks from raw data can help provide insights into the relationships between variables. While real data often contains a mixture of discrete and continuous-valued variables, many Bayesian network structure learning algorithms assume all random variables are discrete. Thus, continuous variables are often discretized when learning a Bayesian network. However, the choice of discretization policy has significant impact on the accuracy, speed, and interpretability of the resulting models. This paper introduces a principled Bayesian discretization method for continuous variables in Bayesian networks with quadratic complexity instead of the cubic complexity of other standard techniques. Empirical demonstrations show that the proposed method is superior to the established minimum description length algorithm. In addition, this paper shows how to incorporate existing methods into the structure learning process to discretize all continuous variables and simultaneously learn Bayesian network structures.

A novel method for Bayesian networks structure learning based on Breeding Swarm algorithm

Bayesian networks (BNs) are widely used as one of the most effective models in bioinformatics, artificial intelligence, text analysis, medical diagnosis, etc. Learning the structure of BNs from data can be viewed as an optimization problem and is proved that this problem is NP-hard. Therefore, heuristic methods can be used as powerful tools to find high-quality networks. In this paper, an interesting approach which is based on Breeding Swarm has been used to learn BNs. Breeding Swarm is a hybrid GA/PSO which enable us to benefits the strengths of particle swarm optimization with genetic algorithms. In order to assess the proposed method, several real-world and benchmark applications are used. Results show that our method is a clear improvement on genetic algorithm and particle swarm optimization.

FUZZY CLUSTERING BASED BAYESIAN FRAMEWORK TO PREDICT MENTAL HEALTH PROBLEMS AMONG CHILDREN

According to World Health Organization, 10-20% of children and adolescents all over the world are experiencing mental disorders. Correct diagnosis of mental disorders at an early stage improves the quality of life of children and avoids complicated problems. Various expert systems using artificial intelligence techniques have been developed for diagnosing mental disorders like Schizophrenia, Depression, Dementia, etc. This study focuses on predicting basic mental health problems of children, like Attention problem, Anxiety problem, Developmental delay, Attention Deficit Hyperactivity Disorder (ADHD), Pervasive Developmental Disorder(PDD), etc. using the machine learning techniques, Bayesian Networks and Fuzzy clustering. The focus of the article is on learning the Bayesian network structure using a novel Fuzzy Clustering Based Bayesian network structure learning framework. The performance of the proposed framework was compared with the other existing algorithms and the experimental results have shown that the proposed framework performs better than the earlier algorithms.

Learning Bayesian network structures under incremental construction curricula

Bayesian networks have been successfully applied to various tasks for probabilistic reasoning and causal modeling. One major challenge in the application of Bayesian networks is to learn the Bayesian network structures from data. In this paper, we take advantage of the idea of curriculum learning and learn Bayesian network structures by stages. At each stage a subnet is learned over a selected subset of the random variables. The selected subset grows with stages and eventually includes all the variables. We show that in our approach each target subnet is closer to the target Bayesian network than any of its predecessors. The experimental results show that our algorithm outperformed the state-of-the-art heuristic approach in learning Bayesian network structures under several different evaluation metrics.

An efficient Bayesian approach for Gaussian Bayesian network structure learning

ABSTRACTThis article proposes a Bayesian computing algorithm to infer Gaussian directed acyclic graphs (DAGs). It has the ability of escaping local modes and maintaining adequate computing speed compared to existing methods. Simulations demonstrated that the proposed algorithm has low false positives and false negatives in comparison to an algorithm applied to DAGs. We applied the algorithm to an epigenetic dataset to infer DAG's for smokers and nonsmokers.

Risk reduction in a project portfolio

The positive impacts of managing projects as a portfolio are quantified by comparing the value of the integrated risk of a project portfolio and the aggregation of single project risks implemented separately. Firstly, the integrated risk is defined by proposing risky events based on set theory. Secondly, as projects interact with each other in a project portfolio, the integrated risk is evaluated by using a Bayesian network structure learning algorithm to construct an interdependent network of risks. Finally, the integrated risk of a practical case is assessed using this method, and the results show that the proposed method is an effective tool for calculating the extent of risk reduction of implementing a project portfolio and identifying the most risky project, so as to assist companies in making comprehensive decisions in the phase of portfolio selection and portfolio controlling.

Bayesian Network Structure Learning with Integer Programming: Polytopes, Facets and Complexity

The challenging task of learning structures of probabilistic graphical models is an important problem within modern AI research. Recent years have witnessed several major algorithmic advances in structure learning for Bayesian networks - arguably the most central class of graphical models - especially in what is known as the score-based setting. A successful generic approach to optimal Bayesian network structure learning (BNSL), based on integer programming (IP), is implemented in the GOBNILP system. Despite the recent algorithmic advances, current understanding of foundational aspects underlying the IP based approach to BNSL is still somewhat lacking. Understanding fundamental aspects of cutting planes and the related separation problem is important not only from a purely theoretical perspective, but also since it holds out the promise of further improving the efficiency of state-of-the-art approaches to solving BNSL exactly. In this paper, we make several theoretical contributions towards these goals: (i) we study the computational complexity of the separation problem, proving that the problem is NP-hard; (ii) we formalise and analyse the relationship between three key polytopes underlying the IP-based approach to BNSL; (iii) we study the facets of the three polytopes both from the theoretical and practical perspective, providing, via exhaustive computation, a complete enumeration of facets for low-dimensional family-variable polytopes; and, furthermore, (iv) we establish a tight connection of the BNSL problem to the acyclic subgraph problem.

Bayesian Network Constraint-Based Structure Learning Algorithms: Parallel and Optimized Implementations in the bnlearn R Package

It is well known in the literature that the problem of learning the structure of Bayesian networks is very hard to tackle: Its computational complexity is super-exponential in the number of nodes in the worst case and polynomial in most real-world scenarios. Efficient implementations of score-based structure learning benefit from past and current research in optimization theory, which can be adapted to the task by using the network score as the objective function to maximize. This is not true for approaches based on conditional independence tests, called constraint-based learning algorithms. The only optimization in widespread use, backtracking, leverages the symmetries implied by the definitions of neighborhood and Markov blanket. In this paper we illustrate how backtracking is implemented in recent versions of the bnlearn R package, and how it degrades the stability of Bayesian network structure learning for little gain in terms of speed. As an alternative, we describe a software architecture and framework that can be used to parallelize constraint-based structure learning algorithms (also implemented in bnlearn) and we demonstrate its performance using four reference networks and two real-world data sets from genetics and systems biology. We show that on modern multi-core or multiprocessor hardware parallel implementations are preferable over backtracking, which was developed when single-processor machines were the norm.

A New Bayesian Network Structure Learning Algorithm Mechanism Based on the Decomposability of Scoring Functions

A New Bayesian Network Structure Learning Algorithm Mechanism Based on the Decomposability of Scoring Functions

Verification of Three-Phase Dependency Analysis Bayesian Network Learning Method for Maize Carotenoid Gene Mining.

Background and Objective Mining the genes related to maize carotenoid components is important to improve the carotenoid content and the quality of maize. Methods On the basis of using the entropy estimation method with Gaussian kernel probability density estimator, we use the three-phase dependency analysis (TPDA) Bayesian network structure learning method to construct the network of maize gene and carotenoid components traits. Results In the case of using two discretization methods and setting different discretization values, we compare the learning effect and efficiency of 10 kinds of Bayesian network structure learning methods. The method is verified and analyzed on the maize dataset of global germplasm collection with 527 elite inbred lines. Conclusions The result confirmed the effectiveness of the TPDA method, which outperforms significantly another 9 kinds of Bayesian network learning methods. It is an efficient method of mining genes for maize carotenoid components traits. The parameters obtained by experiments will help carry out practical gene mining effectively in the future.

A Bayesian Network Model on the Public Bicycle Choice Behavior of Residents: A Case Study of Xi’an

In order to study the main factors affecting the behaviors that city residents make regarding public bicycle choice and to further study the public bicycle user’s personal characteristics and travel characteristics, a travel mode choice model based on a Bayesian network was established. Taking residents of Xi’an as the research object, a K2 algorithm combined with mutual information and expert knowledge was proposed for Bayesian network structure learning. The Bayesian estimation method was used to estimate the parameters of the network, and a Bayesian network model was established to reflect the interactions among the public bicycle choice behaviors along with other major factors. The K-fold cross-validation method was used to validate the model performance, and the hit rate of each travel mode was more than 80%, indicating the precision of the proposed model. Experimental results also present the higher classification accuracy of the proposed model. Therefore, it may be concluded that the resident travel mode choice may be accurately predicted according to the Bayesian network model proposed in our study. Additionally, this model may be employed to analyze and discuss changes in the resident public bicycle choice and to note that they may possibly be influenced by different travelers’ characteristics and trip characteristics.

Duplicate Detection for Bayesian Network Structure Learning

We address the well-known score-based Bayesian network structure learning problem. Breadth-first branch and bound (BFBnB) has been shown to be an effective approach for solving this problem. Duplicate detection is an important component of the BFBnB algorithm. Previously, an external sorting-based technique was used for delayed duplicate detection (DDD). We propose a hashing-based technique for DDD and a bin packing algorithm for minimizing the number of external memory files and operations. We also give a structured duplicate detection approach which completely eliminates DDD. Importantly, these techniques ensure the search algorithms respect any given memory bound. Empirically, we demonstrate that structured duplicate detection is significantly faster than the previous state of the art in limited-memory settings. Our results show that the bin packing algorithm incurs some overhead, but that the overhead is offset by reducing I/O when more memory is available.

Bnstruct: an R package for Bayesian Network structure learning in the presence of missing data.

A Bayesian Network is a probabilistic graphical model that encodes probabilistic dependencies between a set of random variables. We introduce bnstruct, an open source R package to (i) learn the structure and the parameters of a Bayesian Network from data in the presence of missing values and (ii) perform reasoning and inference on the learned Bayesian Networks. To the best of our knowledge, there is no other open source software that provides methods for all of these tasks, particularly the manipulation of missing data, which is a common situation in practice. The software is implemented in R and C and is available on CRAN under a GPL licence. francesco.sambo@unipd.it. Supplementary data are available at Bioinformatics online.

Exploiting Experts' Knowledge for Structure Learning of Bayesian Networks.

Learning Bayesian network structures from data is known to be hard, mainly because the number of candidate graphs is super-exponential in the number of variables. Furthermore, using observational data alone, the true causal graph is not discernible from other graphs that model the same set of conditional independencies. In this paper, it is investigated whether Bayesian network structure learning can be improved by exploiting the opinions of multiple domain experts regarding cause-effect relationships. In practice, experts have different individual probabilities of correctly labeling the inclusion or exclusion of edges in the structure. The accuracy of each expert is modeled by three parameters. Two new scoring functions are introduced that score each candidate graph based on the data and experts' opinions, taking into account their accuracy parameters. In the first scoring function, the experts' accuracies are estimated using an expectation-maximization-based algorithm and the estimated accuracies are explicitly used in the scoring process. The second function marginalizes out the accuracy parameters to obtain more robust scores when it is not possible to obtain a good estimate of experts' accuracies. The experimental results on simulated and real world datasets show that exploiting experts' knowledge can improve the structure learning if we take the experts' accuracies into account.