Bayesian Network Structure Learning Research Articles

Determining the direction of the local search in topological ordering space for Bayesian network structure learning

Local search in a topological ordering space is an efficient way of learning Bayesian network structures in large-scale problems. However, existing algorithms typically focus on stochastically developing the neighborhood of an ordering without a specific direction and can quickly stop at a local optimum. In this study, a novel approach is proposed to improve the capability of a local search by determining the search direction. The direction of the search step is identified with respect to the priority in a score cache. We also design robust terminal conditions and insertion methods based on the proposed operator. The direction of escaping from local optima is identified by transferring the ordering to a new restart, which implies an equivalent class of the optimal structure. Moreover, we adopt a breadth-first search method for the conversion between structures and orderings. The identification of the direction can accelerate the convergence of the local search and acquire the higher quality structure. Furthermore, our experimental results demonstrated that the proposed methods highly improve the accuracy and efficiency of learning the optimal ordering from the training dataset compared with state-of-the-art algorithms.

Leveraging Domain Knowledge in Multitask Bayesian Network Structure Learning

Network structure learning algorithms have aided network discovery in fields such as bioinformatics, neuroscience, ecology and social science. However, challenges remain in learning informative networks for related sets of tasks because the search space of Bayesian network structures is characterized by large basins of approximately equivalent solutions. Multitask algorithms select a set of networks that are near each other in the search space, rather than a score-equivalent set of networks chosen from independent regions of the space. This selection preference allows a domain expert to see only differences supported by the data. However, the usefulness of these algorithms for scientific datasets is limited because existing algorithms naively assume that all pairs of tasks are equally related. We introduce a framework that relaxes this assumption by incorporating domain knowledge about task-relatedness into the learning objective. Using our framework, we introduce the first multitask Bayesian network algorithm that leverages domain knowledge about the relatedness of tasks. We use our algorithm to explore the effect of task-relatedness on network discovery and show that our algorithm learns networks that are closer to ground truth than naive algorithms and that our algorithm discovers patterns that are interesting.

BIC-based node order learning for improving Bayesian network structure learning

BIC-based node order learning for improving Bayesian network structure learning

Selected Artificial Intelligence Methods in the Risk Analysis of Damage to Masonry Buildings Subject to Long-Term Underground Mining Exploitation

This paper presents an advanced computational approach to assess the risk of damage to masonry buildings subjected to negative kinematic impacts of underground mining exploitation. The research goals were achieved using selected tools from the area of artificial intelligence (AI) methods. Ultimately, two models of damage risk assessment were built using the Naive Bayes classifier (NBC) and Bayesian Networks (BN). The first model was used to compare results obtained using the more computationally advanced Bayesian network methodology. In the case of the Bayesian network, the unknown Directed Acyclic Graph (DAG) structure was extracted using Chow-Liu’s Tree Augmented Naive Bayes (TAN-CL) algorithm. Thus, one of the methods involving Bayesian Network Structure Learning from data (BNSL) was implemented. The application of this approach represents a novel scientific contribution in the interdisciplinary field of mining and civil engineering. The models created were verified with respect to quality of fit to observed data and generalization properties. The connections in the Bayesian network structure obtained were also verified with respect to the observed relations occurring in engineering practice concerning the assessment of the damage intensity to masonry buildings in mining areas. This allowed evaluation of the model and justified the utility of the conducted research in the field of protection of mining areas. The possibility of universal application of the Bayesian network, both in the case of damage prediction and diagnosis of its potential causes, was also pointed out.

Examining Bayesian network modeling in identification of dangerous driving behavior

Traffic safety problems are still very serious and human factor is the one of most important factors affecting traffic crashes. Taking Next Generation Simulation (NGSIM) data as the research object, this study defines six control indicators and uses principal component analysis and K-means++ clustering methods to get the driving style of different drivers. Then use the Bayesian Networks Toolbox (BNT) and MCMC algorithm to realize the structure learning of Bayesian network. and parameter learning was completed through Netica software. Finally, the vehicle-based traffic crash risk model was created to conduct sensitivity analysis, posterior probability inference, and simulation data was used to detect the feasibility of the model. The results show that the Bayesian network modeling can not only express the relationship between the crash risk and various driving behaviors, but also dig out the inherent relationship between different influencing factors and investigate the causes of driving risks. The results will be beneficial to accurately identify and prevent risky driving behavior.

Causal constraint pruning for exact learning of Bayesian network structure

How to improve the efficiency of exact learning of the Bayesian network structure is a challenging issue. In this paper, four different causal constraints algorithms are added into score calculations to prune possible parent sets, improving state-of-the-art learning algorithms' efficiency. Experimental results indicate that exact learning algorithms can significantly improve the efficiency with only a slight loss of accuracy. Under causal constraints, these exact learning algorithms can prune about 70% possible parent sets and reduce about 60% running time while only losing no more than 2% accuracy on average. Additionally, with sufficient samples, exact learning algorithms with causal constraints can also obtain the optimal network. In general, adding max-min parents and children constraints has better results in terms of efficiency and accuracy among these four causal constraints algorithms.

Improved Local Search with Momentum for Bayesian Networks Structure Learning.

Bayesian Networks structure learning (BNSL) is a troublesome problem that aims to search for an optimal structure. An exact search tends to sacrifice a significant amount of time and memory to promote accuracy, while the local search can tackle complex networks with thousands of variables but commonly gets stuck in a local optimum. In this paper, two novel and practical operators and a derived operator are proposed to perturb structures and maintain the acyclicity. Then, we design a framework, incorporating an influential perturbation factor integrated by three proposed operators, to escape current local optimal and improve the dilemma that outcomes trap in local optimal. The experimental results illustrate that our algorithm can output competitive results compared with the state-of-the-art constraint-based method in most cases. Meanwhile, our algorithm reaches an equivalent or better solution found by the state-of-the-art exact search and hybrid methods.

A new PC-PSO algorithm for Bayesian network structure learning with structure priors

Bayesian network structure learning is the basis of parameter learning and Bayesian inference. However, it is a NP-hard problem to find the optimal structure of Bayesian networks because the computational complexity increases exponentially with the increasing number of nodes. Hence, numerous algorithms have been proposed to obtain feasible solutions, while almost all of them are of certain limits. In this paper, we adopt a heuristic algorithm to learn the structure of Bayesian networks, and this algorithm can provide a reasonable solution to combine the PC and Particle Swarm Optimization (PSO) algorithms. Moreover, we consider structure priors to improve the performance of our PC-PSO algorithm. Meanwhile, we utilize a new mutation operator called Uniform Mutation by Addition and Deletion (UMAD) and a crossover operator called Uniform Crossover. Experiments on different networks show that the approach proposed in this paper has achieved better Bayesian Information Criterion (BIC) scores than other algorithms.

A fixed structure learning automata‐based optimization algorithm for structure learning of Bayesian networks

AbstractOne of the useful knowledge representation tools, which can describe the joint probability distribution between some random variables with a graphical model and can be trained by a dataset, is the Bayesian network (BN). A BN is composed of a network structure and a conditional probability distribution table for each node. Discovering an optimal BN structure is an NP‐hard optimization problem that various meta‐heuristic algorithms are applied to solve this problem by researchers. The genetic algorithms, ant colony optimization, evolutionary programming, artificial bee colony, and bacterial foraging optimization are some of the meta‐heuristic methods to solve this problem using a dataset. Most of these methods are applying a scoring metric to generate the best network structure from a set of candidates. A Fixed Structure Learning Automata‐Based (FSLA‐B) algorithm is presented in this paper to solve the structure learning problem of BNs. There is a fixed structure learning automaton for each pair of vertices in the BN's graph structure in the proposed algorithm. The action of this automaton determines the presence and direction of an edge between the vertices. The proposed algorithm performs a guided search procedure using the FSLA and escapes from local optimums. Several datasets are utilised in this paper to evaluate the performance of the proposed algorithm. By performing various experiments, multiple meta‐heuristic algorithms are compared with the introduced new one. The obtained results represented that the proposed algorithm could produce competitive results and find the near‐optimal solution for the BN structure learning problem.

Efficient Bayesian Network Structure Learning via Parameterized Local Search on Topological Orderings

In Bayesian Network Structure Learning (BNSL), we are given a variable set and parent scores for each variable and aim to compute a DAG, called Bayesian network, that maximizes the sum of parent scores, possibly under some structural constraints. Even very restricted special cases of BNSL are computationally hard, and, thus, in practice heuristics such as local search are used. In a typical local search algorithm, we are given some BNSL solution and ask whether there is a better solution within some pre-defined neighborhood of the solution. We study ordering-based local search, where a solution is described via a topological ordering of the variables. We show that given such a topological ordering, we can compute an optimal DAG whose ordering is within inversion distance r in subexponential FPT time; the parameter r allows to balance between solution quality and running time of the local search algorithm. This running time bound can be achieved for BNSL without any structural constraints and for all structural constraints that can be expressed via a sum of weights that are associated with each parent set. We show that for other modification operations on the variable orderings, algorithms with an FPT time for r are unlikely. We also outline the limits of ordering-based local search by showing that it cannot be used for common structural constraints on the moralized graph of the network.

Turbocharging Treewidth-Bounded Bayesian Network Structure Learning

We present a new approach for learning the structure of a treewidth-bounded Bayesian Network (BN). The key to our approach is applying an exact method (based on MaxSAT) locally, to improve the score of a heuristically computed BN. This approach allows us to scale the power of exact methods—so far only applicable to BNs with several dozens of random variables—to large BNs with several thousands of random variables. Our experiments show that our method improves the score of BNs provided by state-of-the-art heuristic methods, often significantly.

Evaluation of Intelligent Air Defense algorithm based on Machine Learning

In the complex and rapidly changing combat environment, the enemy interference and sensor performance limitations and other factors lead to insufficient battlefield information. In order to make the UAV have the ability to carry out threat assessment under the condition of insufficient information, a Bayesian network (BN) threat assessment modeling method based on small data sets is proposed in this paper. Starting with the two problems of BN structure learning and parameter learning under the condition of small data set, using the constraint matrix obtained by Bootstrap method as the constraint item added to the score function, a BN structure learning algorithm based on small data set is proposed, and the BN parameter learning algorithm based on interval prior constraint is adopted. The simulation results show that, compared with the traditional BN learning algorithm, the BN learning algorithm proposed in this paper has higher accuracy and availability in UAV threat assessment modeling under the condition of small data set.

Separation and recovery Markov boundary discovery and its application in EEG-based emotion recognition

In a Bayesian network (BN), the Markov boundary (MB) presents the local causal structure around a target. Due to the interpretability and robustness, it has been widely applied to feature selection and BN structure learning. However, existing MB discovery algorithms might fail to identify some true positives, leading to poor performance in real-world applications. To tackle this issue, we introduce a two-phase-discovery strategy to search more true positives. Based on this strategy, we propose a more accurate and data-efficient algorithm, separation and recovery MB discovery algorithm (SRMB). SRMB first discovers an incomplete parent–child set and spouse set via an MB separation process, and then retrieves the ignored true positives via an MB recovery process, which further exploits a symmetry test to improve accuracy in unfaithful cases. Experiments on standard BN and real-world data sets demonstrate the effectiveness and superiority of SRMB in terms of MB discovery, BN structure learning, and feature selection. To demonstrate the superiority of SRMB in data with distribution shift, we further apply SRMB to EEG-based emotion recognition tasks, where distribution shift exists in multiple unstable sessions. We prove that the most predictive features are from Gamma/Beta frequency bands and are distributed at the lateral temporal area.

BNPA: An R package to learn path analysis input models from a data set semi-automatically using Bayesian networks

Epidemiologists constantly search for methodologies that help them better understand how diseases work. Populations urge these improvements to combat these diseases more effectively. The literature presents several authors defending the idea that epidemiologists should be able to develop causal models. In this area, the technique of structural equation models (SEM) has stood out in scientific research. Although SEM has been widely used in several research areas, it has been little explored by epidemiologists. Despite its evolution and efficiency, SEM has a gap in terms of discovering causalities. To fill this gap, this study developed an R package called BNPA, whose methodology joins the best of Bayesian network structural learning algorithms (BNSL) from data and path analysis (PA) a SEM subarea. The BNPA was built with pre-processing functions. Its main algorithm allows creating an input model to start the PA from a data set semi-automatically generating information to analyze the PA performance. An analysis of cardiovascular disease’s main predictors was performed using the BNPA with data from the Canadian Community Health Survey (CCHS). Multiple linear regression (MR) was used as a gold standard methodology; the results of BNPA matched 85% of MR results. In conclusion, BNPA is efficient and can benefit researchers, mainly novices, by enabling them to build PA models from data. Furthermore, statisticians and PA experts will have more time to support these researchers instead of creating an initial model.

Why BDeu? Regular Bayesian network structure learning with discrete and continuous variables

AbstractWe consider the problem of Bayesian network structure learning (BNSL) from data. In particular, we focus on the score‐based approach rather than the constraint‐based approach and address what score we should use for the purpose. The Bayesian Dirichlet equivalent uniform (BDeu) has been mainly used within the community of BNs (not outside of it). We know that for any model selection and any data, the fitter the data to a model, the more complex the model, and vice versa. However, recently, it was proven that BDeu violates regularity, which means that it does not balance the two factors, although it works satisfactorily (consistently) when the sample size is infinitely large. In addition, we claim that the merit of using the regular scores over the BDeu is that tighter bounds of pruning rules are available when we consider efficient BNSL. Finally, using experiments, we compare the performances of the procedures to examine the claim. (This paper is for review and gives a unified viewpoint from the recent progress on the topic.)This article is categorized under: Statistical Learning and Exploratory Methods of the Data Sciences > Modeling Methods Statistical and Graphical Methods of Data Analysis > Bayesian Methods and Theory

A practical analysis of sample complexity for structure learning of discrete dynamic Bayesian networks

ABSTRACT Discrete Dynamic Bayesian Network (dDBN) is used in many challenging causal modelling applications, such as human brain connectivity, due to its multivariate, non-deterministic, and nonlinear capability. Since there is not a ground truth for brain connectivity, the resulting model cannot be evaluated quantitatively. However, we should at least make sure that the best structure results for the used modelling approach and the data. Later, this result can be appreciated by further correlated literature of anatomy and physiology. Nearly all of the previously published studies rest on limited data, which brings doubt to the resulting structures. In theory, an immense number of samples is required, which is impossible to collect in practice. In this study, the appropriate number of data which makes a dDBN modelling trustable is searched by practical experiments and found to be for binary and ternary-valued networks, where K is the cardinality of the random variables and p is the maximum number of parents possibly present in the network. If a modelling approach satisfies this amount of data, we can at least say that the resulting structure is trustable.

Score-based Bayesian belief network structure learning in damage risk modelling of mining areas building development

The article describes the results of using the application of Bayesian approach, to create a decision support system to evaluate the risk of the damages to prefabricated reinforced concrete (RC) buildings, that have been exposed to the impacts from the industrial environment of mines in the form of spatial subsidence and tremors. This problem is important from the point of view of both structural safety and utility aspects. It concerns many buildings, in which the damage is often caused by the excessive consumption of thermal energy used for heating interiors or dehumidifying damp walls. The above issues make the subject extremely important from a technical, socio-economical, and energy point of view, therefore it can be categorised as sustainable development problem. The Bayesian belief network (BBN) is used to build a risk model that encompasses all the issues and variables that describe the damage process associated with the entire scope.The use of BBNs is an innovative approach in this interdisciplinary area, that encompasses civil engineering and mining environmental protection. There are two types of score-based methods for learning the structure of a BBN from data which will be applied and compared. The first is an iterative method for searching the space of potential solutions, based on the Tabu-search algorithm. The second approach used a method of stochastic searching for a space of structures (node orders) based on the global optimisation algorithm Simulated Annealing (SA). The analyses were performed in R, using the catnet and bnlearn packages. The basis for the study was a database relating to 129 prefabricated reinforced concrete buildings with a wall bearing structure, located in a copper mining area in Poland. Finally, there were eight damage intensity indexes (wui) obtained from using the above models when analysing the risk of damage to structural and finishing elements. The most effective method turned out to be using the Tabu-search algorithm with the assumed K2 criterion. As a result, the following damage indexes were obtained – 91.43%–97.14% correctly classified patterns for the training set, while for the test set – 79.17%–100%. The conducted research is interdisciplinary in the field of civil engineering and mining and environmental engineering, and the very issue of damage is also important from the socio-economical and energy point of view, especially for a large number of buildings.

Analysis of Bayesian Network Learning Techniques for a Hybrid Multi-objective Bayesian Estimation of Distribution Algorithm: a case study on MNK Landscape

This work investigates different Bayesian network structure learning techniques by thoroughly studying several variants of Hybrid Multi-objective Bayesian Estimation of Distribution Algorithm (HMOBEDA), applied to the MNK Landscape combinatorial problem. In the experiments, we evaluate the performance considering three different aspects: optimization abilities, robustness and learning efficiency. Results for instances of multi- and many-objective MNK-landscape show that, score-based structure learning algorithms appear to be the best choice. In particular, HMOBEDA $$_{k2}$$ was capable of producing results comparable with the other variants in terms of the runtime of convergence and the coverage of the final Pareto front, with the additional advantage of providing solutions that are less sensible to noise while the variability of the corresponding Bayesian network models is reduced.

Mutual-information-inspired heuristics for constraint-based causal structure learning

In constraint-based approaches to Bayesian network structure learning, when the assumption of orientation-faithfulness is violated, not only the correctness of edge orientation can be greatly degraded, the soaring cost of conditional independence testing also limits their applicability in learning very large causal networks. Inspired by the strong connection between the degree of mutual information shared by two variables and their conditional independence, we extend the PC-MI algorithm in two ways: (a) the Weakest Edge-First (WEF) strategy implemented in PC-MI is further integrated with Markov-chain consistency to reduce the number of independence testing and sustain the number of false positive edges in skeletal learning; (b) the Smaller Adjacency-Set (SAS) strategy is proposed and we prove that the Smaller Adjacency-Set captures sufficient information for determining whether an unshielded triple forms a v-structure. We have conducted experiments with both low-dimensional and high-dimensional data sets, and the results indicate that our MIIPC approach outperforms the state-of-the-art approaches in both the quality of learning and the execution time.

Large-scale empirical validation of Bayesian Network structure learning algorithms with noisy data

Numerous Bayesian Network (BN) structure learning algorithms have been proposed in the literature over the past few decades. Each publication makes an empirical or theoretical case for the algorithm proposed in that publication and results across studies are often inconsistent in their claims about which algorithm is ‘best’. This is partly because there is no agreed evaluation approach to determine their effectiveness. Moreover, each algorithm is based on a set of assumptions, such as complete data and causal sufficiency, and tend to be evaluated with data that conforms to these assumptions, however unrealistic these assumptions may be in the real world. As a result, it is widely accepted that synthetic performance overestimates real performance, although to what degree this may happen remains unknown. This paper investigates the performance of 15 state-of-the-art, well-established, or recent promising structure learning algorithms. We propose a methodology that applies the algorithms to data that incorporates synthetic noise, in an effort to better understand the performance of structure learning algorithms when applied to real data. Each algorithm is tested over multiple case studies, sample sizes, types of noise, and assessed with multiple evaluation criteria. This work involved learning approximately 10,000 graphs with a total structure learning runtime of seven months. In investigating the impact of data noise, we provide the first large scale empirical comparison of BN structure learning algorithms under different assumptions of data noise. The results suggest that traditional synthetic performance may overestimate real-world performance by anywhere between 10% and more than 50%. They also show that while score-based learning is generally superior to constraint-based learning, a higher fitting score does not necessarily imply a more accurate causal graph. The comparisons extend to other outcomes of interest, such as runtime, reliability, and resilience to noise, assessed over both small and large networks, and with both limited and big data. To facilitate comparisons with future studies, we have made all data, raw results, graphs and BN models freely available online.