Learning Bayesian Networks Research Articles

Efficient computational strategies to learn the structure of probabilistic graphical models of cumulative phenomena

Structural learning of Bayesian Networks (BNs) is a NP-hard problem, which is further complicated by many theoretical issues, such as the I-equivalence among different structures. In this work, we focus on a specific subclass of BNs, named Suppes-Bayes Causal Networks (SBCNs), which include specific structural constraints based on Suppes’ probabilistic causation to efficiently model cumulative phenomena. Here we compare the performance, via extensive simulations, of various state-of-the-art search strategies, such as local search techniques and Genetic Algorithms, as well as of distinct regularization methods. The assessment is performed on a large number of simulated datasets from topologies with distinct levels of complexity, various sample size and different rates of errors in the data. Among the main results, we show that the introduction of Suppes’ constraints dramatically improve the inference accuracy, by reducing the solution space and providing a temporal ordering on the variables. We also report on trade-offs among different search techniques that can be efficiently employed in distinct experimental settings. This manuscript is an extended version of the paper “Structural Learning of Probabilistic Graphical Models of Cumulative Phenomena” presented at the 2018 International Conference on Computational Science [1].

Maintaining of internal consistency of algebraic Bayesian networks with linear and stellate structure

Subject of Research. When working with algebraic Bayesian networks, it is necessary to ensure their correctness in terms of the consistency of the probability estimates of their constituent elements.There are several approaches to automating the maintenance of consistency, characterized by their computational complexity (execution time). This complexity depends on the network structure and the chosen type of consistency. The time for internal consistency maintenance in algebraic Bayesian networks with linear and stellate structure is compared with the time for consistency maintenance of a knowledge pattern covering such networks. The comparison is based on statistical estimates. Method.The essence of the method lies in reducing the number of variables and conditions in linear programming problems which solution ensures the maintenance of internal consistency. An experiment was carried out demonstrating the differences between the time of consistency maintenance for different algebraic Bayesian networks with a global structure. Main Results. An improved version of the algorithm for internal consistency maintenance is presented.Solvable linear programming problems are simplified in comparison with the previous version of the algorithm. Two theorems are formulated and proved, refining the estimates of the number of variables and conditions in the linear programming problems to be solved, as well as the number of the problems themselves. An experiment is performed, which showed that the proposed software implementation of internal consistency maintenanceis superior in working time to software implementation of the consistency maintenanceof a complete knowledge pattern. Practical Relevance. The results obtained can be applied in machine learning of algebraic Bayesian networks (including the synthesis of their global structures). The proposed method provides optimal synthesis of global network structures for which it is enough to use the maintenance of internal consistency during learning and further network processing. Owing to the method application these processes will have acceptable computational complexity.

Discovery of Causal Paths in Cardiorespiratory Parameters: A Time-Independent Approach in Elite Athletes.

Training of elite athletes requires regular physiological and medical monitoring to plan the schedule, intensity and volume of training, and subsequent recovery. In sports medicine, ECG-based analyses are well-established. However, they rarely consider the correspondence of respiratory and cardiac activity. Given such mutual influence, we hypothesize that athlete monitoring might be developed with causal inference and that detailed, time-related techniques should be preceded by a more general, time-independent approach that considers the whole group of participants and parameters describing whole signals. The aim of this study was to discover general causal paths among cardiac and respiratory variables in elite athletes in two body positions (supine and standing), at rest. ECG and impedance pneumography signals were obtained from 100 elite athletes. The mean heart rate, the root-mean-square difference of successive RR intervals (RMSSD), its natural logarithm (lnRMSSD), the mean respiratory rate (RR), the breathing activity coefficients, and the resulting breathing regularity (BR) were estimated. Several causal discovery frameworks were applied, comprising Generalized Correlations (GC), Causal Additive Modeling (CAM), Fast Greedy Equivalence Search (FGES), Greedy Fast Causal Inference (GFCI), and two score-based Bayesian network learning algorithms: Hill-Climbing (HC) and Tabu Search. The discovery of cardiorespiratory paths appears ambiguous. The main, still mild, rules best supported by data are: for supine - tidal volume causes heart activity variation, which causes average heart activity, which causes respiratory timing; and for standing - normalized respiratory activity variation causes average heart activity. The presented approach allows data-driven and time-independent analysis of elite athletes as a particular population, without considering prior knowledge. However, the results seem to be consistent with the medical background. Causality inference is an interesting mathematical approach to the analysis of biological responses, which are complex. One can use it to profile athletes and plan appropriate training. In the next step, we plan to expand the study using time-related causality analyses.

New Method for Bayesian Network Learning

This paper presents a new method for learning the structure of Bayesian Networks. Broadly speaking, we leverage the Branch and Bound (B&B) to derive the best Directed Acyclic Graphs (DAGs) that describes the structure of the network. Our contribution consists in introducing two main heuristics: the first one allows the selection of the graph that has the best score among those that contain less cycles, the second one eliminates the shortest cycle from the selected graph; it aims to reduce the number of explored nodes. Our experimental study asserts that the suggested proposal improves the results for multiple data sets. These facts are confirmed by the reduction of the computation time and the memory overhead.

Analysis of Influence Factors for Learning Outcomes with Bayesian Network

This study identifies and analyzes the influence factors for learning outcomes at a university with a Bayesian network. It is based on a fact-finding survey on university student life and learning. Suitable constraints and a score metric for the Bayesian network learning are determined via cross-validation, and the learning outcome variables are categorized into subsets according to six abilities: cooperativeness, expressiveness, foreign language, collecting and organizing information, logical thinking, and sociability. The learned network suggests that two to seven factors influence each ability. In addition, it is confirmed that the probability distributions of all most of the identified factors shift to high agreement/experience levels, as self-knowledge levels for the acquired abilities increase, i.e., positive effects exist for most factors for each identified ability.

Distributed Bayesian networks reconstruction on the whole genome scale.

BackgroundBayesian networks are directed acyclic graphical models widely used to represent the probabilistic relationships between random variables. They have been applied in various biological contexts, including gene regulatory networks and protein–protein interactions inference. Generally, learning Bayesian networks from experimental data is NP-hard, leading to widespread use of heuristic search methods giving suboptimal results. However, in cases when the acyclicity of the graph can be externally ensured, it is possible to find the optimal network in polynomial time. While our previously developed tool BNFinder implements polynomial time algorithm, reconstructing networks with the large amount of experimental data still leads to computations on single CPU growing exceedingly.ResultsIn the present paper we propose parallelized algorithm designed for multi-core and distributed systems and its implementation in the improved version of BNFinder—tool for learning optimal Bayesian networks. The new algorithm has been tested on different simulated and experimental datasets showing that it has much better efficiency of parallelization than the previous version. BNFinder gives comparable results in terms of accuracy with respect to current state-of-the-art inference methods, giving significant advantage in cases when external information such as regulators list or prior edge probability can be introduced, particularly for datasets with static gene expression observations.ConclusionsWe show that the new method can be used to reconstruct networks in the size range of thousands of genes making it practically applicable to whole genome datasets of prokaryotic systems and large components of eukaryotic genomes. Our benchmarking results on realistic datasets indicate that the tool should be useful to a wide audience of researchers interested in discovering dependencies in their large-scale transcriptomic datasets.

Bayesian Network Learning with the PC Algorithm: An Improved and Correct Variation

ABSTRACTPC is a prototypical constraint-based algorithm for learning Bayesian networks, a special case of directed acyclic graphs. An existing variant of it, in the R package pcalg, was developed to make the skeleton phase order independent. In return, it has notably increased execution time. In this paper, we clarify that the PC algorithm the skeleton phase of PC is indeed order independent. The modification we propose outperforms pcalg’s variant of the PC in terms of returning correct networks of better quality as is less prone to errors and in some cases it is a lot more computationally cheaper. In addition, we show that pcalg’s variant does not return valid acyclic graphs.

Gene Regulatory Relationship Mining Using Improved Three-Phase Dependency Analysis Approach.

How to mine the gene regulatory relationship and construct gene regulatory network (GRN) is of utmost interest within the whole biological community, however, which has been consistently a challenging problem since the tremendous complexity in cellular systems. In present work, we construct gene regulatory network using an improved three-phase dependency analysis algorithm (TPDA) Bayesian network learning method, which includes the steps of Drafting, Thickening, and Thinning. In order to solve the problem of learning result is not reliable due to the high order conditional independence test, we use the entropy estimation approach of Gaussian kernel probability density estimator to calculate the (conditional) mutual information between genes. The experiment on the public benchmark data sets show the improved method outperforms the other nine kinds of Bayesian network learning methods when to process the data with large sample size, with small number of discrete values, and the frequency of different discrete values is about same. In addition, the improved TPDA method was further applied on a real large gene expression data set on RNA-seq from a global collection with 368 elite maize inbred lines. Experiment results show it performs better than the original TPDA method and the other nine kinds of Bayesian network learning algorithms significantly.

Reducing COPD Readmissions: A Causal Bayesian Network Model

This letter introduces a causal Bayesian network model to study readmissions reduction for chronic obstructive pulmonary disease (COPD) patients. The model employs a Bayesian network learning method and adopts domain knowledge. Using this model, we analyze the impacts of critical variables on a patient's readmission risk by the manipulation of such variables. Through this analysis, effective intervention options to reduce readmission can be identified, which can provide a quantitative tool for designing personalized interventions to reduce COPD readmissions.

Expert systems in assessing the construction process safety taking account of the risk of disturbances

The objective of the paper is to present the issue of safety management during the construction process. Threats in the form of disturbances may occur in the preparatory phase, during the execution of the construction project and also during its operational use. The article presents the concept of applying the methodology based, among others, on Learning Bayesian Networks, Artificial Neural Networks and Support Vector Machine, which can be used for building a system for diagnostic and decision-making support at each stage of the construction process. The use of expert systems when it comes to making choices related to construction issues can bring many benefits to decision-makers, as it reduces the risk of taking a wrong decision, and, thus, increases the construction process safety.

Vulnerability Characteristic Analysis of Urban Group Buildings Based on the Learning and Reasoning of Bayesian Network

This paper takes the building in Huizhou district of Guangdong Province as the research object, selecting the building with typical destructive features as samples. First of all, through the comparative analysis of seismic hazard factors of the sample building such as building structure types, building floors and years of construction after structure stress research, Bayesian network of structural vulnerability characteristics analysis of buildings based on Python is constructed and then the fitting prediction is performed. The method solves errors caused by the correlation of the index in the traditional prediction process, combines the prior information and the distribution characteristics of the training samples effectively. Finally, the method is verified via typical characteristic data of Huizhou and it compares with traditional Bayesian earthquake prediction methods, proving the advantages and reliability of the method. Use GIS to image related data.

An improved bayesian network intrusion detection algorithm based on deep learning

Based on the preprocessing of training data sets, this paper adopts the convolution neural network technology to realize the reduction target of the redundant attribute of the data set. An improved Bayesian network intrusion detection algorithm based on deep learning is proposed. In this algorithm, sliding window technique and relative Euclidean distance are defined, and the structure updating and parameter learning of Bayesian networks are adaptively carried out on the basis of computing mutual information between attributes. Experimental results show that the new algorithm can effectively improve the computational efficiency and the accuracy of intrusion detection.

A quantitative assessment model for IT system solution selection with Bayesian network for enterprise architecture decisions

Purpose- This research proposes a quantitative assessment model for ICT System Solution Selection. Methodology- For large Enterprise Organizations, ICT Projects may have two alternative solutions such as centralized or distributed ICT Systems. For evaluating these options with Enteprise Architecture principles and frameworks and also bringing expert knowledge in decision support, “Quantitative Assessment Model Survey for IT Solution Selection” study is conducted among ICT professionals of various industrial sectors. To build the model for centralized or distributed ICT system solutions, the influential relations between the surveyed EA components and principles are identified with Structural Learning of Bayesian Network to make the relations of the important EA elements visible and quantifiable. Findings- Using data from our survey, adaptable quantitative model which enables a Bayesian network across organized EA components is developed. Conclusion- The study shows relationships between EA components with experts’ feedbacks which may help practitioners and decision makers to reach a common model for ICT solution evaluation.

Learning Bayesian networks based on bi-velocity discrete particle swarm optimization with mutation operator

Abstract The problem of structures learning in Bayesian networks is to discover a directed acyclic graph that in some sense is the best representation of the given database. Score-based learning algorithm is one of the important structure learning methods used to construct the Bayesian networks. These algorithms are implemented by using some heuristic search strategies to maximize the score of each candidate Bayesian network. In this paper, a bi-velocity discrete particle swarm optimization with mutation operator algorithm is proposed to learn Bayesian networks. The mutation strategy in proposed algorithm can efficiently prevent premature convergence and enhance the exploration capability of the population. We test the proposed algorithm on databases sampled from three well-known benchmark networks, and compare with other algorithms. The experimental results demonstrate the superiority of the proposed algorithm in learning Bayesian networks.

An improved evolutionary approach-based hybrid algorithm for Bayesian network structure learning in dynamic constrained search space

Learning Bayesian network (BN) structures from data is a NP-hard problem due to the vastness of the solution space. To address this issue, hybrid approaches that integrate the constraint-based (CB) method and the score-and-search (SS) method have been developed in the literature, but when the constrained search space is fixed and inaccurate, it is very likely to lose the optimal solution, leading to low learning accuracy. Besides, due to the randomness and uncertainty of the search, it is difficult to preserve the superiority of the structures, resulting in low learning efficiency. Therefore, we propose a novel hybrid algorithm based on an improved evolutionary approach to explore BN structure with highest matching degree of data set in dynamic constrained search space. The proposed algorithm involves two phases, namely the CB phase and the SS phase. In the CB phase, the mutual information is utilized as the restriction to limit the search space, and a binding parameter is introduced to the novel encoding scheme so that the search space can be dynamically changed in the evolutionary process. In the SS phase, a new operator is developed to pass on the excellent genes from generation to generation, and an update principle for the binding parameter is exploited for the dynamic selection of the search space. We conduct the comparative experiments on the benchmark network data sets and provide performance and applicability analysis of our proposed method. The experimental results show that the new algorithm is effective in learning the BN structures.

An improved constraint-based Bayesian network learning method using Gaussian kernel probability density estimator

Bayesian network has powerful expression and reasoning ability of uncertainty knowledge. At present, it is widely used in many fields, such as fault diagnosis, medical diagnosis and prediction, financial analysis and prediction, decision support, etc. How to construct the Bayesian network effectively is an important research problem in the area of data mining. The constraint-based learning algorithm is commonly used for the construction of Bayesian network (BNs), such as Fast Incremental Association (Fast-IAMB) and Max-Min Parents & Children (MMPC), etc. At present, the Chi square test and correlation coefficient are often used to calculate the correlation between variables in these algorithms. But the calculation accuracy of these methods is not high, thus to influence the effectiveness of the constraint-based BNs learning algorithms. On the basis of the entropy estimation approach of Gaussian kernel probability density estimator, this paper uses the (conditional) mutual information to calculate the correlation between different variables and thus to improve the learning accuracy of the constraint-based BNs learning algorithms. The concrete calculation process about the data type of discrete, continuous and mixed has been described separately. Experiment simulations on the Alarm network prove that our algorithm can improve the learning accuracy of the existing three kinds of constraint-based BNs learning algorithms without affecting the learning efficiency. It also has better learning effect when to process the concrete and continuous data with sparsity. The method in this work can be used to learn more accurate Bayesian network, and then served the research of artificial intelligence, expert system decision making, etc.

Using machine learning and a data-driven approach to identify the small fatigue crack driving force in polycrystalline materials

The propagation of small cracks contributes to the majority of the fatigue lifetime for structural components. Despite significant interest, criteria for the growth of small cracks, in terms of the direction and speed of crack advancement, have not yet been determined. In this work, a new approach to identify the microstructurally small fatigue crack driving force is presented. Bayesian network and machine learning techniques are utilized to identify relevant micromechanical and microstructural variables that influence the direction and rate of the fatigue crack propagation. A multimodal dataset, combining results from a high-resolution 4D experiment of a small crack propagating in situ within a polycrystalline aggregate and crystal plasticity simulations, is used to provide training data. The relevant variables form the basis for analytical expressions thus representing the small crack driving force in terms of a direction and a rate equation. The ability of the proposed expressions to capture the observed experimental behavior is quantified and compared to the results directly from the Bayesian network and from fatigue metrics that are common in the literature. Results indicate that the direction of small crack propagation can be reliably predicted using the proposed analytical model and compares more favorably than other fatigue metrics.

Network Modeling of microRNA-mRNA Interactions in Neuroblastoma Tumorigenesis Identifies miR-204 as a Direct Inhibitor of MYCN.

Neuroblastoma is a pediatric cancer of the sympathetic nervous system where MYCN amplification is a key indicator of poor prognosis. However, mechanisms by which MYCN promotes neuroblastoma tumorigenesis are not fully understood. In this study, we analyzed global miRNA and mRNA expression profiles of tissues at different stages of tumorigenesis from TH-MYCN transgenic mice, a model of MYCN-driven neuroblastoma. On the basis of a Bayesian learning network model in which we compared pretumor ganglia from TH-MYCN+/+ mice to age-matched wild-type controls, we devised a predicted miRNA-mRNA interaction network. Among the miRNA-mRNA interactions operating during human neuroblastoma tumorigenesis, we identified miR-204 as a tumor suppressor miRNA that inhibited a subnetwork of oncogenes strongly associated with MYCN-amplified neuroblastoma and poor patient outcome. MYCN bound to the miR-204 promoter and repressed miR-204 transcription. Conversely, miR-204 directly bound MYCN mRNA and repressed MYCN expression. miR-204 overexpression significantly inhibited neuroblastoma cell proliferation in vitro and tumorigenesis in vivo Together, these findings identify novel tumorigenic miRNA gene networks and miR-204 as a tumor suppressor that regulates MYCN expression in neuroblastoma tumorigenesis.Significance: Network modeling of miRNA-mRNA regulatory interactions in a mouse model of neuroblastoma identifies miR-204 as a tumor suppressor and negative regulator of MYCN. Cancer Res; 78(12); 3122-34. ©2018 AACR.

Bayesian-based conflict conversion path discovery for waste management policy implementation in China

PurposeThis study aims to analyze reform path for waste management policy implementation. With reference to the Bayesian theory, this study provides a dynamic policy conversion method through various context settings. Furthermore, this study attempts to present an empirical research paradigm.Design/methodology/approachMatland’s “ambiguity-conflict model” is applied to explain the problems and reform paths of China’s waste management policy implementation. Integrating structure discovery and bibliometrics into qualitative analysis, this study used search data from literature search engine with specific themes to achieve structure learning of Bayesian network with key factors refined in waste management policy.FindingsThe results show that China’s waste management policy implementation belongs to symbolic implementation with high ambiguity and high conflict. Four basic conversion paths for the waste management policy are proposed, which are classified by length and stability. Then, it is possible to locate the factors, paths and types of policy implementation through involvement analysis with features of each path and each district of policy implementation. Public education holds direct but unstable impact on waste management. Economic incentives hold continuous but gradually diminishing impact. Perceived policy effectiveness plays the crucial role like a central bridge. Resident conditions have a positive impact, which could be enhanced through economic development of China. The impact of subjective norm on waste management is not significant. But subjective norm has the potential breakthrough for solving stagnation of waste classification policy. However, the impacts from each factor may change along with economy growth and technology innovation.Originality/valueThis study uses the “ambiguity-conflict model” to position China’s waste classification policy and suggests that structure discovery methods help understand feasible reform paths for reform policy. The integration of theoretical analysis and quantitative simulation can achieve a comprehensive analysis of problems and solutions in waste management policy implementation of China. Promotion and education, economic incentives, perceived value, behavior control, subjective norm, perceived policy effectiveness, informal waste recycling system and residential conditions are explored as key factors for waste classification policy implementation as a representative in waste management policy. The role of each key factor and features of each conversion paths are integrated to position reform paths in the ambiguity-conflict model. This work helps to explain the stagnation of waste management policy implementation from the perspective of dynamic structure evolution, and some specific suggestions to get out of stagnation are proposed.

Approximate structure learning for large Bayesian networks

We present approximate structure learning algorithms for Bayesian networks. We discuss the two main phases of the task: the preparation of the cache of the scores and structure optimization, both with bounded and unbounded treewidth. We improve on state-of-the-art methods that rely on an ordering-based search by sampling more effectively the space of the orders. This allows for a remarkable improvement in learning Bayesian networks from thousands of variables. We also present a thorough study of the accuracy and the running time of inference, comparing bounded-treewidth and unbounded-treewidth models.