Bayesian Network Learning Algorithm Research Articles

Feature Selection for Efficient Local-to-global Bayesian Network Structure Learning

Local-to-global learning approach plays an essential role in Bayesian network (BN) structure learning. Existing local-to-global learning algorithms first construct the skeleton of a DAG (directed acyclic graph) by learning the MB (Markov blanket) or PC (parents and children) of each variable in a dataset, then orient edges in the skeleton. However, existing MB or PC learning methods are often computationally expensive especially with a large-sized BN, resulting in inefficient local-to-global learning algorithms. To tackle the problem, in this article, we link feature selection with local BN structure learning and develop an efficient local-to-global learning approach using filtering feature selection. Specifically, we first analyze the rationale of the well-known Minimum-Redundancy and Maximum-Relevance (MRMR) feature selection approach for learning a PC set of a variable. Based on the analysis, we propose an efficient F2SL (feature selection-based structure learning) approach to local-to-global BN structure learning. The F2SL approach first employs the MRMR approach to learn the skeleton of a DAG, then orients edges in the skeleton. Employing independence tests or score functions for orienting edges, we instantiate the F2SL approach into two new algorithms, F2SL-c (using independence tests) and F2SL-s (using score functions). Compared to the state-of-the-art local-to-global BN learning algorithms, the experiments validated that the proposed algorithms in this article are more efficient and provide competitive structure learning quality than the compared algorithms.

BNSL GOBNILP algorithm in application to damage intensity prognostic system to RC multistorey residential buildings subjected to negative impacts of the industrial environment of mines

Paper presents a method of predicting the damage intensity of multifamily prefabricated RC buildings located in the range of impacts of the industrial environment of mines. The research studied the effects of mining impacts in the form of large-scale continuous ground deformation and mining tremors. For this purpose, in-situ data collected in the database was used. Additional information in the database included: structural and material features, quality of maintenance and durability of the analysed buildings. Finally, a predictive model represented by an optimal Bayesian network structure was created. To obtain the optimal topology of the DAG network structure, a Bayesian network learning algorithm from data GOBNILP was adapted. According to this algorithm, the optimisation process was carried out through Gurobi Optimizer. Thus, the conducted analyses represent the first time the above-mentioned approach has been applied in the interdisciplinary field of civil and environmental engineering. As part of the analyses, the available metric scores were verified and multiple network structures were examined according to their complexity. This made it possible to select the best Bayesian network in terms of generalizing the knowledge acquired while learning its structure. The possibility of using the obtained model for issues of diagnosis of damage causes and their prediction was also presented.

Critique of “A Parallel Framework for Constraint-Based Bayesian Network Learning via Markov Blanket Discovery” by SCC Team From Peking University

Ankit Srivastava et al. [1] proposed a parallel framework for Constraint-Based Bayesian Network (BN) Learning via Markov Blanket Discovery (referred to as ramBLe) and implemented it over three existing BN learning algorithms, namely, GS, IAMB and Inter-IAMB. As part of the Student Cluster Competition at SC21, we reproduce the computational efficiency of ramBLe on our assigned Oracle cluster. The cluster has 4x36 cores in total with 100 Gbps RoCE v2 support and is equipped with Centos-compatible Oracle Linux. Our experiments, covering the same three algorithms of ramBLe, evaluate its strong and weak scalability of the algorithms using real COVID-19 data sets. We verify part of the conclusions in the paper and propose our explanation of the differences.

Towards data fusion-based big data analytics for intrusion detection

ABSTRACT Intrusion detection is seen as the most promising way for computer security. It is used to protect computer networks against different types of attacks. The major problem in the literature is the classification of data into two main classes: normal and intrusion. To solve this problem, several approaches have been proposed but the problem of false alarms is still present. To provide a solution to this problem, we have proposed a new intrusion detection approach based on data fusion. The main objective of this work is to suggest an approach of data fusion-based Big Data analytics to detect intrusions; It is to build one dataset which combines various datasets and contains all the attack types. This research consists in merging the heterogeneous datasets and removing redundancy information using Big Data analytics tools: Hadoop/MapReduce and Neo4j. In the next step, machine learning algorithms are implemented for learning. The first algorithm, called SSDM (Semantically Similar Data Miner), uses fuzzy logic to generate association rules between the different item sets. The second algorithm, called K2, is a score-based greedy search algorithm for learning Bayesian networks from data. Experimentation results prove that – in both cases – data fusion contributes to having very good results.

Integrating unmanned and manned UAVs data network based on combined Bayesian belief network and multi-objective reinforcement learning algorithm

This paper presents and assesses the feasibility and potential of a novel concept: the operation of multiple Unmanned Aerial Vehicles (UAVs) commanded and supported by a manned “Tender” air vehicle carrying a pilot and flight manager(s). The “Tender” is equipped to flexibly and economically monitor and manage multiple diverse UAVs over otherwise inaccessible terrain through wireless communication. The proposed architecture enables operations and analysis supported by the means to detect, assess, and accommodate change and hazards on the spot with effective human observation and coordination. Further, this paper seeks to find the optimal trajectories for UAVs to collect data from sensors in a predefined continuous space. We formulate the path-planning problem for a cooperative, and a diverse swarm of UAVs tasked with optimizing multiple objectives simultaneously with the goal of maximizing accumulated data within a given flight time within cloud data processing constraints as well as minimizing the probable imposed risk during UAVs mission. The risk assessment model determines risk indicators using an integrated Specific Operation Risk Assessment—Bayesian belief network approach, while its resultant analysis is weighted through the analytic hierarchy process ranking model. To this end, as the problem is formulated as a convex optimization model, and we propose a low complexity multi-objective reinforcement learning (MORL) algorithm with a provable performance guarantee to solve the problem efficiently. We show that the MORL architecture can be successfully trained and allows each UAV to map each observation of the network state to an action to make optimal movement decisions. This proposed network architecture enables the UAVs to balance multiple objectives. Estimated MSE measures show that the algorithm produced decreasing errors in the learning process with increasing epoch number.

FROSTY: A High-Dimensional Scale-Free Bayesian Network Learning Method

We propose a scalable Bayesian network learning algorithm based on sparse Cholesky decomposition. Our approach only requires observational data and user-specified confidence level as inputs and can estimate networks with thousands of variables. The computational complexity of the proposed method is $O({p^{3}})$ for a graph with p vertices. Extensive numerical experiments illustrate the usefulness of our method with promising results. In simulation, the initial step in our approach also improves an alternative Bayesian network structure estimation method that uses an undirected graph as an input.

The FEDHC Bayesian Network Learning Algorithm

The paper proposes a new hybrid Bayesian network learning algorithm, termed Forward Early Dropping Hill Climbing (FEDHC), devised to work with either continuous or categorical variables. Further, the paper manifests that the only implementation of MMHC in the statistical software R is prohibitively expensive, and a new implementation is offered. Further, specifically for the case of continuous data, a robust to outliers version of FEDHC, which can be adopted by other BN learning algorithms, is proposed. The FEDHC is tested via Monte Carlo simulations that distinctly show that it is computationally efficient, and that it produces Bayesian networks of similar to, or of higher accuracy than MMHC and PCHC. Finally, an application of FEDHC, PCHC and MMHC algorithms to real data, from the field of economics, is demonstrated using the statistical software R.

A Bayesian network structural learning algorithm for calculating the failure probabilities of complex engineering systems with limited data

The failure of complex engineering systems is easy to lead to disastrous consequences. To prevent the failure, it is necessary to model complex engineering systems using probabilistic techniques with limited data which is a major feature of complex engineering systems. It is a good choice to perform such modeling using Bayesian network because of its advantages in probabilistic modeling. However, few Bayesian network structural learning algorithms are designed for complex engineering systems with limited data. Therefore, an algorithm for learning the Bayesian network structure of them should be developed. Based on the process of self-purification of water, a complex engineering system is segmented into three components according to the degree of difficulty in solving them. And then a Bayesian network learning algorithm with three components (TC), including PC algorithm, MIK algorithm which is originated by the paper through combining Mutual Information and K2 algorithm, and the Hill-Climbing method, is developed, i.e. TC algorithm. To verify its effectiveness, TC algorithm, K2 algorithm, and Max-Min Hill-Climbing are respectively used to learn Alarm network with different sizes of samples. The results imply that TC algorithm has the best performance. Finally, TC algorithm is applied to study tank spill accidents with 220 samples.

BigBraveBN: algorithm of structural learning for bayesian networks with a large number of nodes

Learning a Bayesian network is an NP-hard problem and with an increase in the number of nodes, classical algorithms for learning the structure of Bayesian networks become inefficient. In recent years, some methods and algorithms for learning Bayesian networks with a high number of nodes (more than 50) were developed. But these solutions have their disadvantages, for instance, they only operate one type of data (discrete or continuous) or their algorithm has been created to meet a specific nature of data (medical, social, etc.). The article presents a BigBraveBN algorithm for learning large Bayesian Networks with a high number of nodes (over 100). The algorithm utilizes the Brave coefficient that measures the mutual occurrence of instances in several groups. To form these groups, we use the method of nearest neighbours based on the Mutual information (MI) measure. In the experimental part of the article, we compare the performance of BigBraveBN to other existing solutions on multiple data sets both discrete and continuous. The experimental part also represents tests on real data. The aforementioned experimental results demonstrate the efficiency of the BigBraveBN algorithm in structure learning of Bayesian Networks.

Evaluation of Intelligent Air Defense algorithm based on Machine Learning

In the complex and rapidly changing combat environment, the enemy interference and sensor performance limitations and other factors lead to insufficient battlefield information. In order to make the UAV have the ability to carry out threat assessment under the condition of insufficient information, a Bayesian network (BN) threat assessment modeling method based on small data sets is proposed in this paper. Starting with the two problems of BN structure learning and parameter learning under the condition of small data set, using the constraint matrix obtained by Bootstrap method as the constraint item added to the score function, a BN structure learning algorithm based on small data set is proposed, and the BN parameter learning algorithm based on interval prior constraint is adopted. The simulation results show that, compared with the traditional BN learning algorithm, the BN learning algorithm proposed in this paper has higher accuracy and availability in UAV threat assessment modeling under the condition of small data set.

A Novel BN Learning Algorithm Based on Block Learning Strategy.

Learning accurate Bayesian Network (BN) structures of high-dimensional and sparse data is difficult because of high computation complexity. To learn the accurate structure for high-dimensional and sparse data faster, this paper adopts a divide and conquer strategy and proposes a block learning algorithm with a mutual information based K-means algorithm (BLMKM algorithm). This method utilizes an improved K-means algorithm to block the nodes in BN and a maximum minimum parents and children (MMPC) algorithm to obtain the whole skeleton of BN and find possible graph structures based on separated blocks. Then, a pruned dynamic programming algorithm is performed sequentially for all possible graph structures to get possible BNs and find the best BN by scoring function. Experiments show that for high-dimensional and sparse data, the BLMKM algorithm can achieve the same accuracy in a reasonable time compared with non-blocking classical learning algorithms. Compared to the existing block learning algorithms, the BLMKM algorithm has a time advantage on the basis of ensuring accuracy. The analysis of the real radar effect mechanism dataset proves that BLMKM algorithm can quickly establish a global and accurate causality model to find the cause of interference, predict the detecting result, and guide the parameters optimization. BLMKM algorithm is efficient for BN learning and has practical application value.

A New Scalable Bayesian Network Learning Algorithm with Applications to Economics

This paper proposes a new Bayesian network learning algorithm, termed PCHC, that is designed to work with either continuous or categorical data. PCHC is a hybrid algorithm that consists of the skeleton identification phase (learning the relationships among the variables) followed by the scoring phase that assigns the causal directions. Monte Carlo simulations clearly show that PCHC is dramatically faster, enjoys a nice scalability with respect to the sample size, and produces Bayesian networks of similar to, or of higher accuracy than, a competing state of the art hybrid algorithm. PCHC is finally applied to real data illustrating its performance and advantages.

Causal Learning From Predictive Modeling for Observational Data.

We consider the problem of learning structured causal models from observational data. In this work, we use causal Bayesian networks to represent causal relationships among model variables. To this effect, we explore the use of two types of independencies—context-specific independence (CSI) and mutual independence (MI). We use CSI to identify the candidate set of causal relationships and then use MI to quantify their strengths and construct a causal model. We validate the learned models on benchmark networks and demonstrate the effectiveness when compared to some of the state-of-the-art Causal Bayesian Network Learning algorithms from observational Data.

Efficient and accurate structural fusion of Bayesian networks

Bayesian Network (BN) fusion provides a precise theoretical framework for aggregating the graphical structure of a set of BNs into a consensus network. The fusion process depends on a total ordering of the variables, but both the problem of searching for an optimal consensus structure (according to the standard problem definition) as well as the one of looking for the optimal ordering are NP-hard.In this paper we start with this theoretical framework and extend it from a practical point of view. The two main methodological contributions we make are: (1) an adaptation of the well-known BN learning algorithm GES (Greedy Equivalence Search) to the case of having a set of BNs as input instead of data (we prove the correctness of the adapted algorithm, i.e., under certain standard assumptions the optimal solution for the BN fusion process is obtained); and (2) a heuristic method for identifying a suitable order of the variables, which allows us to obtain consensus BNs having (far) fewer edges than those obtained by using random orderings.Finally, we design several computational experiments to test our proposals. From the results, we can conclude that the consensus network can be efficiently obtained by using the proposed heuristic to compute the total order of the variables, with this DAG being very close to the optimal one.

Leveraging Bayesian networks and information theory to learn risk factors for breast cancer metastasis

BackgroundEven though we have established a few risk factors for metastatic breast cancer (MBC) through epidemiologic studies, these risk factors have not proven to be effective in predicting an individual’s risk of developing metastasis. Therefore, identifying critical risk factors for MBC continues to be a major research imperative, and one which can lead to advances in breast cancer clinical care. The objective of this research is to leverage Bayesian Networks (BN) and information theory to identify key risk factors for breast cancer metastasis from data.MethodsWe develop the Markov Blanket and Interactive risk factor Learner (MBIL) algorithm, which learns single and interactive risk factors having a direct influence on a patient’s outcome. We evaluate the effectiveness of MBIL using simulated datasets, and compare MBIL with the BN learning algorithms Fast Greedy Search (FGS), PC algorithm (PC), and CPC algorithm (CPC). We apply MBIL to learn risk factors for 5 year breast cancer metastasis using a clinical dataset we curated. We evaluate the learned risk factors by consulting with breast cancer experts and literature. We further evaluate the effectiveness of MBIL at learning risk factors for breast cancer metastasis by comparing it to the BN learning algorithms Necessary Path Condition (NPC) and Greedy Equivalent Search (GES).ResultsThe averages of the Jaccard index for the simulated datasets containing 2000 records were 0.705, 0.272, 0.228, and 0.147 for MBIL, FGS, PC, and CPC respectively. MBIL, NPC, and GES all learned that grade and lymph_nodes_positive are direct risk factors for 5 year metastasis. Only MBIL and NPC found that surgical_margins is a direct risk factor. Only NPC found that invasive is a direct risk factor. MBIL learned that HER2 and ER interact to directly affect 5 year metastasis. Neither GES nor NPC learned that HER2 and ER are direct risk factors.DiscussionThe results involving simulated datasets indicated that MBIL can learn direct risk factors substantially better than standard Bayesian network learning algorithms. An application of MBIL to a real breast cancer dataset identified both single and interactive risk factors that directly influence breast cancer metastasis, which can be investigated further.

Data-Driven Bayesian Network Learning: A Bi-Objective Approach to Address the Bias-Variance Decomposition

We present a novel bi-objective approach to address the data-driven learning problem of Bayesian networks. Both the log-likelihood and the complexity of each candidate Bayesian network are considered as objectives to be optimized by our proposed algorithm named Nondominated Sorting Genetic Algorithm for learning Bayesian networks (NS2BN) which is based on the well-known NSGA-II algorithm. The core idea is to reduce the implicit selection bias-variance decomposition while identifying a set of competitive models using both objectives. Numerical results suggest that, in stark contrast to the single-objective approach, our bi-objective approach is useful to find competitive Bayesian networks especially in the complexity. Furthermore, our approach presents the end user with a set of solutions by showing different Bayesian network and their respective MDL and classification accuracy results.

Robust Bayesian networks for low-quality data modeling and process monitoring applications

In this paper, a novel robust Bayesian network is proposed for process modeling with low-quality data. Since unreliable data can cause model parameters to deviate from the real distributions and make network structures unable to characterize the true causalities, data quality feature is utilized to improve the process modeling and monitoring performance. With a predetermined trustworthy center, the data quality measurement results can be evaluated through an exponential function with Mahalanobis distances. The conventional Bayesian network learning algorithms including structure learning and parameter learning are modified by the quality feature in a weighting form, intending to extract useful information and make a reasonable model. The effectiveness of the proposed method is demonstrated through TE benchmark process and a real industrial process.

Combining gene expression data and prior knowledge for inferring gene regulatory networks via Bayesian networks using structural restrictions.

Gene Regulatory Networks (GRNs) are known as the most adequate instrument to provide a clear insight and understanding of the cellular systems. One of the most successful techniques to reconstruct GRNs using gene expression data is Bayesian networks (BN) which have proven to be an ideal approach for heterogeneous data integration in the learning process. Nevertheless, the incorporation of prior knowledge has been achieved by using prior beliefs or by using networks as a starting point in the search process. In this work, the utilization of different kinds of structural restrictions within algorithms for learning BNs from gene expression data is considered. These restrictions will codify prior knowledge, in such a way that a BN should satisfy them. Therefore, one aim of this work is to make a detailed review on the use of prior knowledge and gene expression data to inferring GRNs from BNs, but the major purpose in this paper is to research whether the structural learning algorithms for BNs from expression data can achieve better outcomes exploiting this prior knowledge with the use of structural restrictions. In the experimental study, it is shown that this new way to incorporate prior knowledge leads us to achieve better reverse-engineered networks.

Discovery of Causal Paths in Cardiorespiratory Parameters: A Time-Independent Approach in Elite Athletes.

Training of elite athletes requires regular physiological and medical monitoring to plan the schedule, intensity and volume of training, and subsequent recovery. In sports medicine, ECG-based analyses are well-established. However, they rarely consider the correspondence of respiratory and cardiac activity. Given such mutual influence, we hypothesize that athlete monitoring might be developed with causal inference and that detailed, time-related techniques should be preceded by a more general, time-independent approach that considers the whole group of participants and parameters describing whole signals. The aim of this study was to discover general causal paths among cardiac and respiratory variables in elite athletes in two body positions (supine and standing), at rest. ECG and impedance pneumography signals were obtained from 100 elite athletes. The mean heart rate, the root-mean-square difference of successive RR intervals (RMSSD), its natural logarithm (lnRMSSD), the mean respiratory rate (RR), the breathing activity coefficients, and the resulting breathing regularity (BR) were estimated. Several causal discovery frameworks were applied, comprising Generalized Correlations (GC), Causal Additive Modeling (CAM), Fast Greedy Equivalence Search (FGES), Greedy Fast Causal Inference (GFCI), and two score-based Bayesian network learning algorithms: Hill-Climbing (HC) and Tabu Search. The discovery of cardiorespiratory paths appears ambiguous. The main, still mild, rules best supported by data are: for supine - tidal volume causes heart activity variation, which causes average heart activity, which causes respiratory timing; and for standing - normalized respiratory activity variation causes average heart activity. The presented approach allows data-driven and time-independent analysis of elite athletes as a particular population, without considering prior knowledge. However, the results seem to be consistent with the medical background. Causality inference is an interesting mathematical approach to the analysis of biological responses, which are complex. One can use it to profile athletes and plan appropriate training. In the next step, we plan to expand the study using time-related causality analyses.

Bayesian Network Learning with the PC Algorithm: An Improved and Correct Variation

ABSTRACTPC is a prototypical constraint-based algorithm for learning Bayesian networks, a special case of directed acyclic graphs. An existing variant of it, in the R package pcalg, was developed to make the skeleton phase order independent. In return, it has notably increased execution time. In this paper, we clarify that the PC algorithm the skeleton phase of PC is indeed order independent. The modification we propose outperforms pcalg’s variant of the PC in terms of returning correct networks of better quality as is less prone to errors and in some cases it is a lot more computationally cheaper. In addition, we show that pcalg’s variant does not return valid acyclic graphs.