Learning Bayesian Networks Research Articles

Learning the Parameters of Bayesian Networks from Uncertain Data

The creation of Bayesian networks often requires the specification of a large number of parameters, making it highly desirable to be able to learn these parameters from historical data. In many cases, such data has uncertainty associated with it, including cases in which this data comes from unstructured analysis or from sensors. When creating diagnosis networks, for example, unstructured analysis algorithms can be run on the historical text descriptions or images of previous cases so as to extract data for learning Bayesian network parameters, but such derived data has inherent uncertainty associated with it due to the nature of such algorithms. Because of the inability of current Bayesian network parameter learning algorithms to incorporate such uncertainty, common approaches either ignore this uncertainty, thus reducing the resulting accuracy, or completely disregard such data. We present an approach for learning Bayesian network parameters that explicitly incorporates such uncertainty, and which is a natural extension of the Bayesian network formalism. We present a generalization of the Expectation Maximization parameter learning algorithm that enables it to handle any historical data with likelihood-evidence-based uncertainty, as well as an empirical validation demonstrating the improved accuracy and convergence enabled by our approach. We also prove that our extended algorithm maintains the convergence and correctness properties of the original EM algorithm, while explicitly incorporating data uncertainty in the learning process.

Improving Causal Discovery By Optimal Bayesian Network Learning

Many widely-used causal discovery methods such as Greedy Equivalent Search (GES), although with asymptotic correctness guarantees, have been reported to produce sub-optimal solutions on finite data, or when the causal faithfulness condition is violated. The constraint-based procedure with Boolean satisfiability (SAT) solver, and the recently proposed Sparsest Permutation (SP) algorithm have shown superb performance, but currently they do not scale well. In this work, we demonstrate that optimal score-based exhaustive search is remarkably useful for causal discovery: it requires weaker conditions to guarantee asymptotic correctness, and outperforms well-known methods including PC, GES, GSP, and NOTEARS. In order to achieve scalability, we also develop an approximation algorithm for larger systems based on the A* method, which scales up to 60+ variables and obtains better results than existing greedy algorithms such as GES, MMHC, and GSP. Our results illustrate the risk of assuming the faithfulness assumption, the advantages of exhaustive search methods, and the limitations of greedy search methods, and shed light on the computational challenges and techniques in scaling up to larger networks and handling unfaithful data.

Evaluation of Intelligent Air Defense algorithm based on Machine Learning

In the complex and rapidly changing combat environment, the enemy interference and sensor performance limitations and other factors lead to insufficient battlefield information. In order to make the UAV have the ability to carry out threat assessment under the condition of insufficient information, a Bayesian network (BN) threat assessment modeling method based on small data sets is proposed in this paper. Starting with the two problems of BN structure learning and parameter learning under the condition of small data set, using the constraint matrix obtained by Bootstrap method as the constraint item added to the score function, a BN structure learning algorithm based on small data set is proposed, and the BN parameter learning algorithm based on interval prior constraint is adopted. The simulation results show that, compared with the traditional BN learning algorithm, the BN learning algorithm proposed in this paper has higher accuracy and availability in UAV threat assessment modeling under the condition of small data set.

Learning Bayesian networks using A* search with ancestral constraints

When using a Bayesian network to model a practical problem, weak prior knowledge projected as ancestral constraints is necessary. However, it is difficult to directly utilize these non-decomposable constraints using search strategies based on the decomposable score. In this study, we attempt to solve this problem by conducting an implicate path-space search graph and driving the A* algorithm, which is used to obtain the globally optimal solution satisfying the given constraints. We use a maximum covering principle to provide useful pruning rules based on these constraints in the new framework. Moreover, we improve the simple heuristic and the static k-cycle conflict heuristic to adapt to ancestral constraints. We theoretically prove that the new heuristic functions remain admissible and consistent. Our experiments demonstrate that the proposed framework with the new heuristic functions significantly reduces the space complexity of A* search compared with state-of-the-art frameworks, such as Bayesian network graphs and equivalent class trees, when integrating ancestral constraints.

A data-driven Bayesian network learning method for process fault diagnosis

This paper presents a data-driven methodology for fault detection and diagnosis (FDD) by integrating the principal component analysis (PCA) with the Bayesian network (BN). Though the integration of PCA-BN for FDD purposes has been studied in the past, the present work makes two contributions for process systems. First, the application of correlation dimension (CD) to select principal components (PCs) automatically. Second, the use of Kullback-Leibler divergence (KLD) and copula theory to develop a data-based BN learning technique. The proposed method uses a combination of vine copula and Bayes’ theorem (BT) to capture nonlinear dependence of high-dimensional process data which eliminates the need for discretization of continuous data. The data-driven integrated PCA-BN framework has been applied to two processing systems. Performance of the proposed methodology is compared with the independent component analysis (ICA), kernel principal component analysis (KPCA), kernel independent component analysis (KICA), and their integrated frameworks with the BN. The comparative study suggests that the proposed framework provides superior performance.

Transfer learning of Bayesian network for measuring QoS of virtual machines

The Quality of Service (QoS) of virtual machines (VMs) are ensured through the Service Level Agreements (SLAs) signed between the consumers and the cloud providers. A main way to avoid the SLAs violation is to analyze the relationships among the multiple VM-related features and then measure the QoS of VMs accurately. Therefore, we first propose to construct a QoS Bayesian Network (QBN), so as to quantify the uncertain dependencies among the VM-related features and then measure the QoS of VMs effectively. Moreover, we show that the dynamical changes of hardware\software setting or the different types of loads will affect the measurement decisions of QBN. Thus, we further resort to the instance-based transfer learning and then propose a novel QBN updating method (QBNtransfer). QBNtransfer re-weights the constantly updated data instances, and then combine the Maximum Likelihood Estimation and the hill-climbing methods to revise the parameters and structures of QBN accordingly. The experiments conducted on the Alibaba published datasets and the benchmark running results on our simulated platform have shown that the QBN can measure the QoS of VMs accurately and QBNtransfer can update the QBN effectively.

Application of machine learning algorithms for the evaluation of seismic soil liquefaction potential

This study investigates the performance of four machine learning (ML) algorithms to evaluate the earthquake-induced liquefaction potential of soil based on the cone penetration test field case history records using the Bayesian belief network (BBN) learning software Netica. The BBN structures that were developed by ML algorithms-K2, hill climbing (HC), tree augmented naive (TAN) Bayes, and Tabu search were adopted to perform parameter learning in Netica, thereby fixing the BBN models. The performance measure indexes, namely, overall accuracy (OA), precision, recall, F-measure, and area under the receiver operating characteristic curve, were used to evaluate the training and testing BBN models’ performance and highlight the capability of the K2 and TAN Bayes models over the Tabu search and HC models. The sensitivity analysis results showed that the cone tip resistance and vertical effective stress are the most sensitive factors, whereas the mean grain size is the least sensitive factor in the prediction of seismic soil liquefaction potential. The results of this study can provide theoretical support for researchers in selecting appropriate ML algorithms and improving the predictive performance of seismic soil liquefaction potential models.

A decision support system based on machined learned Bayesian network for predicting successful direct sales marketing

This paper proposes a decision support system based on a machine-learned Bayesian network (BN) to predict the success rate of telemarketing calls for long-term bank deposits. Telemarketing is one of the most common interactive techniques of direct marketing, widely used by financial institutions such as banks to sell long-term deposits. In this study, we develop a BN model that predicts the likelihood that a potential client subscribes to a long-term deposit, which is considered an output variable. The causal relationship among client attributes and outcomes has been identified using the augmented Naïve Bayes approach, a well-known supervised learning algorithm. The impact of each client's attribute on the likelihood of subscribing is predicted. Further, we carry out multiple simulation scenarios using BN’s unique features (forward and backward propagation) to provide more in-depth discussions and analysis on predicting the likelihood of subscription for clients with particular characteristics.

A practical analysis of sample complexity for structure learning of discrete dynamic Bayesian networks

ABSTRACT Discrete Dynamic Bayesian Network (dDBN) is used in many challenging causal modelling applications, such as human brain connectivity, due to its multivariate, non-deterministic, and nonlinear capability. Since there is not a ground truth for brain connectivity, the resulting model cannot be evaluated quantitatively. However, we should at least make sure that the best structure results for the used modelling approach and the data. Later, this result can be appreciated by further correlated literature of anatomy and physiology. Nearly all of the previously published studies rest on limited data, which brings doubt to the resulting structures. In theory, an immense number of samples is required, which is impossible to collect in practice. In this study, the appropriate number of data which makes a dDBN modelling trustable is searched by practical experiments and found to be for binary and ternary-valued networks, where K is the cardinality of the random variables and p is the maximum number of parents possibly present in the network. If a modelling approach satisfies this amount of data, we can at least say that the resulting structure is trustable.

A Novel Qualitative Maximum a Posteriori Estimation for Bayesian Network Parameters Based on Computing the Center Point of Constrained Parameter Regions

Introducing parameter constraints has become a mainstream approach for learning Bayesian network parameters with small datasets. The QMAP (Qualitative Maximum a Posteriori) estimation has produced the best learning accuracy among existing learning approaches. However, the rejection-acceptance sampling employed in the QMAP algorithm for determining average BN(Bayesian Network) parameter values is time-consuming, particularly when the number of parameter constraints is large. This paper proposes a new analytical approach that enhances the learning efficiency of the QMAP algorithm without reducing its learning accuracy by treating the average value of the parameters as the center point of the constrained parameter region, which is a much more efficient method than the rejection-acceptance sampling method employed in the traditional QMAP algorithm. First, a novel objective function is designed and a constrained objective optimization model is constructed based on parameter constraints. Second, the constructed model is employed to obtain the center point of the constrained parameter region based on its boundary points, and the average parameter value is the average of all boundary points. The results obtained from a large number of simulation experiments with four benchmark Bayesian networks demonstrate that of the parameter learning accuracy of the proposed algorithm is slightly better than that the original QMAP algorithm under specific conditions, and the computational efficiency is substantially increased under all conditions.

Comparing the Electrophysiology and Morphology of Human and Mouse Layer 2/3 Pyramidal Neurons With Bayesian Networks.

Pyramidal neurons are the most common neurons in the cerebral cortex. Understanding how they differ between species is a key challenge in neuroscience. We compared human temporal cortex and mouse visual cortex pyramidal neurons from the Allen Cell Types Database in terms of their electrophysiology and dendritic morphology. We found that, among other differences, human pyramidal neurons had a higher action potential threshold voltage, a lower input resistance, and larger dendritic arbors. We learned Gaussian Bayesian networks from the data in order to identify correlations and conditional independencies between the variables and compare them between the species. We found strong correlations between electrophysiological and morphological variables in both species. In human cells, electrophysiological variables were correlated even with morphological variables that are not directly related to dendritic arbor size or diameter, such as mean bifurcation angle and mean branch tortuosity. Cortical depth was correlated with both electrophysiological and morphological variables in both species, and its effect on electrophysiology could not be explained in terms of the morphological variables. For some variables, the effect of cortical depth was opposite in the two species. Overall, the correlations among the variables differed strikingly between human and mouse neurons. Besides identifying correlations and conditional independencies, the learned Bayesian networks might be useful for probabilistic reasoning regarding the morphology and electrophysiology of pyramidal neurons.

From heterogeneous healthcare data to disease-specific biomarker networks: A hierarchical Bayesian network approach.

In this work, we introduce an entirely data-driven and automated approach to reveal disease-associated biomarker and risk factor networks from heterogeneous and high-dimensional healthcare data. Our workflow is based on Bayesian networks, which are a popular tool for analyzing the interplay of biomarkers. Usually, data require extensive manual preprocessing and dimension reduction to allow for effective learning of Bayesian networks. For heterogeneous data, this preprocessing is hard to automatize and typically requires domain-specific prior knowledge. We here combine Bayesian network learning with hierarchical variable clustering in order to detect groups of similar features and learn interactions between them entirely automated. We present an optimization algorithm for the adaptive refinement of such group Bayesian networks to account for a specific target variable, like a disease. The combination of Bayesian networks, clustering, and refinement yields low-dimensional but disease-specific interaction networks. These networks provide easily interpretable, yet accurate models of biomarker interdependencies. We test our method extensively on simulated data, as well as on data from the Study of Health in Pomerania (SHIP-TREND), and demonstrate its effectiveness using non-alcoholic fatty liver disease and hypertension as examples. We show that the group network models outperform available biomarker scores, while at the same time, they provide an easily interpretable interaction network.

Analysis of Bayesian Network Learning Techniques for a Hybrid Multi-objective Bayesian Estimation of Distribution Algorithm: a case study on MNK Landscape

This work investigates different Bayesian network structure learning techniques by thoroughly studying several variants of Hybrid Multi-objective Bayesian Estimation of Distribution Algorithm (HMOBEDA), applied to the MNK Landscape combinatorial problem. In the experiments, we evaluate the performance considering three different aspects: optimization abilities, robustness and learning efficiency. Results for instances of multi- and many-objective MNK-landscape show that, score-based structure learning algorithms appear to be the best choice. In particular, HMOBEDA $$_{k2}$$ was capable of producing results comparable with the other variants in terms of the runtime of convergence and the coverage of the final Pareto front, with the additional advantage of providing solutions that are less sensible to noise while the variability of the corresponding Bayesian network models is reduced.

A novel approach to fully representing the diversity in conditional dependencies for learning Bayesian network classifier

Bayesian network classifiers (BNCs) have proved their effectiveness and efficiency in the supervised learning framework. Numerous variations of conditional independence assumption have been proposed to address the issue of NP-hard structure learning of BNC. However, researchers focus on identifying conditional dependence rather than conditional independence, and information-theoretic criteria cannot identify the diversity in conditional (in)dependencies for different instances. In this paper, the maximum correlation criterion and minimum dependence criterion are introduced to sort attributes and identify conditional independencies, respectively. The heuristic search strategy is applied to find possible global solution for achieving the trade-off between significant dependency relationships and independence assumption. Our extensive experimental evaluation on widely used benchmark data sets reveals that the proposed algorithm achieves competitive classification performance compared to state-of-the-art single model learners (e.g., TAN, KDB, KNN and SVM) and ensemble learners (e.g., ATAN and AODE).

More Investment in Research and Development for Better Education in the Future?

The question in this paper is whether R\&D efforts affect education performance in small classes. Merging two datasets collected from the PISA studies and the World Development Indicators and using Learning Bayesian Networks, we prove the existence of a statistical causal relationship between investment in R\&D of a country and its education performance (PISA scores). We also prove that the effect of R\&D on Education is long term as a country has to invest in R&D at least 10 years before beginning to improve the level of young pupils.

Dynapenia in middle-aged and older persons with and without abdominal obesity and the complex relationship with behavioral, physical-health and mental-health variables: Learning Bayesian network structures.

Few studies have analyzed the relationship between dynapenia and lifestyle, physical-health and mental-health variables in middle-aged and older persons classified according to their waist circumference. The aim of this study was to identify the relationships between dynapenia and behavioral, physical-health and mental-health variables in middle-aged and older persons with and without abdominal obesity using Bayesian networks, a graphical analysis technique that allows simultaneous associations between variables to be identified. A cross-sectional study of 2526 participants without abdominal obesity and 2729 with abdominal obesity (waist circumference>102cm in men and >88cm in women) aged 50 years or older who took part in the Brazilian Longitudinal Study of Aging. Two network structures including behavioral, physical-health and mental-health variables and dynapenia (grip strength<26kg for men and <16kg for women) were learnt with the hill-climbing score-based algorithm for a sample of individuals with and without abdominal obesity. Statistical resampling tests were performed to quantify the strength of the relationships between factors associated with dynapenia. In middle-aged and older persons without abdominal obesity, dynapenia was strongly associated with and influenced by at-risk alcohol consumption while in middle-aged and older persons with abdominal obesity the condition was strongly associated with and influenced by diabetes and depression. In the latter group, dynapenia was also strongly associated with and influenced the disability. The use of Bayesian networks allowed different factors associated with dynapenia in middle-aged and older persons classified according to the presence or absence of abdominal obesity to be identified.

Object recognition for power equipment via human‐level concept learning

Inspection robots are popularized in substations due to the lack of personnel for operation and maintenance. However, these inspection robots remain at the level of perceptual intelligence, rather than cognition intelligence. To enable a robot to automatically detect defects of power equipment, object recognition is a critical step because criteria of infrared diagnosis vary with types of equipment. Since this task is not a big-sample learning problem, prior knowledge needs to be added to improve existing methods. Here, an object recognition model based on human-level concept learning is proposed, which utilizes relationship between equipment. The proposed method is composed of three parts: Mask RCNN, Bayesian Context Network and human-level concept learning. As the backbone network, Mask RCNN, a pixel-wise segmentation network, gives preliminary recognition results. Then, based on the object relationship graph of Bayesian Context Network, human-level concept learning corrects the results in sequence by maximizing the conditional probability of an object given its neighbourhood. Experiments show that the accuracy of the proposed method increases 9.7% compared with its backbone network, making industrial application of this technology possible.

Modeling multiple phenotypes in wheat using data-driven genomic exploratory factor analysis and Bayesian network learning.

Inferring trait networks from a large volume of genetically correlated diverse phenotypes such as yield, architecture, and disease resistance can provide information on the manner in which complex phenotypes are interrelated. However, studies on statistical methods tailored to multidimensional phenotypes are limited, whereas numerous methods are available for evaluating the massive number of genetic markers. Factor analysis operates at the level of latent variables predicted to generate observed responses. The objectives of this study were to illustrate the manner in which data‐driven exploratory factor analysis can map observed phenotypes into a smaller number of latent variables and infer a genomic latent factor network using 45 agro‐morphological, disease, and grain mineral phenotypes measured in synthetic hexaploid wheat lines (Triticum aestivum L.). In total, eight latent factors including grain yield, architecture, flag leaf‐related traits, grain minerals, yellow rust, two types of stem rust, and leaf rust were identified as common sources of the observed phenotypes. The genetic component of the factor scores for each latent variable was fed into a Bayesian network to obtain a trait structure reflecting the genetic interdependency among traits. Three directed paths were consistently identified by two Bayesian network algorithms. Flag leaf‐related traits influenced leaf rust, and yellow rust and stem rust influenced grain yield. Additional paths that were identified included flag leaf‐related traits to minerals and minerals to architecture. This study shows that data‐driven exploratory factor analysis can reveal smaller dimensional common latent phenotypes that are likely to give rise to numerous observed field phenotypes without relying on prior biological knowledge. The inferred genomic latent factor structure from the Bayesian network provides insights for plant breeding to simultaneously improve multiple traits, as an intervention on one trait will affect the values of focal phenotypes in an interrelated complex trait system.

Dynamic Bayesian Network Modeling, Learning, and Inference: A Survey

Since the introduction of Dynamic Bayesian Networks (DBNs), their efficiency and effectiveness have increased through the development of three significant aspects: (i) modeling, (ii) learning and (iii) inference. However, no reviews of the literature have been found that chronicle their importance and development over time. The aim of this study is to provide a systematic review of the literature that details the evolution and advancement of DBNs, focusing in the period 1997–2019 that emphasize the aspects of modeling, learning and inference. While the literature presents temporal event networks, knowledge encapsulation, relational and time varying representations as the four predominant DBN modeling approaches, this work groups them as essential techniques within DBNs and help practitioners by associating each to various challenge that arise in pattern discovery and prediction in dynamic processes. Regarding learning, the predominant methods mainly focus on scoring with greedy search. Finally, our study suggests that the main methods used in DBN inference extend or adapt those used in static BNs, and are oriented to either optimize processing time or error rate.

Knowledge Tracing: A Review of Available Technologies

As a student modeling technique, knowledge tracing is widely used by various intelligent tutoring systems to infer and trace the individual’s knowledge state during the learning process. In recent years, various models were proposed to get accurate and easy-to-interpret results. To make sense of the wide Knowledge tracing (KT) modeling landscape, this paper conducts a systematic review to provide a detailed and nuanced discussion of relevant KT techniques from the perspective of assumptions, data, and algorithms. The results show that most existing KT models consider only a fragment of the assumptions that relate to the knowledge components within items and student’s cognitive process. Almost all types of KT models take “quize data” as input, although it is insufficient to reflect a clear picture of students’ learning process. Dynamic Bayesian network, logistic regression and deep learning are the main algorithms used by various knowledge tracing models. Some open issues are identified based on the analytics of the reviewed works and discussed potential future research directions.