Bayesian Network Parameter Learning Research Articles

Learning bayesian network parameters from limited data by integrating entropy and monotonicity

High-accuracy parameter learning in Bayesian Networks (BNs) is a key challenge in real-time decision support applications, particularly when the available data are limited. Prior/Expert knowledge was introduced to eliminate the drawbacks of insufficient information; however, this method is subjective. In this study, we explored the use of monotonicity constraints to control the causal relationships between the nodes and their parents-nodes in BNs and proposed a new learning algorithm called global domain monotonicity based on maximum information entropy (GDM-MIE), which was designed for parameter learning in uncertain discrete BNs with nonlinear equality constraints when only finite data are available. In the proposed algorithm, a class of monotonicity is encoded as a constraint on information entropy and the parameter learning problem is transformed into constraints among the node parameters based on a known network. Furthermore, we considered the parameters as uncertain entropy information and discussed the monotonicity among parameters in the global spatial domain, proving the accuracy of the logical relationships of the model, system reliability, and time complexity. Finally, the proposed method was validated using standard BNs, and its performance was analyzed by comparing the proposed method with the existing learning algorithms. The results showed that the proposed method is more accurate and has better Kullback–Leibler divergence. To revalidate the rationality of the proposed method, the Alarm and Asia networks were employed as special cases. The GDM-MIE was found to achieve the intended goal of observing the estimated parameters by closely approximating the original real parameters with a small sample size, indicating that the proposed algorithm can served as an efficient and feasible method for learning Bayesian parameter.

The challenges of ideological and political education in universities in the era of big data and its optimization path

Abstract How to propose effective solutions for ideological and political education in colleges and universities to adapt to the development requirements of the new era is an urgent problem to be solved. In this paper, we first study the data mining algorithm Bayesian network and describe the components of the Bayesian network. Then the parameter learning of Bayesian networks is discussed in depth, and the conditional probabilities of each node in the network are calculated by counting the prior probabilities of relevant variables according to the structure graph and the corresponding training set. Then based on the structure learning, the optimal Bayesian network is selected based on the search and conditional independent testing methods. Finally, the current situation of modern ideological and political education is described in terms of students’ learning paths, learning purposes, and the problems of ideological and political education, and suggestions for optimal reform are made. Regarding learning motivation, 48% of the students said they study for school requirements and credits and 10% for establishing correct social values. Regarding problems, 67% said the teaching format is not rich, 90% said the teaching presents utilitarianism, and 83% said practical awareness is not strong. It is important to seek effective paths to enhance ideological and political education.

ACGAN and BN based method for downhole incident diagnosis during the drilling process with small sample data size

During the drilling process, the complicated geological environment makes drilling operations more difficult as the drilling depth increases, leading to a greater susceptibility to drilling incidents. The parameters obtained from a drilling incident are usually incomplete and the sample size is small, which is difficult to be used for incident analysis. This paper proposes a new method for the diagnosis of downhole drilling incidents. The drilling data is generated based on an Auxiliary Classifier Generative Adversarial Networks (ACGAN) and an incident diagnosis model is built using the Bayesian network (BN). The effectiveness and practicality of the proposed method are proved by the actual case study. Based on historical data, data augmentation is performed using the ACGAN model, and then parameter learning of BN is conducted. The established BN model based on large data samples can be used for the diagnosis of downhole incidents. The precision and F1-score of diagnosis are above 80%. Root cause diagnosis of downhole incidents can be performed by backward inference of Bayesian methods. It can prevent the occurrence of downhole incidents. The results prove the proposed method can diagnose downhole incidents in real time and obtain the causes of downhole incidents.

Dynamic embeddings for efficient parameter learning of Bayesian network with multiple latent variables

Latent variables (LVs), representing the unobservable abstract concepts, such as patient disease and customer credit, play an important role in the simplification of network structure and improving the interpretability of Bayesian network (BN). However, LVs incorporated into BN lead to missing probability parameters due to the missing observation. As the classic method for parameter estimation for the situation with LVs, the expectation–maximization (EM) suffers high complexity and slow convergence. To this end, we propose dynamic embeddings for parameter learning of BN with LVs. Firstly, we reconstruct the E-step of EM and propose to use the dynamic embeddings to calculate the weights of fractional samples that could reduce the computational complexity of parameter learning. Secondly, we propose to construct the point mutual information (PMI) matrix to represent directed weighted graphs (DWGs) transformed from the updated parameters. Thirdly, the incremental singular value decomposition (SVD) is adopted to generate dynamic embeddings while capturing the updated parameters and preserving BN’s graphical structure. Experimental results show that our proposed methods are efficient and effective. On real-world BNs, the efficiency, convergence and accuracy of our method outperform those of the state-of-the-art methods for parameter learning of BN with multiple LVs.

Bearing Fault Diagnosis Under Small Data Set Condition: A Bayesian Network Method With Transfer Learning for Parameter Estimation

Bearings are broadly applied in various types of industrial systems. Fault diagnosis, as a promising way for reliability of modern industrial internet of thing applications, has attracted increasing attention from both academia and industry fields. Being ideal modeling and inference tool in uncertainty situations, Bayesian network (BN) is becoming increasingly popular in many systems. However, in practical uncertain and complicated engineering surroundings, it’s difficult or expensive to collect massive labeled fault data for the sake of fault diagnosis model learning. To address the issue of BN parameter learning under small data set conditions, this paper proposes a Varying Coefficient Transfer Learning (VCTL) algorithm based on aggregation and transfer learning, that considers both knowledge from the resource domain and the resource relevance contributions. The balancing weight function is designed to determine whether the learning task in the resource domain is activated. Relevance weight factors are proposed to measure the relevance of resource and target parameters quantitatively, by combing parameter information from resource domains with those obtained from the target domain, using maximum a posterior (MAP) or maximum likelihood estimation (MLE). Finally, the target parameters are aggregated with both the target initial parameters and the parameter knowledge from the resource domain. Based on VCTL, a bearing fault diagnosis approach is proposed and verified. The experimental results show that, under the condition of the small data set, learning accuracy of VCTL algorithm with varying coefficient aggregation is better than MLE algorithm, MAP algorithm or state-of-the-art parameter transfer method, local linear pooling transfer learning (LoLP) algorithm. Under the condition of sufficient data set, learning accuracy of VCTL algorithm approaches the classical MLE or MAP, and the correctness of the proposed algorithm is verified. Moreover, we illustrate the successful application to real-world bearing fault diagnosis case with VCTL, where we had access to expert-provided resource knowledge and real fault diagnosis data.

Bayesian network parameter learning algorithm based on improved QMAP

Small data sets make the statistical information in Bayesian network parameter learning inaccurate, which makes it difficult to get accurate Bayesian network parameters based on data. Qualitative maximum a posteriori estimation (QMAP) is the most accurate algorithm for Bayesian network parameter learning under the condition of small data sets. However, when the number of parameter constraints is large or the parameter feasible region is small, the rejection-acceptance sampling process in QMAP algorithm will become extremely time-consuming. In order to improve the learning efficiency of QMAP algorithm and not affect its learning accuracy as much as possible, a new analytical calculation method of the center point of constrained region is designed to replace the original rejection-acceptance sampling calculation method. Firstly, a new objective function is designed, and a constrained objective optimization problem for solving the boundary points of the constrained region is constructed. Secondly, a new optimization engine is used to solve the objective optimization problem, and the boundary points and center points of the constrained region are obtained. Finally, the existing QMAP algorithm is improved by the obtained center points. The simulation results show that the CMAP algorithm proposed in this paper has a slightly worse parameter learning accuracy than the QMAP algorithm, but its computational efficiency is 2-5 times higher than that of the QMAP algorithm.

Constrained Adjusted Maximum a Posteriori Estimation of Bayesian Network Parameters.

Maximum a posteriori estimation (MAP) with Dirichlet prior has been shown to be effective in improving the parameter learning of Bayesian networks when the available data are insufficient. Given no extra domain knowledge, uniform prior is often considered for regularization. However, when the underlying parameter distribution is non-uniform or skewed, uniform prior does not work well, and a more informative prior is required. In reality, unless the domain experts are extremely unfamiliar with the network, they would be able to provide some reliable knowledge on the studied network. With that knowledge, we can automatically refine informative priors and select reasonable equivalent sample size (ESS). In this paper, considering the parameter constraints that are transformed from the domain knowledge, we propose a Constrained adjusted Maximum a Posteriori (CaMAP) estimation method, which is featured by two novel techniques. First, to draw an informative prior distribution (or prior shape), we present a novel sampling method that can construct the prior distribution from the constraints. Then, to find the optimal ESS (or prior strength), we derive constraints on the ESS from the parameter constraints and select the optimal ESS by cross-validation. Numerical experiments show that the proposed method is superior to other learning algorithms.

Learning the Parameters of Bayesian Networks from Uncertain Data

The creation of Bayesian networks often requires the specification of a large number of parameters, making it highly desirable to be able to learn these parameters from historical data. In many cases, such data has uncertainty associated with it, including cases in which this data comes from unstructured analysis or from sensors. When creating diagnosis networks, for example, unstructured analysis algorithms can be run on the historical text descriptions or images of previous cases so as to extract data for learning Bayesian network parameters, but such derived data has inherent uncertainty associated with it due to the nature of such algorithms. Because of the inability of current Bayesian network parameter learning algorithms to incorporate such uncertainty, common approaches either ignore this uncertainty, thus reducing the resulting accuracy, or completely disregard such data. We present an approach for learning Bayesian network parameters that explicitly incorporates such uncertainty, and which is a natural extension of the Bayesian network formalism. We present a generalization of the Expectation Maximization parameter learning algorithm that enables it to handle any historical data with likelihood-evidence-based uncertainty, as well as an empirical validation demonstrating the improved accuracy and convergence enabled by our approach. We also prove that our extended algorithm maintains the convergence and correctness properties of the original EM algorithm, while explicitly incorporating data uncertainty in the learning process.

Evaluation of Intelligent Air Defense algorithm based on Machine Learning

In the complex and rapidly changing combat environment, the enemy interference and sensor performance limitations and other factors lead to insufficient battlefield information. In order to make the UAV have the ability to carry out threat assessment under the condition of insufficient information, a Bayesian network (BN) threat assessment modeling method based on small data sets is proposed in this paper. Starting with the two problems of BN structure learning and parameter learning under the condition of small data set, using the constraint matrix obtained by Bootstrap method as the constraint item added to the score function, a BN structure learning algorithm based on small data set is proposed, and the BN parameter learning algorithm based on interval prior constraint is adopted. The simulation results show that, compared with the traditional BN learning algorithm, the BN learning algorithm proposed in this paper has higher accuracy and availability in UAV threat assessment modeling under the condition of small data set.

Reliabilities analysis of evacuation on offshore platforms: A dynamic Bayesian Network model

An offshore platform is naturally vulnerable to accidents, such as the leakage of dangerous chemicals, fire and explosion. Oil and gas are explosive and all the equipment and pipes are squeezed into a limited area on a platform. Escape, Evacuation, and Rescue (EER) plans play a vital role as the last barrier to ensure the safety of personnel in the event of a major accident. As a result, the main contributors leading to evacuation failure need to be analyzed to prioritize technology development and select a robust EER strategy. This research aims to undertake the quantitative reliability analysis of various EER strategies on offshore platforms. First, a reliability prediction model of emergency evacuation is established for offshore platforms based on the K2 structure learning algorithm and a Bayesian network parameter learning method. The conditional probability table of each node is determined by combining the Bayesian estimation method and a junction tree reasoning engine. The reliability of emergency evacuation on a platform is predicted using a dynamic Bayesian network model. The transition probability is determined through a Markov method. The main factors leading to evacuation failure are investigated using the diagnostic reasoning method of Bayesian Network. The findings can provide insights for the development of cost effective EER strategies for an offshore platform.

Bayesian Network Parameter Learning Method Based on Transfer Learning

Bayesian Network Parameter Learning Method Based on Transfer Learning

Hard and Soft EM in Bayesian Network Learning from Incomplete Data

Incomplete data are a common feature in many domains, from clinical trials to industrial applications. Bayesian networks (BNs) are often used in these domains because of their graphical and causal interpretations. BN parameter learning from incomplete data is usually implemented with the Expectation-Maximisation algorithm (EM), which computes the relevant sufficient statistics (“soft EM”) using belief propagation. Similarly, the Structural Expectation-Maximisation algorithm (Structural EM) learns the network structure of the BN from those sufficient statistics using algorithms designed for complete data. However, practical implementations of parameter and structure learning often impute missing data (“hard EM”) to compute sufficient statistics instead of using belief propagation, for both ease of implementation and computational speed. In this paper, we investigate the question: what is the impact of using imputation instead of belief propagation on the quality of the resulting BNs? From a simulation study using synthetic data and reference BNs, we find that it is possible to recommend one approach over the other in several scenarios based on the characteristics of the data. We then use this information to build a simple decision tree to guide practitioners in choosing the EM algorithm best suited to their problem.

Bayesian network parameter learning method based on expert priori knowledge and monotonic constraints

Bayesian network parameter learning method based on expert priori knowledge and monotonic constraints

Learning Bayesian Network Parameters With Small Data Set: A Parameter Extension under Constraints Method

Recent advances have illustrated substantial benefits from learning Bayesian networks (BNs). However, when the available data size is small, the BN parameter learning becomes a key challenge in many intelligent applications. By integrating both sample data and expert constraints, we propose a BN parameter learning algorithm with extension method-parameter extension under constraints (PEUC) by introducing related domain expert knowledge. Knowledge is transformed into inequality constraints which candidate parameter sets arise from the relative constraints space.The maximum entropy principle helps to estimate the parameter in statistical averaging model while candidate sets of BN parameters satisfy the constrained knowledge by bootstrapping techniques. Then BN parameters are estimated based on the real available sampled data set and extension streams of candidate parameters samples from the constraints space. The sample size is also taken into account according to the contribution to the final parameters. Experimental results of benchmark BN modeling problems demonstrate that PEUC algorithm tends to the classical MLE algorithm when the modeling data size is sufficient. Furthermore, when the available data size is small, the parameters of BN can be estimated by PEUC as well, and the learned accuracy is superior to MLE, MAP or QMAP algorithm. Finally, PEUC is also applied to a real bearing fault diagnosis case. The presented approach provides a new promising BN parameter learning way for more intelligent system modeling problems, particularly when the data sets are small.

A Novel Method of Emergency Situation Evaluation for Deep-Sea Based on Bayesian Network

In order to make effective emergency decisions timely, this paper proposes an intelligent emergency situation evaluation method based on the Bayesian network for deep-sea emergency response, which is used to evaluate the deep-sea emergency situation quantitatively by calculating the occurrence probability of emergency situation risk at each risk level. In this research, a Bayesian network is constructed to analyze the deep-sea emergency situation firstly. Aiming at the bottleneck problem of insufficient case data for the Bayesian network's parameter learning, an automatic parameter allocation algorithm is presented based on the domain knowledge. Then, in order to obtain the quantitative analysis results of the overall emergency situation, a comprehensive situation evaluation model is presented based on the Bayesian network reasoning results, and the fuzzy comprehensive evaluation theory. Finally, a case study is given to illustrate this proposed method's application, which can provide a scientific basis for decision-making in the deep-sea emergency practice.

Learning bipartite Bayesian networks under monotonicity restrictions

ABSTRACTLearning parameters of a probabilistic model is a necessary step in machine learning tasks. We present a method to improve learning from small datasets by using monotonicity conditions. Monotonicity simplifies the learning and it is often required by users. We present an algorithm for Bayesian Networks parameter learning. The algorithm and monotonicity conditions are described, and it is shown that with the monotonicity conditions we can better fit underlying data. Our algorithm is tested on artificial and empiric datasets. We use different methods satisfying monotonicity conditions: the proposed gradient descent, isotonic regression EM, and non-linear optimization. We also provide results of unrestricted EM and gradient descent methods. Learned models are compared with respect to their ability to fit data in terms of log-likelihood and their fit of parameters of the generating model. Our proposed method outperforms other methods for small sets, and provides better or comparable results for larger sets.

Learning parameters of Bayesian networks from datasets with systematically missing data: A meta–analytic approach

Previous research suggested that using additional data sources could improve parameter learning in Bayesian networks. However, when additional datasets do not include all network variables, neither standard Bayesian network learning techniques nor standard missing data methods can be applied. In such situations, the use of a meta–analytic approach is proposed. The performance of one such meta–analytic approach was evaluated by simulating several study results on two real–life biomedical examples (one discrete and one Gaussian Bayesian network). Regardless of the network type, the meta–analytic approach showed higher mean log–likelihood values, less sensitive to the presence of heterogeneity, than a single dataset analysis. The difference between the two methods was most pronounced when sample sizes were small (N=100). For the meta–analytic approach, the increase in log–likelihood was in most cases positively related to the number of nodes estimated with additional data. However, as in the case of single dataset analysis, care is needed when estimating rare event probabilities from small datasets due to the problems with unidentifiability and increased bias.

Bayesian Network Parameter Learning Algorithm for Target Damage Assessment

Aiming at the problem that the existing methods of target damage assessment based on Bayesian network mainly determine the structural parameters of Bayesian network by giving conditional probability tables based on expert experience, which results in too subjective and having large errors in the evaluation results, an improved learning algorithm of conditional probability tables of Bayesian network is proposed in this paper. We divided the E step of the EM algorithm (Expectation Maximization Algorithm) into three steps. Firstly, the range of the missing variable is determined by expert experience, then the Gibbs sampling algorithm is used to complete the sample set, and finally the sample is weighted. The proposed algorithm is compared with EM algorithm, Gibbs algorithm, EM and Gibbs algorithm. The experimental results show that the proposed algorithm has good stability and high precision.

An improved bayesian network intrusion detection algorithm based on deep learning

Based on the preprocessing of training data sets, this paper adopts the convolution neural network technology to realize the reduction target of the redundant attribute of the data set. An improved Bayesian network intrusion detection algorithm based on deep learning is proposed. In this algorithm, sliding window technique and relative Euclidean distance are defined, and the structure updating and parameter learning of Bayesian networks are adaptively carried out on the basis of computing mutual information between attributes. Experimental results show that the new algorithm can effectively improve the computational efficiency and the accuracy of intrusion detection.

A Bayesian Network for the Transportation Accidents of Hazardous Materials Handling Time Assessment

Abstract To improve the efficiency of emergency rescue in transportation accidents of hazardous materials(HAZMAT), a Bayesian network(BN) was developed in this paper to estimate the accident handling time. Also, based on this BN, the difficulty of handling every types of accidents can be quantified. According to theoretical analysis and literature review, 7 nodes (season, time, type of road, type of HAZMAT, the former accident, the secondary accident and handling time) are used to set up the BN. The value of mutual information was calculated to refine the BN. A database of 902 transportation accident of HAZMAT cases was built up for Bayesian parameter learning. Based on the parameter learning of BN, the results were summarized as follow: (1) The BN could be used to estimate the probabilities of handling time in different periods which include ‘0 to 2 hours’, ‘2 to 4 hours’, ‘4 hours and more’. (2) The difficulty of each type of accident can be ordered as follow: rollover> rear-end> internal fault≈ impact> falling> tire fault> vehicle body fire. Leakage>combustion explosion.