Bayesian Network Parameters Research Articles

A New Multi-Objective Decision-Making Approach Applied to the Tennessee Eastman Process

In this paper, a generic framework and a new methodology aiming to decisions fusion of various Fault Detection and Diagnosis (FDD) methods are proposed. The framework consists of a discrete Bayesian Network (BN) and can handle all FDD methods, regardless of their a prior knowledge or requirements. The methodology expresses the FDD objectives to achieve the desired performance and results in a theoretical learning of the BN parameters. The development leads to a multi-objective problem under constraints, resolved with a lexicographic method. The effectiveness of the proposed Multi-Objective Decision-Making (MODM) approach is validated through the Tennessee Eastman Process (TEP), as a challenging industrial benchmark problem. The application shows the significant improvement in FDD performances that can be ensured by the proposed methodology, in terms of high fault detection rate and small false alarm rate.

An effective Bayesian network parameters learning algorithm for autonomous mission decision-making under scarce data

In this paper, a constrained parameter evolutionary learning (CPEL) algorithm for Bayesian network (BN) parameters learning under scarce data is proposed, which can be applied to UAV autonomous mission decision-making. In detail, firstly qualitative domain knowledge is employed into BN parameters learning process to reduce the parameter search space where two types of qualitative domain knowledge with experts’ confidence are presented; and then evolutionary strategy is introduced into the process to avoid the problem that classical learning technique falls into local optimum easily in which the special encoding for the BN parameters is presented and some evolutionary strategies are discussed. Moreover, the global convergence of the proposed algorithm is proven. According to numerical experiments, it’s demonstrated that the CPEL algorithm has better accuracy and timeliness performance than the classical EM algorithm under the same condition. Additionally, a case study of the proposed algorithm in UAV autonomous mission decision-making has been conducted, showing that CPEL algorithm can satisfy the need of UAV autonomous mission decision-making in a complex dynamic environment.

Bayesian Networks in Fault Diagnosis

Fault diagnosis is useful in helping technicians detect, isolate, and identify faults, and troubleshoot. Bayesian network (BN) is a probabilistic graphical model that effectively deals with various uncertainty problems. This model is increasingly utilized in fault diagnosis. This paper presents bibliographical review on use of BNs in fault diagnosis in the last decades with focus on engineering systems. This work also presents general procedure of fault diagnosis modeling with BNs; processes include BN structure modeling, BN parameter modeling, BN inference, fault identification, validation, and verification. The paper provides series of classification schemes for BNs for fault diagnosis, BNs combined with other techniques, and domain of fault diagnosis with BN. This study finally explores current gaps and challenges and several directions for future research.

Learning Bayesian network parameters from small data sets: A further constrained qualitatively maximum a posteriori method

To improve the learning accuracy of the parameters in a Bayesian network from a small data set, domain knowledge is normally incorporated into the learning process as parameter constraints. MAP-based (Maximum a Posteriori) methods that utilize both sample data and domain knowledge have been well studied in the literature. Among all the MAP-based methods, the QMAP (Qualitatively Maximum a Posteriori) method exhibits the best learning performance. However, when the data is insufficient, the estimation given by the QMAP often fails to satisfy all the convex parameter constraints, and this has made the overall QMAP estimation unreliable. To ensure that QMAP estimation does not violate any given parameter constraints and to further improve the learning accuracy, a FC-QMAP (Further Constrained Qualitatively Maximum a Posteriori) algorithm is proposed in this paper. The algorithm regulates QMAP estimation by replacing data estimation with a further constrained estimation via convex optimization. Experiments and theoretical analysis show that the proposed algorithm outperforms most of the existing parameter learning methods (namely, Maximum Likelihood, Constrained Maximum Likelihood, Maximum Entropy, Constrained Maximum Entropy, Maximum a Posteriori, and Qualitatively Maximum a Posteriori).

The research of elevator health diagnosis method based on Bayesian network

Elevator, as a complex mechanical system, is hard to determine the factors that affect components’ status. In accordance with this special characteristic, the Elevator Fault Diagnosis Model is proposed based on Bayesian Network in this paper. The method uses different samples of the elevator and adopts Monte Carlo inference mechanism for Bayesian Network Model structure and parameter learning. Eventually, an elevator fault diagnosis model based on Bayesian network is established, which accords with the theory of elevator operation. In this paper, we use different kinds of fault data samples to test the method. Experimental results demonstrate the higher accuracy of our method. This paper provides a good assistant method by means of Fault prediction and Health diagnosis of elevator system at present.

Data Missing Bayesian Network Parameters Learning Optimization Algorithm Based on EM

The paper proposes an optimization algorithm for the parameter learning of missing data Bayesian networks. Expectation Maximization (EM) algorithm is the common parameter learning algorithms. The maximum likelihood estimation (MLE) and maximum a posterity estimation (MAP) of EM are local estimate rather than global estimate and are not easy to achieve the global optimal. So this paper puts forward a point estimate relative error minimum optimization algorithm which based on EM algorithm (EM-MLE-MAP). Applying the algorithm to the fault diagnosis of the rotor vibration Bayesian network, simulations and experiments show that the improved algorithm has good precision in diagnosing vibration fault when the loss ratio less than 3 percent.

Estimating bayesian networks parameters using EM and Gibbs sampling

A method based on Expectation Maximization (EM) algorithm and Gibbs sampling is proposed to estimate Bayesian networks (BNs) parameters. We employ the Gibbs sampling to approximate the E-step of EM algorithm. According to transition probability, Gibbs sampling is utilized in data completion of E-step, which can reduce the computational complexity of EM algorithm. The experiments for comparison between the proposed method and EM algorithm are made. For the proposed method, the consumed time and the number of iterations are all less than those of EM algorithm. However, the KL divergence is higher than that of EM algorithm, which is a limitation for the proposed method.

An empirical study of Bayesian network parameter learning with monotonic influence constraints

Learning the conditional probability table (CPT) parameters of Bayesian networks (BNs) is a key challenge in real-world decision support applications, especially when there are limited data available. A conventional way to address this challenge is to introduce domain knowledge/expert judgments that are encoded as qualitative parameter constraints. In this paper we focus on a class of constraints which is naturally encoded in the edges of BNs with monotonic influences. Experimental results indicate that such monotonic influence constraints are widespread in practical BNs (all BNs used in the study contain such monotonic influences). To exploit expert knowledge about such constraints we have developed an improved constrained optimization algorithm, which achieves good parameter learning performance using these constraints, especially when data are limited. Specifically, this algorithm outperforms the previous state-of-the-art and is also robust to errors in labelling the monotonic influences. The method is applied to a real world medical decision support BN where we had access to expert-provided constraints and real hospital data. The results suggest that incorporating expert judgments about monotonic influence constraints can lead to more accurate BNs for decision support and risk analysis.

Fast learning of relational dependency networks

A relational dependency network (RDN) is a directed graphical model widely used for multi-relational data. These networks allow cyclic dependencies, necessary to represent relational auto-correlations. We describe an approach for learning both the RDN’s structure and its parameters, given an input relational database: First learn a Bayesian network (BN), then transform the Bayesian network to an RDN. Thus fast Bayesian network learning translates into fast RDN learning. The BN-to-RDN transform comprises a simple, local adjustment of the Bayesian network structure and a closed-form transform of the Bayesian network parameters. This method can learn an RDN for a dataset with a million tuples in minutes. We empirically compare our approach to a state-of-the-art RDN learning approach that applies functional gradient boosting, using six benchmark datasets. Learning RDNs via BNs scales much better to large datasets than learning RDNs with current boosting methods.

When and where to transfer for Bayesian network parameter learning

Learning Bayesian networks from scarce data is a major challenge in real-world applications where data are hard to acquire. Transfer learning techniques attempt to address this by leveraging data from different but related problems. For example, it may be possible to exploit medical diagnosis data from a different country. A challenge with this approach is heterogeneous relatedness to the target, both within and across source networks. In this paper we introduce the Bayesian network parameter transfer learning (BNPTL) algorithm to reason about both network and fragment (sub-graph) relatedness. BNPTL addresses (i) how to find the most relevant source network and network fragments to transfer, and (ii) how to fuse source and target parameters in a robust way. In addition to improving target task performance, explicit reasoning allows us to diagnose network and fragment relatedness across Bayesian networks, even if latent variables are present, or if their state space is heterogeneous. This is important in some applications where relatedness itself is an output of interest. Experimental results demonstrate the superiority of BNPTL at various scarcities and source relevance levels compared to single task learning and other state-of-the-art parameter transfer methods. Moreover, we demonstrate successful application to real-world medical case studies.

Bayesian prediction of earthquake network based on space–time influence domain

Bayesian networks (BNs) are used to analyze the conditional dependencies among different events, which are expressed by conditional probability. Scientists have already investigated the seismic activities by using BNs. Recently, earthquake network is used as a novel methodology to analyze the relationships among the earthquake events. In this paper, we propose a way to predict earthquake from a new perspective. The BN is constructed after processing, which is derived from the earthquake network based on space–time influence domain. And then, the BN parameters are learnt by using the cases which are designed from the seismic data in the period between 00:00:00 on January 1, 1992 and 00:00:00 on January 1, 2012. At last, predictions are done for the data in the period between 00:00:00 on January 1, 2012 and 00:00:00 on January 1, 2015 combining the BN with the parameters. The results show that the success rate of the prediction including delayed prediction is about 65%. It is also discovered that the predictions for some nodes have high rate of accuracy under investigation.

Application of Bayesian networks in composite power system reliability assessment and reliability‐based analysis

Bayesian network (BN) is a strong framework for handling probabilistic events which have received limited attention in power system reliability assessment. By applying BN for bulk power system reliability assessment, additional capabilities are provided at modelling and analysis levels in comparison with conventional methods. This study proposes a methodology to apply BNs to composite power system (CPS) reliability modelling, reliability assessment and reliability-based analyses. A minimal cutset (MC)-based method is proposed to extract the BN structure. Moreover, BN parameters are defined based on logical relationships between components, MCs and system failure. In addition, some issues of BN application to large systems are investigated. A variety of reliability-based analyses, useful in different power system studies are introduced and discussed in details which are provided by applying BNs to CPS reliability. The computational efficiency of the presented method is demonstrated by comparing it with state enumeration and Monte Carlo simulation methods. The proposed methodology is implemented on Roy Billinton Test System to extract its BN model based on which analysis issues are considered. Finally, the proposed methodology is applied to the simple, but representative, system the IEEE-reliability test system to show its feasibility in larger systems.

An industrial Fault Diagnosis System based on Bayesian Networks

This paper presents a DC motor fault diagnosis system based on Bayesian networks. This was done by the design of a new electromechanical test bed allowing the collection of functioning data from a real world industrial Direct current (DC) Motor. The data collection will help in the construction of Bayesian networks models. These data are collected from sensors measuring different types of variables that are directly related to the industrial system. Without doing any mathematical modeling that describes the physical properties of the studied DC motor, the proposed tool provides with the help of Bayesian networks parameters and structure learning algorithms, the base to construct a fault diagnosis tool that can be extended to a fault prognosis tool.

Fault diagnosis for a solar assisted heat pump system under incomplete data and expert knowledge

Fault diagnosis for a solar assisted heat pump (SAHP) system in the presence of incomplete data and expert knowledge is discussed in this article. A method for parameter learning of Bayesian networks (BNs) from incomplete data based on the back-propagation (BP) neural network and maximum likelihood estimation (MLE), which is called BP-MLE method, is presented. The BP neural network is utilized to impute the missing data and the complete data sets are addressed with MLE to obtain the parameters of BN. A method for parameter estimation under incomplete expert knowledge based on BP neural networks and fuzzy set theory is also presented, which is called BP-FS method. Similarly, the missing information is imputed by the trained BP neural network. Fuzzy set theory is employed to quantify the parameters of BN based on complete qualitative expert knowledge. The presented methods are applied to parameter learning of diagnostic BN for a SAHP system with incomplete simulation data and expert knowledge. The developed BN can perform fault diagnosis with complete or incomplete symptoms.

Learning discrete Bayesian network parameters from continuous data streams: What is the best strategy?

We compare three approaches to learning numerical parameters of discrete Bayesian networks from continuous data streams: (1) the EM algorithm applied to all data, (2) the EM algorithm applied to data increments, and (3) the online EM algorithm. Our results show that learning from all data at each step, whenever feasible, leads to the highest parameter accuracy and model classification accuracy. When facing computational limitations, incremental learning approaches are a reasonable alternative. While the differences in speed between incremental algorithms are not large (online EM is slightly slower), for all but small data sets online EM tends to be more accurate than incremental EM.

Optimized Structure for Facial Action Unit Relationship Using Bayesian Network

Facial expression recognition has been a very important task for human-computer interactions. Computer vision techniques have been much employed to get the automated recognition of facial expression. Facial Action Coding System has best described on facial expression, which includes 46 action units that involve facial muscle movements. In this paper, the relationships between Action Units are modeled using Bayesian Network structure. Action Units relationship and modeling are explained, and the learning algorithms for Bayesian Network structure are discussed. The aim of this work is to get the optimized structure to best explain the relationships of AUs based on two AU-coded databases. The relationships among AUs are important for further facial expression recognition. Experiments demonstrate that a prior network is essential to start up and ease the searching process. Due to the complexity of the possible structures, constraints are applied to simplify the work. Bayesian Network parameters are also learned from two databases for different structures to do Bayesian Network inference and classification based on the data set. Average recognition rate for each AU is illustrated, and the overall recognition performance is analyzed to get the optimized structure to best describe the AU relationship.

Learning parameters of fuzzy Bayesian Network based on imprecise observations

In recent years, Bayesian Network has become an important modeling method for decision making problems of real-world applications. In this paper learning parameters of a fuzzy Bayesian Network (BN) based on imprecise/fuzzy observations is considered, where imprecise observations particularly refers to triangular fuzzy numbers. To achieve this, an extension to fuzzy probability theory based on imprecise observations is proposed which employs both the truth concept of Yager and the Extension Principle in fuzzy set theory. In addition, some examples are given to demonstrate the concepts of the proposed idea. The aim of our suggestion is to be able to estimate joint fuzzy probability and the conditional probability tables (CPTs) of Bayesian Network based on imprecise observations. Two real-world datasets, Car Evaluation Database (CED) and Extending Credibility (EC), are employed where some of attributes have crisp (exact) and some of them have fuzzy observations. Estimated parameters of the CED's corresponding network, using our extension, are shown in tables. Then, using Kullback-Leibler divergence, two scenarios are considered to show that fuzzy parameters preserve more knowledge than that of crisp parameters. This phenomenon is also true in cases where there are a small number of observations. Finally, to examine a network with fuzzy parameters versus the network with crisp parameters, accuracy result of predictions is provided which shows improvements in the predictions.

Starting Point, Duration of Acquisition and Averaged of Time Series Over Subjects: Issues Related to the Reliability and Accuracy of the Constructed Group Bayesian Network for Resting-state fMRI Studies

Bayesian Network (BN) modeling has recently been proposed and utilized in resting-state functional magnetic resonance imaging (fMRI) studies to derive directed connectivity among brain regions in an integrated network. This report focuses on three issues related to the resting-state fMRI BN approach: 1) How the disturbance variable and noise affect BN inference and whether it is reasonable to use extracted fMRI time series averaged over multiple subjects to characterize the effective connectivity common to all of these subjects, 2) the effects of the fMRI time series length (duration) for a given TR(repetition time) on the BN results and 3) the effects of varying the time point at which the data acquisition starts (starting point). Both synthetic dataset and real resting-state fMRI dataset were employed. It is found that the averaged group BN inference is more robust than individual BN inference under most conditions according to the results of synthetic dataset, and the BN structure and parameter estimation are robust to both the variation of the time length (with a minimum) and starting point. Our results might demonstrate the stability of averaged group BN inference on resting fMRI dataset under most conditions, and justify its use to estimate the effective connectivity, which is independent of the start point and the length of resting-state fMRI time series. Keywords: Bayesian network, functional MRI, effective connectivity, group inference, resting-state.

A novel method for combining Bayesian networks, theoretical analysis, and its applications

Effective knowledge integration plays a very important role in knowledge engineering and knowledge-based machine learning. The combination of Bayesian networks (BNs) has shown a promising technique in knowledge fusion and the way of combining BNs remains a challenging research topic. An effective method of BNs combination should not impose any particular constraints on the underlying BNs such that the method is applicable to a variety of knowledge engineering scenarios. In general, a sound method of BNs combination should satisfy three fundamental criteria, that is, avoiding cycles, preserving the conditional independencies of BN structures, and preserving the characteristics of individual BN parameters, respectively. However, none of the existing BNs combination method satisfies all the aforementioned criteria. Accordingly, there are only marginal theoretical contributions and limited practical values of previous research on BNs combination. In this paper, following the approach adopted by existing BNs combination methods, we assume that there is an ancestral ordering shared by individual BNs that helps avoid cycles. We first design and develop a novel BNs combination method that focuses on the following two aspects: (1) a generic method for combining BNs that does not impose any particular constraints on the underlying BNs, and (2) an effective approach ensuring that the last two criteria of BNs combination are satisfied. Further through a formal analysis, we compare the properties of the proposed method and that of three classical BNs combination methods, and hence to demonstrate the distinctive advantages of the proposed BNs combination method. Finally, we apply the proposed method in recommender systems for estimating users' ratings based on their implicit preferences, bank direct marketing for predicting clients' willingness of deposit subscription, and disease diagnosis for assessing patients' breast cancer risk.

Impact of precision of Bayesian network parameters on accuracy of medical diagnostic systems

One of the hardest technical tasks in employing Bayesian network models in practice is obtaining their numerical parameters. In the light of this difficulty, a pressing question, one that has immediate implications on the knowledge engineering effort, is whether precision of these parameters is important. In this paper, we address experimentally the question whether medical diagnostic systems based on Bayesian networks are sensitive to precision of their parameters. The test networks include Hepar II, a sizeable Bayesian network model for diagnosis of liver disorders and six other medical diagnostic networks constructed from medical data sets available through the Irvine Machine Learning Repository. Assuming that the original model parameters are perfectly accurate, we lower systematically their precision by rounding them to progressively courser scales and check the impact of this rounding on the models' accuracy. Our main result, consistent across all tested networks, is that imprecision in numerical parameters has minimal impact on the diagnostic accuracy of models, as long as we avoid zeroes among parameters. The experiments' results provide evidence that as long as we avoid zeroes among model parameters, diagnostic accuracy of Bayesian network models does not suffer from decreased precision of their parameters.