Methods For Learning Bayesian Networks Research Articles

A Bayesian network learning method for sparse and unbalanced data with GNN-based multilabel classification application

Creating a well-defined Bayesian network (BN) is helpful for developing effective graph neural networks (GNNs) that exploit annotated labels in multilabel classification (MLC) tasks. Obtaining correct BNs can be challenging when the labels are highly unbalanced and sparse. This study proposes a novel scoring method to address data imbalance and sparsity by introducing an imbalanced weight term. Comparison results show that the BNs learned from the proposed scoring method are simpler and more accurate than the well-known methods. The obtained BNs based on the proposed method are used to construct GNNs for MLC tasks based on three datasets. The findings suggest that the adjacency matrices based on moral graphs of BNs exhibit more excellent stability than those derived from mathematical approaches. By selecting the parameter thoughtfully, GNNs based on the proposed BDsp scores showcase up to 12% classification performance improvement compared to GNNs based on existing Bayesian scores.

Application of a Bayesian Network Learning Model to Predict Longitudinal Trajectories of Executive Function Difficulties in Elementary School Students.

Executive function is the mental ability to modulate behavior or thinking to accomplish a task. This is developmentally important for children’s academic achievements and ability to adjust to school. We classified executive function difficulties (EFDs) in longitudinal trajectories in Korean children from 7 to 10 years old. We found predictors of EFDs using latent class growth analysis and Bayesian network learning methods with Panel Study data. Three types of latent class models of executive function difficulties were identified: low, intermediate, and high EFDs. The modeling performance of the high EFD group was excellent (AUC = .91), and the predictors were the child’s gender, temperamental emotionality, happiness, DSM (Diagnostic and Statistical Manual of Mental Disorders) anxiety problems, and the mother’s depression as well as coparenting conflict recognized by the mother. The results show that using latent class growth analysis and Bayesian network learning are helpful in classifying the longitudinal EFD patterns in elementary school students. Furthermore, school-age EFD is affected by emotional problems in parents and children that continue from early life. These findings can support children’s development and prevent risk by preclassifying children who may experience persistent EFD and tracing causes.

A data-driven Bayesian network learning method for process fault diagnosis

This paper presents a data-driven methodology for fault detection and diagnosis (FDD) by integrating the principal component analysis (PCA) with the Bayesian network (BN). Though the integration of PCA-BN for FDD purposes has been studied in the past, the present work makes two contributions for process systems. First, the application of correlation dimension (CD) to select principal components (PCs) automatically. Second, the use of Kullback-Leibler divergence (KLD) and copula theory to develop a data-based BN learning technique. The proposed method uses a combination of vine copula and Bayes’ theorem (BT) to capture nonlinear dependence of high-dimensional process data which eliminates the need for discretization of continuous data. The data-driven integrated PCA-BN framework has been applied to two processing systems. Performance of the proposed methodology is compared with the independent component analysis (ICA), kernel principal component analysis (KPCA), kernel independent component analysis (KICA), and their integrated frameworks with the BN. The comparative study suggests that the proposed framework provides superior performance.

A naive learning algorithm for class‐bridge‐decomposable multidimensional Bayesian network classifiers

SummaryMultidimensional Bayesian network classifier (MBC) has become a popular classification model because of their intuitive graphical representation ability among class variables. But learning MBC and performing multidimensional classification based on the MBC can be very computationally demanding. For the tractability of performing multidimensional classification, a class‐bridge‐decomposable (CB‐decomposable) MBC model is proposed and it alleviates the computation complexity. But there are few works to efficiently and systematically learn the CB‐decomposable MBC model. Thus, we focus on addressing a naive learning algorithm of CB‐decomposable MBCs. Briefly, we learn the CB‐decomposable MBC model by dividing it into three components: class subgraph, bridge subgraph, and feature subgraph. First, we analyze why the class subgraph can be learned based on general Bayesian network learning methods. Second, we give how to learn bridge subgraph based on information gain ratio. Third, to make the CB‐decomposable MBC model effective and simple, we also study the learning and updating strategies of feature subgraph. Further, we propose the naive learning algorithm of the CB‐decomposable MBC. Finally, by comparing with other methods on several benchmark datasets, experimental results illustrate that our naive learning algorithm not only has higher accuracies, lower learning, and classification times but also has simple and intuitive representation ability.

Maize Carotenoid Gene Locus Mining Based on Conditional Gaussian Bayesian Network

How to mine the gene locus for maize carotenoid components is an important research problem in biology study. Along with the rapid development of high-throughput biotechnologies, we have produced a large number of maize multi-omics data, including genome, transcriptome, metabolome, phenotype, etc. How to conjointly analyze these continuous and discrete data, and thus to mine the genetic loci that control the maize carotenoid components have a very important biological significance. In this work, we use the conditional Gaussian Bayesian network learning method to construct the network of maize gene, SNP locus and carotenoid components, aim to get the possible significant loci about four reported genes for the carotenoid component traits. The method is validated using the multi-omics data of maize global germplasm collection with 368 elite inbred lines. Four algorithms are used to do the comparison, and experiment results show the method can mine the effective locus for the phenotype traits. It is concluded that the conditional Gaussian Bayesian network learning method is an effective way of analyzing multi-omics data conjointly, mining the possible gene locus for maize carotenoid component traits, and thus to provide genetic resources and useful information for molecular breeding of maize.

Gene Regulatory Relationship Mining Using Improved Three-Phase Dependency Analysis Approach.

How to mine the gene regulatory relationship and construct gene regulatory network (GRN) is of utmost interest within the whole biological community, however, which has been consistently a challenging problem since the tremendous complexity in cellular systems. In present work, we construct gene regulatory network using an improved three-phase dependency analysis algorithm (TPDA) Bayesian network learning method, which includes the steps of Drafting, Thickening, and Thinning. In order to solve the problem of learning result is not reliable due to the high order conditional independence test, we use the entropy estimation approach of Gaussian kernel probability density estimator to calculate the (conditional) mutual information between genes. The experiment on the public benchmark data sets show the improved method outperforms the other nine kinds of Bayesian network learning methods when to process the data with large sample size, with small number of discrete values, and the frequency of different discrete values is about same. In addition, the improved TPDA method was further applied on a real large gene expression data set on RNA-seq from a global collection with 368 elite maize inbred lines. Experiment results show it performs better than the original TPDA method and the other nine kinds of Bayesian network learning algorithms significantly.

Reducing COPD Readmissions: A Causal Bayesian Network Model

This letter introduces a causal Bayesian network model to study readmissions reduction for chronic obstructive pulmonary disease (COPD) patients. The model employs a Bayesian network learning method and adopts domain knowledge. Using this model, we analyze the impacts of critical variables on a patient's readmission risk by the manipulation of such variables. Through this analysis, effective intervention options to reduce readmission can be identified, which can provide a quantitative tool for designing personalized interventions to reduce COPD readmissions.

An improved constraint-based Bayesian network learning method using Gaussian kernel probability density estimator

Bayesian network has powerful expression and reasoning ability of uncertainty knowledge. At present, it is widely used in many fields, such as fault diagnosis, medical diagnosis and prediction, financial analysis and prediction, decision support, etc. How to construct the Bayesian network effectively is an important research problem in the area of data mining. The constraint-based learning algorithm is commonly used for the construction of Bayesian network (BNs), such as Fast Incremental Association (Fast-IAMB) and Max-Min Parents & Children (MMPC), etc. At present, the Chi square test and correlation coefficient are often used to calculate the correlation between variables in these algorithms. But the calculation accuracy of these methods is not high, thus to influence the effectiveness of the constraint-based BNs learning algorithms. On the basis of the entropy estimation approach of Gaussian kernel probability density estimator, this paper uses the (conditional) mutual information to calculate the correlation between different variables and thus to improve the learning accuracy of the constraint-based BNs learning algorithms. The concrete calculation process about the data type of discrete, continuous and mixed has been described separately. Experiment simulations on the Alarm network prove that our algorithm can improve the learning accuracy of the existing three kinds of constraint-based BNs learning algorithms without affecting the learning efficiency. It also has better learning effect when to process the concrete and continuous data with sparsity. The method in this work can be used to learn more accurate Bayesian network, and then served the research of artificial intelligence, expert system decision making, etc.

Scoring Bayesian Networks of Mixed Variables.

In this paper we outline two novel scoring methods for learning Bayesian networks in the presence of both continuous and discrete variables, that is, mixed variables. While much work has been done in the domain of automated Bayesian network learning, few studies have investigated this task in the presence of both continuous and discrete variables while focusing on scalability. Our goal is to provide two novel and scalable scoring functions capable of handling mixed variables. The first method, the Conditional Gaussian (CG) score, provides a highly efficient option. The second method, the Mixed Variable Polynomial (MVP) score, allows for a wider range of modeled relationships, including non-linearity, but it is slower than CG. Both methods calculate log likelihood and degrees of freedom terms, which are incorporated into a Bayesian Information Criterion (BIC) score. Additionally, we introduce a structure prior for efficient learning of large networks and a simplification in scoring the discrete case which performs well empirically. While the core of this work focuses on applications in the search and score paradigm, we also show how the introduced scoring functions may be readily adapted as conditional independence tests for constraint-based Bayesian network learning algorithms. Lastly, we describe ways to simulate networks of mixed variable types and evaluate our proposed methods on such simulations.

Using targeted Bayesian network learning for suspect identification in communication networks

This paper proposes a machine learning application to identify mobile phone users suspected of involvement in criminal activities. The application characterizes the behavioral patterns of suspect users versus non-suspect users based on usage metadata such as call duration, call distribution, interaction time preferences and text-to-call ratios while avoiding any access to the content of calls or messages. The application is based on targeted Bayesian network learning method. It generates a graphical network that can be used by domain experts to gain intuitive insights about the key features that can help identify suspect users. The method enables experts to manage the trade-off between model complexity and accuracy using information theory metrics. Unlike other graphical Bayesian classifiers, the proposed application accomplishes the task required of a security company, namely an accurate suspect identification rate (recall) of at least 50% with no more than a 1% false identification rate. The targeted Bayesian network learning method is also used for additional tasks such as anomaly detection, distinction between “relevant” and “irrelevant” anomalies, and for associating anonymous telephone numbers with existing users by matching behavioral patterns.

A new hybrid method for learning bayesian networks: Separation and reunion

Most existing algorithms for learning Bayesian networks (BNs) can be categorized as constraint-based or score-based methods. Constraint-based algorithms use conditional independence (CI) tests to judge the presence or absence of an edge. Though effective and applicable to (high-dimensional data) large-scale networks, CI tests require a large number of samples to determine the independencies. Thus they can be unreliable especially when the sample size is small. On the other hand, score-based methods employ a score metric to evaluate each candidate network structure, but they are inefficient in learning large-scale networks due to the extremely large search space. In this paper, we propose a new hybrid Bayesian network learning method, SAR (the abbreviation of Separation And Reunion), which maintains the merits of both types of learning techniques while avoiding their drawbacks. Extensive experiments show that our method generally achieves better performance than state-of-the-art methods.

Verification of Three-Phase Dependency Analysis Bayesian Network Learning Method for Maize Carotenoid Gene Mining.

Background and Objective Mining the genes related to maize carotenoid components is important to improve the carotenoid content and the quality of maize. Methods On the basis of using the entropy estimation method with Gaussian kernel probability density estimator, we use the three-phase dependency analysis (TPDA) Bayesian network structure learning method to construct the network of maize gene and carotenoid components traits. Results In the case of using two discretization methods and setting different discretization values, we compare the learning effect and efficiency of 10 kinds of Bayesian network structure learning methods. The method is verified and analyzed on the maize dataset of global germplasm collection with 527 elite inbred lines. Conclusions The result confirmed the effectiveness of the TPDA method, which outperforms significantly another 9 kinds of Bayesian network learning methods. It is an efficient method of mining genes for maize carotenoid components traits. The parameters obtained by experiments will help carry out practical gene mining effectively in the future.

Reconstruction of metabolic pathways by combining probabilistic graphical model-based and knowledge-based methods.

Automatic reconstruction of metabolic pathways for an organism from genomics and transcriptomics data has been a challenging and important problem in bioinformatics. Traditionally, known reference pathways can be mapped into an organism-specific ones based on its genome annotation and protein homology. However, this simple knowledge-based mapping method might produce incomplete pathways and generally cannot predict unknown new relations and reactions. In contrast, ab initio metabolic network construction methods can predict novel reactions and interactions, but its accuracy tends to be low leading to a lot of false positives.Here we combine existing pathway knowledge and a new ab initio Bayesian probabilistic graphical model together in a novel fashion to improve automatic reconstruction of metabolic networks. Specifically, we built a knowledge database containing known, individual gene / protein interactions and metabolic reactions extracted from existing reference pathways. Known reactions and interactions were then used as constraints for Bayesian network learning methods to predict metabolic pathways. Using individual reactions and interactions extracted from different pathways of many organisms to guide pathway construction is new and improves both the coverage and accuracy of metabolic pathway construction. We applied this probabilistic knowledge-based approach to construct the metabolic networks from yeast gene expression data and compared its results with 62 known metabolic networks in the KEGG database. The experiment showed that the method improved the coverage of metabolic network construction over the traditional reference pathway mapping method and was more accurate than pure ab initio methods.

Restricted Bayesian classification networks

Bayesian networks are graphical models that describe dependency relationships between variables, and are powerful tools for studying probability classifiers. At present, the causal Bayesian network learning method is used in constructing Bayesian network classifiers while the contribution of attribute to class is overlooked. In this paper, a Bayesian network specifically for classification-restricted Bayesian classification networks is proposed. Combining dependency analysis between variables, classification accuracy evaluation criteria and a search algorithm, a learning method for restricted Bayesian classification networks is presented. Experiments and analysis are done using data sets from UCI machine learning repository. The results show that the restricted Bayesian classification network is more accurate than other well-known classifiers.

A hybrid method for learning Bayesian networks based on ant colony optimization

As a powerful formalism, Bayesian networks play an increasingly important role in the Uncertainty Field. This paper proposes a hybrid method to discover the knowledge represented in Bayesian networks. The hybrid method combines dependency analysis, ant colony optimization (ACO), and the simulated annealing strategy. Firstly, the new method uses order-0 independence tests with a self-adjusting threshold value to reduce the size of the search space, so that the search process takes less time to find the near-optimal solution. Secondly, better Bayesian network models are generated by using an improved ACO algorithm, where a new heuristic function is introduced to further enhance the search effectiveness and efficiency. Finally, an optimization scheme based on simulated annealing is employed to improve the optimization efficiency in the stochastic search process of ants. In a number of experiments and comparisons, the hybrid method outperforms the original ACO-B which uses ACO and some other network learning algorithms.

Impact of censoring on learning Bayesian networks in survival modelling

Bayesian networks are commonly used for presenting uncertainty and covariate interactions in an easily interpretable way. Because of their efficient inference and ability to represent causal relationships, they are an excellent choice for medical decision support systems in diagnosis, treatment, and prognosis. Although good procedures for learning Bayesian networks from data have been defined, their performance in learning from censored survival data has not been widely studied. In this paper, we explore how to use these procedures to learn about possible interactions between prognostic factors and their influence on the variate of interest. We study how censoring affects the probability of learning correct Bayesian network structures. Additionally, we analyse the potential usefulness of the learnt models for predicting the time-independent probability of an event of interest. We analysed the influence of censoring with a simulation on synthetic data sampled from randomly generated Bayesian networks. We used two well-known methods for learning Bayesian networks from data: a constraint-based method and a score-based method. We compared the performance of each method under different levels of censoring to those of the naive Bayes classifier and the proportional hazards model. We did additional experiments on several datasets from real-world medical domains. The machine-learning methods treated censored cases in the data as event-free. We report and compare results for several commonly used model evaluation metrics. On average, the proportional hazards method outperformed other methods in most censoring setups. As part of the simulation study, we also analysed structural similarities of the learnt networks. Heavy censoring, as opposed to no censoring, produces up to a 5% surplus and up to 10% missing total arcs. It also produces up to 50% missing arcs that should originally be connected to the variate of interest. Presented methods for learning Bayesian networks from data can be used to learn from censored survival data in the presence of light censoring (up to 20%) by treating censored cases as event-free. Given intermediate or heavy censoring, the learnt models become tuned to the majority class and would thus require a different approach.

Exploiting association and correlation rules parameters for learning Bayesian networks

In data mining, association and correlation rules are inferred from data in order to highlight statistical dependencies among attributes. The metrics defined for evaluating these rules can be exploited to score relationships between attributes in Bayesian network learning. In this paper, we propose two novel methods for learning Bayesian networks from data that are based on the K2 learning algorithm and that improve it by exploiting parameters normally defined for association and correlation rules. In particular, we propose the algorithms K2-Lift and K2-$X^{2}$, that exploit the lift metric and the $X^2$ metric respectively. We compare K2-Lift, K2-$X^{2}$ with K2 on artificial data and on three test Bayesian networks. The experiments show that both our algorithms improve K2 with respect to the quality of the learned network. Moreover, a comparison of K2-Lift and K2-$X^{2}$ with a genetic algorithm approach on two benchmark networks show superior results on one network and comparable results on the other.

A heuristic method for learning Bayesian networks using discrete particle swarm optimization

Bayesian networks are a powerful approach for representing and reasoning under conditions of uncertainty. Many researchers aim to find good algorithms for learning Bayesian networks from data. And the heuristic search algorithm is one of the most effective algorithms. Because the number of possible structures grows exponentially with the number of variables, learning the model structure from data by considering all possible structures exhaustively is infeasible. PSO (particle swarm optimization), a powerful optimal heuristic search algorithm, has been applied in various fields. Unfortunately, the classical PSO algorithm only operates in continuous and real-valued space, and the problem of Bayesian networks learning is in discrete space. In this paper, two modifications of updating rules for velocity and position are introduced and a Bayesian networks learning based on binary PSO is proposed. Experimental results show that it is more efficient because only fewer generations are needed to obtain optimal Bayesian networks structures. In the comparison, this method outperforms other heuristic methods such as GA (genetic algorithm) and classical binary PSO.

BNFinder: exact and efficient method for learning Bayesian networks

Motivation: Bayesian methods are widely used in many different areas of research. Recently, it has become a very popular tool for biological network reconstruction, due to its ability to handle noisy data. Even though there are many software packages allowing for Bayesian network reconstruction, only few of them are freely available to researchers. Moreover, they usually require at least basic programming abilities, which restricts their potential user base. Our goal was to provide software which would be freely available, efficient and usable to non-programmers.Results: We present a BNFinder software, which allows for Bayesian network reconstruction from experimental data. It supports dynamic Bayesian networks and, if the variables are partially ordered, also static Bayesian networks. The main advantage of BNFinder is the use exact algorithm, which is at the same time very efficient (polynomial with respect to the number of observations).Availability: The software, supplementary information and manual is available at http://bioputer.mimuw.edu.pl/software/bnf/. Besides the availability of the standalone application and the source code, we have developed a web interface to BNFinder application running on our servers. A web tutorial on different options of BNFinder is also available.Contact: dojer@mimuw.edu.pl

Learning Bayesian networks from incomplete databases using a novel evolutionary algorithm

This paper proposes a novel method for learning Bayesian networks from incomplete databases in the presence of missing values, which combines an evolutionary algorithm with the traditional Expectation Maximization (EM) algorithm. A data completing procedure is presented for learning and evaluating the candidate networks. Moreover, a strategy is introduced to obtain better initial networks to facilitate the method. The new method can also overcome the problem of getting stuck in sub-optimal solutions which occurs in most existing learning algorithms. The experimental results on the databases generated from several benchmark networks illustrate that the new method has better performance than some state-of-the-art algorithms. We also apply the method to a data mining problem and compare the performance of the discovered Bayesian networks with the models generated by other learning algorithms. The results demonstrate that our method outperforms other algorithms.