Learning Bayesian Networks Research Articles

Novel statistically equivalent signature-based hybrid feature selection and ensemble deep learning LSTM and GRU for chronic kidney disease classification

Chronic kidney disease (CKD) involves numerous variables, but only a few significantly impact the classification task. The statistically equivalent signature (SES) method, inspired by constraint-based learning of Bayesian networks, is employed to identify essential features in CKD. Unlike conventional feature selection methods, which typically focus on a single set of features with the highest predictive potential, the SES method can identify multiple predictive feature subsets with similar performance. However, most feature selection (FS) classifiers perform suboptimally with strongly correlated data. The FS approach faces challenges in identifying crucial features and selecting the most effective classifier, particularly in high-dimensional data. This study proposes using the Least Absolute Shrinkage and Selection Operator (LASSO) in conjunction with the SES method for feature selection in CKD identification. Following this, an ensemble deep-learning model combining long short-term memory (LSTM) and gated recurrent unit (GRU) networks is proposed for CKD classification. The features selected by the hybrid feature selection method are fed into the ensemble deep-learning model. The model’s performance is evaluated using accuracy, precision, recall, and F1 score metrics. The experimental results are compared with individual classifiers, including decision tree (DT), Random Forest (RF), logistic regression (LR), and support vector machine (SVM). The findings indicate a 2% improvement in classification accuracy when using the proposed hybrid feature selection method combined with the LSTM and GRU ensemble deep-learning model. Further analysis reveals that certain features, such as HEMO, POT, bacteria, and coronary artery disease, contribute minimally to the classification task. Future research could explore additional feature selection methods, including dynamic feature selection that adapts to evolving datasets and incorporates clinical knowledge to enhance CKD classification accuracy further.

Assessing Credibility in Bayesian Networks Structure Learning.

Learning Bayesian networks from data aims to create a Directed Acyclic Graph that encodes significant statistical relationships between variables and their joint probability distributions. However, when using real-world data with limited knowledge of the original dynamical system, it is challenging to determine if the learned DAG accurately reflects the underlying relationships, especially when the data come from multiple independent sources. This paper describes a methodology capable of assessing the credible interval for the existence and direction of each edge within Bayesian networks learned from data, without previous knowledge of the underlying dynamical system. It offers several advantages over classical methods, such as data fusion from multiple sources, identification of latent variables, and extraction of the most prominent edges with their respective credible interval. The method is evaluated using simulated datasets of various sizes and a real use case. Our approach was verified to achieve results comparable to the most recent studies in the field, while providing more information on the model's credibility.

Distributed fusion-based algorithms for learning high-dimensional Bayesian Networks: Testing ring and star topologies

Learning Bayesian Networks (BNs) from high-dimensional data is a complex and time-consuming task. Although there are approaches based on horizontal (instances) or vertical (variables) partitioning in the literature, none can guarantee the same theoretical properties as the Greedy Equivalence Search (GES) algorithm, except those based on the GES algorithm itself. This paper proposes a parallel distributed framework that uses GES as its local learning algorithm, obtaining results similar to those of GES and guaranteeing its theoretical properties but requiring less execution time. The framework involves splitting the set of all possible edges into clusters and constraining each framework node to only work with the received subset of edges. The global learning process is an iterative algorithm that carries out rounds until a convergence criterion is met. We have designed a ring and a star topology to distribute node connections. Regardless of the topology, each node receives a BN as input; it then fuses it with its own BN model and uses the result as the starting point for a local learning process, limited to its own subset of edges. Once finished, the result is then sent to another node as input. Experiments were carried out on a large repertory of domains, including large BNs up to more than 1000 variables. Our results demonstrate our proposal's effectiveness compared to GES and its fast version (fGES), generating high-quality BNs in less execution time.

Updating strategy of safe operation control model for dense medium coal preparation process based on Bayesian network and incremental learning

AbstractThe effectiveness of control decisions provided by the safe operation control model for the dense medium coal preparation process may decline due to its inability to adapt to changing working conditions. To address this issue, this paper investigates a safe operation control model update strategy based on Bayesian network and incremental learning. This strategy can update the model structure and parameters according to different conditions, ensuring the effectiveness of the updated model. Considering that the old model has effective information to explain the new working conditions, the Bayesian network structure update learning method based on incremental learning is proposed. This method retains the components of the old model that can describe the joint probability distribution of the sampled data under the new working conditions while updating the remaining structure. This approach improves the efficiency of model updating. The simulation results show that the updated model obtained by the proposed method can effectively deal with new abnormal conditions.

Generative broad Bayesian (GBB) imputer for missing data imputation with uncertainty quantification

Generative broad Bayesian (GBB) imputer, a novel nonparametric data-driven tool for missing data imputation with uncertainty quantification, is proposed. The proposed imputer aims to generate missing data in an iterative manner based on an augmentable broad Bayesian learning network. The procedure consists of the preparatory and tuning phase. The preparatory phase provides preliminary imputation of the missing data to develop a complete dataset. The tuning phase refines the accuracy of the imputation results based on the augmented learning network. There are three appealing features of the proposed GBB imputer: (i) the nonparametric generative scheme provides a universal tool for missing data imputation without constraints on the type of data attribution, missing data pattern, or requirement of the prior information about the dataset; (ii) the quantified uncertainty of the imputation results reflects the associated reliability and provides a rational termination indicator for the iterative imputation procedure; and (iii) the learning network can be augmented progressively to adopt architectural reconfigurations based on the inherited information of the trained network for efficient imputation. To demonstrate the efficacy and applicability of the proposed GBB imputer, we present two simulated examples under various scenarios and a case study with the achieved in-situ seismic records of the 2016 Mw6.5 Norcia earthquake.

Exploring the Influence of Human System Interfaces: Introducing Support Tools and an Experimental Study

Situational awareness and decision support tools such as procedures and alarm systems are vital for effective interaction among control room operators, especially in safety-critical situations. In safety-critical environments such as process plants, there remains a gap in evaluating specific tools during actual operations, or ”work-as-done.” Additionally, the underlying factors that might impact operators’ cognitive states and performance concerning safety have not been thoroughly explored. The need for such an evaluation is further bolstered by current interaction configurations where operators are more passive than active, thus reducing their cognitive performance. Therefore, this experimental study addresses the highlighted evaluation gap by introducing and comparing three human system interfaces/decision support tools in four human-in-the-loop configurations. The supports include two alarm design formats (prioritized vs. non-prioritized) and three procedure representation formats (paper, screen-based digitized, and an AI-based support system built with an integrated Bayesian network and reinforcement learning model). Ninety-two people (n = 92) participated voluntarily in the test. They were divided equally into four groups. Each group tested three safety-related events in a simulated formaldehyde production facility. Individuals belonging to the group with prioritized alarms and utilized paper procedures rated procedural support slightly higher on average than others in different groups. Unlike the other groups, their assessment of alarm prioritization support remained consistent across all scenarios. Further analysis of the impact of the setup on cognitive states and actual performance will be performed.

A Scalable Accelerator for Local Score Computation of Structure Learning in Bayesian Networks

A Bayesian network is a powerful tool for representing uncertainty in data, offering transparent and interpretable inference, unlike neural networks’ black-box mechanisms. To fully harness the potential of Bayesian networks, it is essential to learn the graph structure that appropriately represents variable interrelations within data. Score-based structure learning, which involves constructing collections of potentially optimal parent sets for each variable, is computationally intensive, especially when dealing with high-dimensional data in discrete random variables. Our proposed novel acceleration algorithm extracts high levels of parallelism, offering significant advantages even with reduced reusability of computational results. In addition, it employs an elastic data representation tailored for parallel computation, making it FPGA-friendly and optimizing module occupancy while ensuring uniform handling of diverse problem scenarios. Demonstrated on a Xilinx Alveo U50 FPGA, our implementation significantly outperforms optimal CPU algorithms and is several times faster than GPU implementations on an NVIDIA TITAN RTX. Furthermore, the results of performance modeling for the accelerator indicate that, for sufficiently large problem instances, it is weakly scalable, meaning that it effectively utilizes increased computational resources for parallelization. To our knowledge, this is the first study to propose a comprehensive methodology for accelerating score-based structure learning, blending algorithmic and architectural considerations.

Time-varying dynamic Bayesian network learning for an fMRI study of emotion processing.

This article presents a novel method for learning time-varying dynamic Bayesian networks. The proposed method breaks down the dynamic Bayesian network learning problem into a sequence of regression inference problems and tackles each problem using the Markov neighborhood regression technique. Notably, the method demonstrates scalability concerning data dimensionality, accommodates time-varying network structure, and naturally handles multi-subject data. The proposed method exhibits consistency and offers superior performance compared to existing methods in terms of estimation accuracy and computational efficiency, as supported by extensive numerical experiments. To showcase its effectiveness, we apply the proposed method to an fMRI study investigating the effective connectivity among various regions of interest (ROIs) during an emotion-processing task. Our findings reveal the pivotal role of the subcortical-cerebellum in emotion processing.

Risk analysis of lithium-ion battery accidents based on physics-informed data-driven Bayesian networks

The catastrophic consequences of lithium-ion battery (LIB) accidents have attracted high attention from society and industry. Accordingly, risk analysis is indispensable for the risk prevention and control of LIBs. Nevertheless, it is difficult to establish a physics-informed risk analysis model due to the complex material characteristics and aging mechanisms of LIBs. Meanwhile, the data-driven approach requires historical information of LIBs and does not merely rely on knowledge of the internal mechanisms of LIBs. This study proposes a method integrating the physics-informed Bayesian network (BN) (i.e., mapping from fault tree) and data-driven BN (i.e., learning from data) to conduct risk analysis of LIBs. First, we establish physics-informed and data-driven BNs. Subsequently, we bridge physics-informed and data-driven BNs to establish a Bayesian network for risk analysis of LIB accidents. Second, we set up safety barriers in the system, including detectors, emergency response, and firefighting facilities. Third, we evaluate the performance of safety barriers. We validate the proposed model using data from LIBs in air transportation. The results indicate that safety barriers can reduce the accidental risk of LIBs. Eventually, we propose suggestions for the risk control of LIBs in air transportation. This study is supposed to provide theoretical basis for the risk prevention and control of LIB accidents.

Bayesian network structure learning using scatter search

Learning the structure of Bayesian networks (BNs) from data is an NP-hard problem as the solution space grows super-exponentially with the number of nodes. Many algorithms have been developed to efficiently find the best structures, with score-based algorithms being those that use heuristics or metaheuristics to explore potential structures in the search space. This paper proposes a new score-based algorithm that relies on a well-known metaheuristic called scatter search, which, to the best of our knowledge, has not been used in learning BN structure. The core of scatter search is to maintain a reference set that stores both high-quality and diverse solutions, thereby continuously tracking and improving the high-quality solutions, while exploring different search directions indicated by the diverse solutions. By incorporating a distance metric in the learning process, the exploration can be more systematic than purely random, as is often the case in most existing algorithms. The effectiveness and efficiency of the proposed algorithm are evaluated through computational experiments. In addition to learning higher-score structures, the results show that scatter search provides a higher degree of robustness compared with benchmark algorithms.

Optimizing Construction Engineering Management Using Metaheuristic Methods and Bayesian Networks

The construction of buildings invariably involves time and costs, and disruptions impact ongoing construction projects. Crisis situations in management strategies, structural confusion, and financial miscalculations often arise due to misguided decision-making. This article proposes a method that combines the learning of Bayesian Networks and heuristic techniques to optimize decision-making processes in construction scheduling. As an innovative approach in order to enhance construction management, the functioning of biological, molecular, and physical objects and nervous systems is considered, applying bionic features to mimic their efficiency and precision, thereby optimizing construction processes and improving coordination and decision-making. Bayesian Networks are used for probabilistic analysis, and heuristic methods guide quick decision-making. The results demonstrate the effectiveness of Bayesian Networks and heuristic methods in data analysis and decision-making in construction engineering. The developed algorithm can be successfully applied to both erecting and planning construction projects.

Causal inference approaches reveal both positive and negative unintended effects of agricultural and urban management practices on instream biological condition

Agricultural and urban management practices (MPs) are primarily designed and implemented to reduce nutrient and sediment concentrations in streams. However, there is growing interest in determining if MPs produce any unintended positive effects, or co-benefits, to instream biological and habitat conditions. Identifying co-benefits is challenging though because of confounding variables (i.e., those that affect both where MPs are applied and stream biota), which can be accounted for in novel causal inference approaches. Here, we used two causal inference approaches, propensity score matching (PSM) and Bayesian network learning (BNL), to identify potential MP co-benefits in the Chesapeake Bay watershed portion of Maryland, USA. Specifically, we examined how MPs may modify instream conditions that impact fish and macroinvertebrate indices of biotic integrity (IBI) and functional and taxonomic endpoints. We found evidence of positive unintended effects of MPs for both benthic macroinvertebrates and fish indicated by higher IBI scores and specific endpoints like the number of scraper macroinvertebrate taxa and lithophilic spawning fish taxa in a subset of regions. However, our results also suggest MPs have negative unintended effects, especially on sensitive benthic macroinvertebrate taxa and key instream habitat and water quality metrics like specific conductivity. Overall, our results suggest MPs offer co-benefits in some regions and catchments with largely degraded conditions but can have negative unintended effects in some regions, especially in catchments with good biological conditions. We suggest the number and types of MPs drove these mixed results and highlight carefully designed MP implementation that incorporates instream biological data at the catchment scale could facilitate co-benefits to instream biological conditions. Our study underscores the need for more research on identifying effects of individual MP types on instream biological and habitat conditions.

62 Implications of factor analyses and Bayesian network learning to develop composite traits of size attributes in developing beef heifers

Abstract Advancements in phenotypic data collection methods provide opportunities to capture complex biological traits more efficiently. Even so, interpreting those phenotypes and using them in selection decisions is challenging in beef cattle production. Objectives of this study were to 1) develop composite phenotype(s) of economic importance in beef heifers given attributes of their body conformation and ultrasound-based measures of reproductive and carcass traits, and 2) identify associated genomic regions under selection using the structure equation modeling (SEM) approach. In this study, we hypothesized that the interplay of factor analysis and Bayesian Network Learning (BNL) approaches would better define the complex composite traits for size and carcass-related traits in beef heifers. Phenotypic data of admixed beef heifers (n = 336) for different reproductive, body conformational, and carcass-associated phenotypic traits (n = 16) were collected. Using exploratory and confirmatory factor analyses, two latent variables explaining 14 phenotypic traits were identified, herein called Body Size (BS; n = 7 traits represented) and Body Composition (BC; n = 7 traits represented). To efficiently model them in genome-wide association studies (GWAS), fixed effects of data collection year, dam age, and primary breed group as well as causal network structure identified through BNL were used within SEM model of GWAS. Body Size was directly or indirectly contributing to BC based on BNL structure. Given the data-driven approach and novelty of the data, BayesCπ model was applied with the sliding window of 1 Mb to avoid noise and linkage disequilibrium implications. Regions capturing large amounts of genetic variance (Posterior Probability of Association, PPA ≥ 0.8) were mapped to their closest gene for enrichment analysis. Twenty-four different genes were identified associated with both composite phenotypes. Functional enrichment analysis of significantly involved pathways (FDR < 0.05) indicated that genes from the HERC family (HERC3, HERC6, and HERC5) were found involved in cellular growth, and those from desmosomes (DC2, and DC3) were involved in morphogenesis like BS. Genes related to energy metabolism like IMPAD1, and PIGY, along with genes associated with traits like muscular development, and other carcass traits (i.e., FAM184B, NCAPG, and LCORL), were also found associated with BS and BC. Heritability ranged from 0.551 ± 0.048 to 0.897 ± 0.145 for BS and BC, respectively. The combined heritability of reproductive and carcass traits (BC) was much greater than the heritability of any related individual trait reported to date and was comparable for individual BS-related traits. Given this, we conclude that the implications of factor analyses with BNL can provide better biological insights for improved genomic selection in beef cattle, even for traits of low heritability.

Being Bayesian about learning Bayesian networks from ordinal data

In this paper we propose a Bayesian approach for inferring Bayesian network (BN) structures from ordinal data. Our approach can be seen as the Bayesian counterpart of a recently proposed frequentist approach, referred to as the ‘ordinal structure expectation maximization’ (OSEM) method. Like for the OSEM method, the key idea is to assume that each ordinal variable originates from a Gaussian variable that can only be observed in discretized form, and that the dependencies in the latent Gaussian space can be modeled by BNs; i.e. by directed acyclic graphs (DAGs). Our Bayesian method combines the ‘structure MCMC sampler’ for DAG posterior sampling, a slightly modified version of the ‘Bayesian metric for Gaussian networks having score equivalence’ (BGe score), the concept of the ‘extended rank likelihood’, and a recently proposed algorithm for posterior sampling the parameters of Gaussian BNs. In simulation studies we compare the new Bayesian approach and the OSEM method in terms of the network reconstruction accuracy. The empirical results show that the new Bayesian approach leads to significantly improved network reconstruction accuracies.

VertiBayes: learning Bayesian network parameters from vertically partitioned data with missing values

Federated learning makes it possible to train a machine learning model on decentralized data. Bayesian networks are widely used probabilistic graphical models. While some research has been published on the federated learning of Bayesian networks, publications on Bayesian networks in a vertically partitioned data setting are limited, with important omissions, such as handling missing data. We propose a novel method called VertiBayes to train Bayesian networks (structure and parameters) on vertically partitioned data, which can handle missing values as well as an arbitrary number of parties. For structure learning we adapted the K2 algorithm with a privacy-preserving scalar product protocol. For parameter learning, we use a two-step approach: first, we learn an intermediate model using maximum likelihood, treating missing values as a special value, then we train a model on synthetic data generated by the intermediate model using the EM algorithm. The privacy guarantees of VertiBayes are equivalent to those provided by the privacy preserving scalar product protocol used. We experimentally show VertiBayes produces models comparable to those learnt using traditional algorithms. Finally, we propose two alternative approaches to estimate the performance of the model using vertically partitioned data and we show in experiments that these give accurate estimates.

Parallel structural learning of Bayesian networks: Iterative divide and conquer algorithm based on structural fusion

Learning Bayesian Networks (BNs) from high-dimensional data is a complex and time-consuming task. Although the literature includes approaches based on horizontal (instances) or vertical (variables) partitioning, none can guarantee the same theoretical properties as the Greedy Equivalence Search (GES) algorithm, except those based on the GES algorithm itself. This paper proposes a distributed BN learning algorithm that uses GES as the local learning algorithm, ensuring the same theoretical properties as GES but requiring less CPU time. The two main novelties in our proposed method are (1) the distribution of the set of possible edges among local learning processes, which are constrained to only use its local edge set; and (2) the use of BN fusion to aggregate the networks learned constrained to local edge sets. The algorithm is iterative, and at each step, the last aggregated network is used as the starting point by each local BN process. After a comprehensive experimental evaluation, the results show that the proposed algorithm (pGES) obtains networks of equal or better quality than GES in less computational time. This improvement is especially noticeable in high-dimensional BNs.

Learning Bayesian Networks: A Copula Approach for Mixed-Type Data.

Estimating dependence relationships between variables is a crucial issue in many applied domains and in particular psychology. When several variables are entertained, these can be organized into a network which encodes their set of conditional dependence relations. Typically however, the underlying network structure is completely unknown or can be partially drawn only; accordingly it should be learned from the available data, a process known as structure learning. In addition, data arising from social and psychological studies are often of different types, as they can include categorical, discrete and continuous measurements. In this paper, we develop a novel Bayesian methodology for structure learning of directed networks which applies to mixed data, i.e.,possibly containing continuous, discrete, ordinal and binary variables simultaneously. Whenever available, our method can easily incorporate known dependence structures among variables represented by paths or edge directions that can be postulated in advance based on the specific problem under consideration. We evaluate the proposed method through extensive simulation studies, with appreciable performances in comparison with current state-of-the-art alternative methods. Finally, we apply our methodology to well-being data from a social survey promoted by the United Nations, and mental health data collected from a cohort of medical students. R code implementing the proposed methodology is available at https://github.com/FedeCastelletti/bayes_networks_mixed_data .

Credit risk prediction based on causal machine learning: Bayesian network learning, default inference, and interpretation

AbstractThe predictive and interpretable power of models is crucial for financial risk management. The purpose of this study was to perform credit risk prediction in a structured causal network with four stages—data processing, structural learning, parameter learning, and interpretation of inferences—and use six real credit datasets to conduct empirical research on the proposed model. Compared with traditional machine learning algorithms, we comprehensively explain the results of credit default through forward and reverse reasoning. We also compared our model with the post hoc interpretation models local interpretable model‐agnostic explanations (LIME) and shapley additive explanations (SHAP) to verify the interpretability of Bayesian networks. The experimental results show that the prediction performance of Bayesian networks is superior to traditional machine learning models and similar to the performance of ensemble models. Furthermore, Bayesian networks offer valuable insights into the interplay of features by considering their causal relationships and enable an assessment of how individual features influence the prediction outcome. In this study, what‐if analysis was performed to assess credit default probabilities under various conditions. This analysis provides decision‐makers with the necessary tools to make informed judgments about the risk profile of borrowers. Consequently, we consider Bayesian networks as a viable tool for credit risk prediction models in terms of prediction performance and interpretability.

Learning bayesian network parameters from limited data by integrating entropy and monotonicity

High-accuracy parameter learning in Bayesian Networks (BNs) is a key challenge in real-time decision support applications, particularly when the available data are limited. Prior/Expert knowledge was introduced to eliminate the drawbacks of insufficient information; however, this method is subjective. In this study, we explored the use of monotonicity constraints to control the causal relationships between the nodes and their parents-nodes in BNs and proposed a new learning algorithm called global domain monotonicity based on maximum information entropy (GDM-MIE), which was designed for parameter learning in uncertain discrete BNs with nonlinear equality constraints when only finite data are available. In the proposed algorithm, a class of monotonicity is encoded as a constraint on information entropy and the parameter learning problem is transformed into constraints among the node parameters based on a known network. Furthermore, we considered the parameters as uncertain entropy information and discussed the monotonicity among parameters in the global spatial domain, proving the accuracy of the logical relationships of the model, system reliability, and time complexity. Finally, the proposed method was validated using standard BNs, and its performance was analyzed by comparing the proposed method with the existing learning algorithms. The results showed that the proposed method is more accurate and has better Kullback–Leibler divergence. To revalidate the rationality of the proposed method, the Alarm and Asia networks were employed as special cases. The GDM-MIE was found to achieve the intended goal of observing the estimated parameters by closely approximating the original real parameters with a small sample size, indicating that the proposed algorithm can served as an efficient and feasible method for learning Bayesian parameter.

Bayesian network structure learning based on HC-PSO algorithm

Structure learning is the core of graph model Bayesian Network learning, and the current mainstream single search algorithm has problems such as poor learning effect, fuzzy initial network, and easy falling into local optimum. In this paper, we propose a heuristic learning algorithm HC-PSO combining the HC (Hill Climbing) algorithm and PSO (Particle Swarm Optimization) algorithm, which firstly uses HC algorithm to search for locally optimal network structures, takes these networks as the initial networks, then introduces mutation operator and crossover operator, and uses PSO algorithm for global search. Meanwhile, we use the DE (Differential Evolution) strategy to select the mutation operator and crossover operator. Finally, experiments are conducted in four different datasets to calculate BIC (Bayesian Information Criterion) and HD (Hamming Distance), and comparative analysis is made with other algorithms, the structure shows that the HC-PSO algorithm is superior in feasibility and accuracy.