Bayesian Network Structure Learning Research Articles

Model-Driven Bayesian Network Learning for Factory-Level Fault Diagnostics and Resilience

We propose to use engineering models for Bayesian Network (BN) learning for fault diagnostics at the factory-level using key performance indicators (KPIs) such as overall equipment effectiveness (OEE). OEE is widely used in industry and it measures sustainability by capturing product quality (e.g., less scrap) and measures resilience by capturing availability. A major advantage of the proposed approach is that the engineering models are likely to be available long before the corresponding digitalized smart factory becomes fully operational. Specifically, for BN structure learning, we propose to use analytical queueing theory models of the factory to elicit the structure, and to carry out intervention we propose to use designed experiments based on discrete-event simulation models of the factory. For parameter learning, we apply a qualitative maximum a posteriori (QMAP) method and propose additional expert constraints based on the law of propagation of uncertainty from queueing theory. Furthermore, the proposed approach overcomes the challenge of obtaining balanced-class data in BN learning for fault diagnostics. We apply the proposed BN learning approach to (i) a 4-robot cell in our laboratory and (ii) a robotic machining cell in a commercial vehicle factory. In both cases, the proposed method is found to be efficacious in accurately learning the BN structure and parameter, as measured using structural-hamming distance and Kullback–Leibler divergence score, respectively. The proposed approach can pave the way for a new class of resilient and sustainable smart manufacturing systems.

Learning and Optimization of Patient–Physician Matching Index in Specialty Care

It is challenging for a patient without medical knowledge to select a ‘capable’ physician based on nontransparent medical information. The situation becomes pervasive in specialty care. Motivated by this existing patient-physician matching problem, we propose a novel physician matching index (PMI) obtained by an analytical framework integrated with an improved multi-disease pre-diagnosing Bayesian network (BN) model. The pre-diagnosis BN structure learning is critical since it provides the causal map among diseases and symptoms, but it has been proved to be NP-hard. To improve the computational tractability of the BN structure learning, we propose a dynamic programming based cache calculation algorithm integrated with expert knowledge. The optimal BN structure is obtained through an improved branch-and-bound algorithm. Given patients’ symptoms and physicians’ specialty information, we apply the trained pre-diagnosis BN model to obtain PMI, which can be extended to the weighted matching index by considering patient preferences. A case study of the patient-physician matching problem in the ear, nose, and throat (ENT) department is conducted. The branch-and-bound algorithm with the proposed cache calculation algorithm learns the optimal BN structure with high pre-diagnosing accuracy and time efficiency. We disclose that the proposed PMI can rectify the misdiagnosis since the highly related diseases usually belong to one specialty. Moreover, we demonstrate the significance of the consistency between the physicians’ specialty and the patients’ disease distribution. We also highlight that the proposed PMI guides the patients in choosing physicians more appropriately under independent patient preferences. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —This research is motivated by a potential healthcare quality issue caused by the mismatch between patient and physician in specialty care, which is critical for both referral and non-referral systems. We propose a novel physician matching index (PMI) according to patients’ symptoms and physicians’ specialties based on potential disease(s) pre-diagnosed by a Bayesian network model. In the case study of the ear, nose, and throat (ENT) department, we show that PMI can rectify a large portion of misdiagnosed cases. Under the consideration of independent patient preferences, we also find that PMI improves the matching performance. To improve the systematic matching outcome, we suggest that physicians’ specialty cross-training programs be consistent with the patient’s disease distribution. However, we assume that patients’ symptoms are known and physicians’ specialties are clearly described, which might need to be further investigated. The framework proposed in this research can be extended to other specialty care departments, e.g., neurology and orthopedics.

Analysis of the Impact of Curriculum Civics on Students’ Quality Cultivation Based on Bayesian Networks

Abstract This paper first analyzes the Bayesian network model based on probabilistic reasoning to study and explore the random phenomenon law of curriculum thinking. Secondly, probability theory and graph theory are utilized to deal with uncertain information and reduce the meta-multiplicity of the analysis of uncertain knowledge. Then, the Bayesian network structure learning problem is analyzed, and a constraint-based structure learning method is proposed to score students’ qualities by a given scoring function, and using a search algorithm, the best scoring network structure is searched. Finally, in order to analyze the influence of the Civics curriculum on the quality of students in college A, we should analyze the impact of the curriculum. The results show that from the innovative quality of students, the distribution density of innovative habit and innovative ability scores in the interval of 70~75 is (0.02, 0.035), and the distribution density of innovative consciousness scores in the interval of 70~86.25 is (0.005, 0.07). The data show that curriculum Civics can effectively promote the cultivation of innovative talents and the construction of an innovative country by internalizing the quality of innovation in the heart and externalizing it in the form of cultivating students’ innovative habits, improving their innovative consciousness and enhancing their innovative ability.

Bayesian Network Structural Learning Using Adaptive Genetic Algorithm with Varying Population Size

A Bayesian network (BN) is a probabilistic graphical model that can model complex and nonlinear relationships. Its structural learning from data is an NP-hard problem because of its search-space size. One method to perform structural learning is a search and score approach, which uses a search algorithm and structural score. A study comparing 15 algorithms showed that hill climbing (HC) and tabu search (TABU) performed the best overall on the tests. This work performs a deeper analysis of the application of the adaptive genetic algorithm with varying population size (AGAVaPS) on the BN structural learning problem, which a preliminary test showed that it had the potential to perform well on. AGAVaPS is a genetic algorithm that uses the concept of life, where each solution is in the population for a number of iterations. Each individual also has its own mutation rate, and there is a small probability of undergoing mutation twice. Parameter analysis of AGAVaPS in BN structural leaning was performed. Also, AGAVaPS was compared to HC and TABU for six literature datasets considering F1 score, structural Hamming distance (SHD), balanced scoring function (BSF), Bayesian information criterion (BIC), and execution time. HC and TABU performed basically the same for all the tests made. AGAVaPS performed better than the other algorithms for F1 score, SHD, and BIC, showing that it can perform well and is a good choice for BN structural learning.

Multi-source multi-modal markers for Bayesian Networks: Application to the extremely preterm born brain

The preterm phenotype results from the interplay of multiple disorders affecting the brain and cognitive outcomes. Accurately characterising these interactions can reveal prematurity markers. Bayesian Networks (BNs) are powerful tools to disentangle these relationships, as they inherently measure associations between variables while mitigating confounding factors. We present Modified PC-HC (MPC-HC), a Bayesian Network (BN) structural learning algorithm. MPC-HC employs statistical testing and search-and-score techniques to explore equivalent classes. We employ MPC-HC to estimate BNs for extremely preterm (EP) young adults and full-term controls. Using MRI measurements and cognitive performance markers, we investigate predictive relationships and mutual influences through predictions and sensitivity analysis. We assess the confidence in the estimated BN structures using bootstrapping. Furthermore, MPC-HC’s validation involves assessing its ability to recover benchmark BN structures. MPC-HC achieves an average prediction accuracy of 72.5% compared to 62.5% of PC, 64.5% of MMHC, and 71.5% of HC, while it outperforms PC, MMHC, and HC algorithms in reconstructing the true structure of benchmark BNs. The sensitivity analysis shows that MRI measurements mainly affect EP cognitive scores. Our work has two key contributions: first, the introduction and validation of a new BN structure learning method. Second, demonstrating the potential of BNs in modelling variable relationships, predicting variables of interest, modelling uncertainty, and evaluating how variables impact each other. Finally, we demonstrate this by characterising complex phenotypes, such as preterm birth, and discovering results consistent with literature findings.

Bayesian Network structure learning algorithm for highly missing and non imputable data: Application to breast cancer radiotherapy data

It is not uncommon for real-life data produced in healthcare to have a higher proportion of missing data than in other scopes. To take into account these missing data, imputation is not always desired by healthcare experts, and complete case analysis can lead to a significant loss of data. The algorithm proposed here, allows the learning of Bayesian Network graphs when both imputation and complete case analysis are not possible. The learning process is based on a set of local bootstrap learnings performed on complete sub-datasets which are then aggregated and locally optimized. This learning method presents competitive results compared to other structure learning algorithms, whatever the mechanism of missing data.

Feature Selection for Efficient Local-to-global Bayesian Network Structure Learning

Local-to-global learning approach plays an essential role in Bayesian network (BN) structure learning. Existing local-to-global learning algorithms first construct the skeleton of a DAG (directed acyclic graph) by learning the MB (Markov blanket) or PC (parents and children) of each variable in a dataset, then orient edges in the skeleton. However, existing MB or PC learning methods are often computationally expensive especially with a large-sized BN, resulting in inefficient local-to-global learning algorithms. To tackle the problem, in this article, we link feature selection with local BN structure learning and develop an efficient local-to-global learning approach using filtering feature selection. Specifically, we first analyze the rationale of the well-known Minimum-Redundancy and Maximum-Relevance (MRMR) feature selection approach for learning a PC set of a variable. Based on the analysis, we propose an efficient F2SL (feature selection-based structure learning) approach to local-to-global BN structure learning. The F2SL approach first employs the MRMR approach to learn the skeleton of a DAG, then orients edges in the skeleton. Employing independence tests or score functions for orienting edges, we instantiate the F2SL approach into two new algorithms, F2SL-c (using independence tests) and F2SL-s (using score functions). Compared to the state-of-the-art local-to-global BN learning algorithms, the experiments validated that the proposed algorithms in this article are more efficient and provide competitive structure learning quality than the compared algorithms.

Innovative Path of Civic Education Management Mode in Colleges and Universities Based on Bayesian Networks

Abstract This paper constructs a Bayesian network structure learning model using both the Bayesian network and the Notears algorithm. Combined with acyclic graph and conditional probability table, the structure and parameters of BN are described respectively. Focusing on the basic methods of Bayesian network structure, the hybrid algorithm based on Notears is chosen as the basic method of structure learning by analyzing the application of the evaluation search method, constraint method and hybrid search algorithm. The results show that the ideal education management method should focus on the integration with real life, so much so that it has a low rate of detachment from reality as well as a low rate of theory, accounting for 0.35 and 0.4, respectively. The traditional way of management of ideology and political education in colleges and universities is more theoretical and detached from the actual situation of education and management, with theoretical management accounting for 0.75, and management is inefficient, with a percentage of 0.6. The research in this paper promotes, to a certain extent, the innovation of the management of the ideological education in colleges and universities, for the management of ideological education in colleges and universities.

Advocating the Use of Bayesian Network in Analyzing the Modes of Occurrence of Elements in Coal.

Modes of occurrence of elements in coal are important because they can be used not only to understand the origin of inorganic components in coal but also to determine the impact on the environment and human health and the deposition process of coal seams as well. Statistical analysis is one of the commonly used indirect methods used to analyze the modes of occurrence of elements in coal, among which hierarchical clustering is widely used. However, hierarchical clustering may lead to misleading results due to its limitation that it focuses on the clusters of elements rather than a single element. To tackle this issue, we use the first part of a well-known Bayesian network structure learning algorithm, i.e., Peter-Clark (PC) algorithm, to explore the relationships of the coal elemental data and then infer modes of occurrence of elements in coal. A data set containing 95 Late Paleozoic coal samples from the Datanhao and Adaohai mines in Inner Mongolia, China, is used for the performance evaluation. Analytical results show that many instructive and surprising insights can be concluded from the first part of the PC algorithm. Compared with the hierarchical clustering algorithm, the first part of the PC algorithm demonstrates superiority in analyzing the modes of occurrence of elements in coal.

Bayesian network based probabilistic weighted high-order fuzzy time series forecasting

The present article proposes a probabilistic weighted high-order fuzzy time series (FTS) forecasting model employing Bayesian network (BN) to address complex relationships and uncertainty hidden in time series. As considerable FTS forecasting models considers the fuzzy relationships between precedent moments and the consequent moment as the complex relationships in time series, BN structure learning is utilized to discover and model dependence relationships between each moment in time series. The combination of the fuzzy relationships modeled by fuzzy logical relationships and the dependence relationships modeled by the BN provides a comprehensive establishment and representation of the complex relationships inherent in time series data. The proposed FTS forecasting method calculates the fuzzy-probabilistic weights of each fuzzy logical relationship group using the improved fuzzy empirical probabilities to model both aleatoric and epistemic uncertainty in time series. To this end, the improved fuzzy empirical probabilities are formulated by integrating fuzzy empirical probabilities with the BN to incorporate dependence relationships from the original time series into the FTS forecasting procedure. The efficiency of the proposed forecasting model is validated on fourteen publicly available time series. Experimental results confirm the better performance of the proposed method comparing with nine existing FTS models and six numeric models. Hypothesis tests also validate the robustness and reliability of the proposed method.

Exploring factors related to heart attack complicated with hypertension using a Bayesian network model: a study based on the China Health and Retirement Longitudinal Study.

While Bayesian networks (BNs) represents a good approach to discussing factors related to many diseases, little attention has been poured into heart attack combined with hypertension (HAH) using BNs. This study aimed to explore the complex network relationships between HAH and its related factors, and to achieve the Bayesian reasoning for HAH, thereby, offering a scientific reference for the prevention and treatment of HAH. The data was downloaded from the Online Open Database of CHARLS 2018, a population-based longitudinal survey. In this study, we included 16 variables from data on demographic background, health status and functioning, and lifestyle. First, Elastic Net was first used to make a feature selection for highly-related variables for HAH, which were then included into BN model construction. The structural learning of BNs was achieved using Tabu algorithm and the parameter learning was conducted using maximum likelihood estimation. Among 19,752 individuals (9,313 men and 10,439 women) aged 64.73 ± 10.32 years, Among 19,752 individuals (9,313 men and 10,439 women), there are 8,370 ones without HAH (42.4%) and 11,382 ones with HAH (57.6%). What's more, after feature selection using Elastic Net, Physical activity, Residence, Internet access, Asset, Marital status, Sleep duration, Social activity, Educational levels, Alcohol consumption, Nap, BADL, IADL, Self report on health, and age were included into BN model establishment. BNs were constructed with 15 nodes and 25 directed edges. The results showed that age, sleep duration, physical activity and self-report on health are directly associated with HAH. Besides, educational levels and IADL could indirectly connect to HAH through physical activity; IADL and BADL could indirectly connect to HAH through Self report on health. BNs could graphically reveal the complex network relationship between HAH and its related factors. Besides, BNs allows for risk reasoning for HAH through Bayesian reasoning, which is more consistent with clinical practice and thus holds some application prospects.

Sequential Modeling and Knowledge Source Integration for Identifying the Structure of a Bayesian Network for Multistage Process Monitoring and Diagnosis

Abstract As a popular applied artificial intelligence tool, Bayesian networks are increasingly being used to model multistage manufacturing processes for fault diagnosis purposes. However, the major issue limiting the practical adoption of Bayesian networks is the difficulty of learning the network structure for large multistage processes. Traditionally, Bayesian network structures are learned either with the help of domain experts or by utilizing data-driven structure learning algorithms through trial and error. Both approaches have their limitations. On the one hand, the expert-driven approach is costly, time-consuming, cumbersome for large networks, and susceptible to errors in assessing probabilities; on the other hand, data-driven approaches suffer from noise, biases, and inadequacy of training data and often fail to capture the physical causal structure of the data. Therefore, in this article, we propose a Bayesian network structure learning approach where popular manufacturing knowledge sources like the failure mode and effect analysis (FMEA) and hierarchical variable ordering are used as structural priors to guide the data-driven structure learning process. In addition, to introduce modularity and flexibility into the learning process, we present a sequential modeling approach for structure learning so that large multistage networks can be learned stage by stage progressively. Furthermore, through simulation studies, we compare and analyze the performance of the knowledge source–based structurally biased networks in the context of multistage process fault diagnosis.

PSO-K2PC: Bayesian structure learning using optimized K2 algorithm for parents-children detection

Bayesian networks, revered for their adeptness in modeling uncertainty and predicting outcomes, encounter a formidable hurdle during the structure learning phase – an NP-hard problem, posing insurmountable computational challenges for large networks. To surmount this barrier and advance the field, we propose an innovative optimization of the K2PC algorithm for Bayesian network structure learning. Derived from the popular K2 algorithm, our novel optimization ingeniously tackles K2PC's vulnerability to predetermined node order. Leveraging the power of a particle swarm optimization algorithm, we adeptly seek the optimal node ordering, yielding exceptional results. Through rigorous evaluations on benchmark networks, our proposed method surpasses prior approaches in structure difference and accuracy, affirming its potential as a promising avenue for Bayesian network structure learning in large, complex networks. We posit that our novel approach constitutes an important advance in the field of Bayesian network structure learning, with the potential to stimulate additional progress through further scientific investigation.

Estimating Treatment Effect of Adjuvant Chemotherapy in Elderly Patients With Stage III Colon Cancer Using Bayesian Networks.

While adjuvant therapy with capecitabine and oxaliplatin (CAPOX) has been proven to be effective in stage III colon cancer, capecitabine monotherapy (CapMono) might be equally effective in elderly patients. Unfortunately, the elderly are under-represented in clinical trials and patients included may not be representative of the routine care population. Observational data might alleviate this problem but is sensitive to biases such as confounding by indication. Here, we build causal models using Bayesian Networks (BNs), identify confounders, and estimate the effect of adjuvant chemotherapy using survival analyses. Patients 70 years and older were selected from the Netherlands Cancer Registry (N = 982). We developed several BNs using constraint-based, score-based, and hybrid algorithms while precluding noncausal relations. In addition, we created models using a limited set of recurrence and survival nodes. Potential confounders were identified through the resulting graphs. Several Cox models were fitted correcting for confounders and for propensity scores. When comparing adjuvant treatment with surgery only, pathological lymph node classification, physical status, and age were identified as potential confounders. Adjuvant treatment was significantly associated with survival in all Cox models, with hazard ratios between 0.39 and 0.45; CIs overlapped. BNs investigating CAPOX versus CapMono did not find any association between the treatment choice and survival and thus no confounders. Analyses using Cox models did not identify significant association either. We were able to successfully leverage BN structure learning algorithms in conjunction with clinical knowledge to create causal models. While confounders differed depending on the algorithm and included nodes, results were not contradictory. We found a strong effect of adjuvant therapy on survival in our cohort. Additional oxaliplatin did not have a marked effect and should be avoided in elderly patients.

An improved Harris Hawks optimization for Bayesian network structure learning via genetic operators

An improved Harris Hawks optimization for Bayesian network structure learning via genetic operators

Bi-objective evolutionary Bayesian network structure learning via skeleton constraint

Bi-objective evolutionary Bayesian network structure learning via skeleton constraint

Bayesian Network Structure Learning Using Improved A* with Constraints from Potential Optimal Parent Sets

Learning the structure of a Bayesian network and considering the efficiency and accuracy of learning has always been a hot topic for researchers. This paper proposes two constraints to solve the problem that the A* algorithm, an exact learning algorithm, is not efficient enough to search larger networks. On the one hand, the parent–child set constraints reduce the number of potential optimal parent sets. On the other hand, the path constraints are obtained from the potential optimal parent sets to constrain the search process of the A* algorithm. Both constraints are proposed based on the potential optimal parent sets. Experiments show that the time efficiency of the A* algorithm can be significantly improved, and the ability of the A* algorithm to search larger Bayesian networks can be improved by the two constraints. In addition, compared with the globally optimal Bayesian network learning using integer linear programming (GOBNILP) algorithm and the max–min hill-climbing (MMHC) algorithm, which are state of the art, the A* algorithm enhanced by constraints still performs well in most cases.

Bayesian network model structure based on binary evolutionary algorithm.

With the continuous development of new technologies, the scale of training data is also expanding. Machine learning algorithms are gradually beginning to be studied and applied in places where the scale of data is relatively large. Because the current structure of learning algorithms only focus on the identification of dependencies and ignores the direction of dependencies, it causes multiple labeled samples not to identify categories. Multiple labels need to be classified using techniques such as machine learning and then applied to solve the problem. In the environment of more training data, it is very meaningful to explore the structure extension to identify the dependencies between attributes and take into account the direction of dependencies. In this article, Bayesian network structure learning, analysis of the shortcomings of traditional algorithms, and binary evolutionary algorithm are applied to the randomized algorithm to generate the initial population. In the optimization process of the algorithm, it uses a Bayesian network to do a local search and uses a depth-first algorithm to break the loop. Finally, it finds a higher score for the network structure. In the simulation experiment, the classic data sets, ALARM and INSURANCE, are introduced to verify the effectiveness of the algorithm. Compared with NOTEARS and the Expectation-Maximization (EM) algorithm, the weight evaluation index of this article was 4.5% and 7.3% better than other schemes. The clustering effect was improved by 13.5% and 15.2%. The smallest error and the highest accuracy are also better than other schemes. The discussion of Bayesian reasoning in this article has very important theoretical and practical significance. This article further improves the Bayesian network structure and optimizes the performance of the classifier, which plays a very important role in promoting the expansion of the network structure and provides innovative thinking.

An improved hybrid structure learning strategy for Bayesian networks based on ensemble learning

In the application of Bayesian networks to solve practical problems, it is likely to encounter the situation that the data set is expensive and difficult to obtain in large quantities and the small data set is easy to cause the inaccuracy of Bayesian network (BN) scoring functions, which affects the BN optimization results. Therefore, how to better learn Bayesian network structures under a small data set is an important problem we need to pay attention to and solve. This paper introduces the idea of parallel ensemble learning and proposes a new hybrid Bayesian network structure learning algorithm. The algorithm adopts the elite-based structure learner using genetic algorithm (ESL-GA) as the base learner. Firstly, the adjacency matrices of the network structures learned by ESL-GA are weighted and averaged. Then, according to the preset threshold, the edges between variables with weak dependence are filtered to obtain a fusion matrix. Finally, the fusion matrix is modified as the adjacency matrix of the integrated Bayesian network so as to obtain the final Bayesian network structure. Comparative experiments on the standard Bayesian network data sets show that the accuracy and reliability of the proposed algorithm are significantly better than other algorithms.

The dual PC algorithm and the role of Gaussianity for structure learning of Bayesian networks

Learning the graphical structure of Bayesian networks is key to describing data-generating mechanisms in many complex applications but poses considerable computational challenges. Observational data can only identify the equivalence class of the directed acyclic graph underlying a Bayesian network model, and a variety of methods exist to tackle the problem. Under certain assumptions, the popular PC algorithm can consistently recover the correct equivalence class by reverse-engineering the conditional independence (CI) relationships holding in the variable distribution. The dual PC algorithm is a novel scheme to carry out the CI tests within the PC algorithm by leveraging the inverse relationship between covariance and precision matrices. By exploiting block matrix inversions we can also perform tests on partial correlations of complementary (or dual) conditioning sets. The multiple CI tests of the dual PC algorithm proceed by first considering marginal and full-order CI relationships and progressively moving to central-order ones. Simulation studies show that the dual PC algorithm outperforms the classic PC algorithm both in terms of run time and in recovering the underlying network structure, even in the presence of deviations from Gaussianity. Additionally, we show that the dual PC algorithm applies for Gaussian copula models, and demonstrate its performance in that setting.