Markov Blanket Discovery Research Articles

A wrapper feature selection approach using Markov blankets

In feature selection, Markov Blanket (MB) based approaches have attracted considerable attention with most MB discovery algorithms being categorized as filter based techniques. Typically, the Conditional Independence (CI) test employed by such methods is different for different data types. In this article, we propose a novel Markov Blanket based wrapper feature selection method. The proposed approach employs Predictive Permutation Independence (PPI), a novel Conditional Independence (CI) test that allows it to work out-of-the-box for both classification and regression tasks on mixed data. PPI can work with any supervised algorithm to estimate the association of a feature with the target variable while also providing a measure of feature importance. The proposed approach also includes an optional MB aggregation step that can be used to find the optimal MB under non-faithful conditions. Our method11Implementation and experimental results are available at https://github.com/AnonymousMLSubmissions/PatternRecognitionSubmission. outperforms other MB discovery methods, in terms of F1-score, by 7% on average, over 3 large-scale BN datasets. It also outperforms state-of-the-art feature selection techniques on 13 real-world datasets.

Feature selection and interpretability analysis of compound faults in rolling bearings based on the causal feature weighted network

Feature selection is a crucial step in fault diagnosis. When rolling bearings are susceptible to compound faults, causal relationships are hidden within the signal features. Complex network analysis methods provide a tool for causal relationship modeling and feature importance assessment. Existing studies mainly focus on unweighted networks, overlooking the impact of the strength of causal relationships on feature selection. To address this issue, we propose a compound fault feature selection method based on the causal feature weighted network. First, we construct a weighted network using the incremental association Markov blanket discovery algorithm and Pearson correlation coefficient. Then, we quantify the importance of features by treating node strength as a centrality index and rank them to partition the feature subset. Finally, the optimal feature subset is obtained through a neural network with the accuracy of compound fault diagnosis as the threshold. Analysis of public datasets and comparative experiments demonstrate the advantages of our method. Compared to existing research, our method not only effectively reduces the number of optimal feature subsets to 11 but also improves the accuracy of compound fault diagnosis to 95.2%. Furthermore, we employ the SHapley Additive exPlanations to interpret the contribution of each feature in the optimal subset to the accuracy of compound fault diagnosis. This provides reference from both physical and network perspectives to feature selection and compound fault diagnosis in rolling bearings in practical working conditions.

A Constrained Local Neighborhood Approach for Efficient Markov Blanket Discovery in Undirected Independent Graphs

A Constrained Local Neighborhood Approach for Efficient Markov Blanket Discovery in Undirected Independent Graphs

A Two-State-Based Hybrid Model for Degradation and Capacity Prediction of Lithium-Ion Batteries with Capacity Recovery

The accurate prediction of Li-ion battery capacity is important because it ensures mission and personnel safety during operations. However, the phenomenon of capacity recovery (CR) may impede the progress of improving battery capacity prediction performance. Therefore, in this study, we focus on the phenomenon of capacity recovery during battery degradation and propose a hybrid lithium-ion battery capacity prediction framework based on two states. First, to improve the density of capacity-related information, the simultaneous Markov blanket discovery algorithm (STMB) is used to screen the causal features of capacity from the initial feature set. Then, the life-long cycle sequence of batteries is partitioned into global degradation regions and recovery regions, as part of the proposed prediction framework. The prediction branch for the global degradation region is implemented through a long short-term memory network (LSTM) and the other prediction branch for the recovery region is implemented through Gaussian process regression (GPR). A support vector machine (SVM) model is applied to identify recovery points to switch the branch of the prediction framework. The prediction results are integrated to obtain the final prediction results. Experimental studies based on NASA’s lithium battery aging data highlight the trustworthy capacity prediction ability of the proposed method considering the capacity recovery phenomenon. In contrast to the comparative methods, the mean absolute error and the root mean square error are reduced by up to 0.0013 Ah and 0.0043 Ah, which confirms the validity of the proposed method.

Loose-to-strict Markov blanket learning algorithm for feature selection

The Markov blanket (MB) represents a crucial concept in a Bayesian network (BN) and is theoretically the optimal solution to the feature selection problem. Methods based on conditional independence (CI) tests are prevalent for MB discovery. Currently, the main challenge is how to improve both the efficiency and effectiveness of this type of method. In this paper, we propose a novel divide-and-conquer discovery algorithm, loose-to-strict MB (LSMB), to discover MBs faster while maintaining high accuracy. LSMB first discovers the approximate parent–child (PC) and spouse sets of a target variable via the loose CI test strategy, a constraint for the condition set of CI tests. Then, by strict CI tests, i.e., without constraint for the size of the condition set, LSMB first removes non-MB nodes in the discovered approximate PC set and then categorizes and removes non-MB nodes in the discovered approximate spouse set. Finally, LSMB combines the discovered sets to obtain the MB of the target. Experiments on benchmark BN datasets show that LSMB can improve the efficiency of MB discovery while maintaining higher accuracy than the state-of-the-art MB discovery algorithms, and experiments on real-world datasets demonstrate the excellent performance of LSMB in feature selection.

Composite Fault Diagnosis of Rolling Bearings: A Feature Selection Approach Based on the Causal Feature Network

A rolling bearing is a complex system consisting of the inner race, outer race, rolling element, etc. The interaction of components may lead to composite faults. Selecting the features that can accurately identify the fault type from the composite fault features with causality among components is key to composite fault diagnosis. To tackle this issue, we propose a feature selection approach for composite fault diagnosis based on the causal feature network. Based on the incremental association Markov blanket discovery, we first use the algorithm to mine the causal relationships between composite fault features and construct the causal feature network. Then, we draw upon the nodes’ centrality indicators in the complex network to quantify the importance of composite fault features. We also propose the criteria for threshold selection to determine the number of features in the optimal feature subset. Experimental results on the standard dataset for composite fault diagnosis show that our approach of using the causal relationship between features and the nodes’ centrality indicators of complex network can effectively identify the key features in composite fault signals and improve the accuracy of composite fault diagnosis. Experimental results thus verify our approach’s effectiveness.

Critique of “A Parallel Framework for Constraint-Based Bayesian Network Learning via Markov Blanket Discovery” by SCC Team From Tsinghua University

Srivastava et al. propose a parallel framework to optimize Bayesian network learning in the SC20 paper entitled “A Parallel Framework for Constraint-Based Bayesian Network Learning via Markov Blanket Discovery”. They parallelize all the phases in network constructing algorithms to achieve high performance and scalability. In this paper, we reproduce the strong scaling and weak scaling experiments in that SC paper. We conduct experiments on a 4-node cluster with Intel CPUs provided by the SCC committee. We further analyze the results of communication overhead. Our results show that the proposed method in that SC paper scales well on the provided cluster, in accordance with the SC paper.

Critique of “A Parallel Framework for Constraint-Based Bayesian Network Learning via Markov Blanket Discovery” by SCC Team From Peking University

Ankit Srivastava et al. [1] proposed a parallel framework for Constraint-Based Bayesian Network (BN) Learning via Markov Blanket Discovery (referred to as ramBLe) and implemented it over three existing BN learning algorithms, namely, GS, IAMB and Inter-IAMB. As part of the Student Cluster Competition at SC21, we reproduce the computational efficiency of ramBLe on our assigned Oracle cluster. The cluster has 4x36 cores in total with 100 Gbps RoCE v2 support and is equipped with Centos-compatible Oracle Linux. Our experiments, covering the same three algorithms of ramBLe, evaluate its strong and weak scalability of the algorithms using real COVID-19 data sets. We verify part of the conclusions in the paper and propose our explanation of the differences.

Critique of: “A Parallel Framework for Constraint-Based Bayesian Network Learning via Markov Blanket Discovery” by SCC Team From UC San Diego

Bayesian networks (BNs) have become popular in recent years to describe natural phenomena in situations where causal linkages are important to understand. In order to get around the inherent non-tractability of learning BNs, Srivastava et al. propose a markov blanket discovery-based approach to learning in their paper titled <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">”A Parallel Framework for Constraint-based Bayesian Network Learning via Markov Blanket Discovery.”</i> We are able to reproduce both the strong and weak scaling experiments from the paper up to 128 cores, and verify communication cost scaling for all three algorithms in the paper. We also introduce methodological improvements to weak scaling that show the paper's findings are unique to the methodology and not the datasets used. Slight variations in performance were observed due to differences in datasets, core count, and job scheduling.

A Parallel Framework for Constraint-Based Bayesian Network Learning via Markov Blanket Discovery

Bayesian networks (BNs) are a widely used graphical model in machine learning. As learning the structure of BNs is NP-hard, high-performance computing methods are necessary for constructing large-scale networks. In this paper, we present a parallel framework to scale BN structure learning algorithms to tens of thousands of variables. Our framework is applicable to learning algorithms that rely on the discovery of Markov blankets (MBs) as an intermediate step. We demonstrate the applicability of our framework by parallelizing three different algorithms: <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Grow-Shrink</i> ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">GS</i> ), <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Incremental Association MB</i> ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">IAMB</i> ), and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Interleaved IAMB</i> ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Inter-IAMB</i> ). Our implementations are available as part of an open-source software called <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ramBLe</i> , and are able to construct BNs from real data sets with tens of thousands of variables and thousands of observations in less than a minute on 1024 cores, with a speedup of up to 845X and 82.5% efficiency. Furthermore, we demonstrate using simulated data sets that our proposed parallel framework can scale to BNs of even higher dimensionality. Our implementations were selected for the reproducibility challenge component of the 2021 student cluster competition (SCC'21), which tasked undergraduate teams from around the world with reproducing the results that we obtained using the implementations. We discuss details of the challenge and the results of the experiments conducted by the top teams in the competition. The results of these experiments indicate that our key results are reproducible, despite the use of completely different data sets and experiment infrastructure, and validate the scalability of our implementations.

Critique of “A Parallel Framework for Constraint-Based Bayesian Network Learning via Markov Blanket Discovery” by SCC Team From ShanghaiTech University

In SC20, Srivastava <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">et al.</i> [6] proposed a Parallel F <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ram</b> ework for <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">B</b> ayesian <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Le</b> arning, or ramBLe, for short, which is a highly parallel and efficient framework for learning the structure of Bayesian Networks (BNs) from samples, particularly large genome-scale networks. As part of our participation in the SC21 Student Cluster Competition, our task was to verify conclusions from the original work [6]. Here we present the outcome of our experiments, which were performed on a four-node cluster from the Oracle Cloud HPC platform. We reproduce the numerical results from [6], namely the algorithm's performance and scaling behavior using MPI and different Python and Boost libraries on the Oracle cloud.

Online Causal Feature Selection for Streaming Features

Recently, causal feature selection (CFS) has attracted considerable attention due to its outstanding interpretability and predictability performance. Such a method primarily includes the Markov blanket (MB) discovery and feature selection based on Granger causality. Representatively, the max-min MB (MMMB) can mine an optimal feature subset, i.e., MB; however, it is unsuitable for streaming features. Online streaming feature selection (OSFS) via online process streaming features can determine parents and children (PC), a subset of MB; however, it cannot mine the MB of the target attribute ( T ), i.e., a given feature, thus resulting in insufficient prediction accuracy. The Granger selection method (GSM) establishes a causal matrix of all features by performing excessively time; however, it cannot achieve a high prediction accuracy and only forecasts fixed multivariate time series data. To address these issues, we proposed an online CFS for streaming features (OCFSSFs) that mine MB containing PC and spouse and adopt the interleaving PC and spouse learning method. Furthermore, it distinguishes between PC and spouse in real time and can identify children with parents online when identifying spouses. We experimentally evaluated the proposed algorithm on synthetic datasets using precision, recall, and distance. In addition, the algorithm was tested on real-world and time series datasets using classification precision, the number of selected features, and running time. The results validated the effectiveness of the proposed algorithm.

Decomposition-Based Bayesian Network Structure Learning Algorithm for Abnormity Diagnosis Model for Coal Mill Process

In the structure learning of the large-scale Bayesian network (BN) model for the coal mill process, taking the view of the problem that the decomposition-based method cannot guarantee the sufficient learning of abnormal state node neighborhood in the diagnosis model, this paper proposes a new BN structure learning method based on decomposition. Firstly, a sketch is constructed based on an improved Markov blanket discovery algorithm and edge thickening and thinning. Second, the node centrality of k-path is used to search the important nodes, and the subgraph decomposition is realized by extracting these important nodes and their neighborhoods from the sketch. Then, through the targeted design of subgraph de-duplication, subgraph learning, and subgraph reorganization methods, the learning of large-scale BN is realized. This method is applied to public data sets, and its advantages and disadvantages are analyzed by comparing them with other methods. The advantage of the BN structure learning method of the abnormal condition diagnosis model is further verified by applying the method to the coal mill process, which is consistent with the original design intention.

Bayesian Coherence Analysis for Microcircuit Structure Learning

Functional microcircuits model the coordinated activity of neurons and play an important role in physiological computation and behaviors. Most existing methods to learn microcircuit structures are correlation-based and often generate dense microcircuits that cannot distinguish between direct and indirect association. We treat microcircuit structure learning as a Markov blanket discovery problem and propose Bayesian Coherence Analysis (BCA) which utilizes a Bayesian network architecture called Bayesian network with inverse-tree structure to efficiently and effectively detect Markov blankets for high-dimensional neural activity data. BCA achieved balanced sensitivity and specificity on simulated data. For the real-world anterior lateral motor cortex study, BCA identified microcircuit subtypes that predicted trial types with an accuracy of 0.92. BCA is a powerful method for microcircuit structure learning.

Accelerating Causal Inference and Feature Selection Methods through G-Test Computation Reuse.

This article presents a novel and remarkably efficient method of computing the statistical G-test made possible by exploiting a connection with the fundamental elements of information theory: by writing the G statistic as a sum of joint entropy terms, its computation is decomposed into easily reusable partial results with no change in the resulting value. This method greatly improves the efficiency of applications that perform a series of G-tests on permutations of the same features, such as feature selection and causal inference applications because this decomposition allows for an intensive reuse of these partial results. The efficiency of this method is demonstrated by implementing it as part of an experiment involving IPC–MB, an efficient Markov blanket discovery algorithm, applicable both as a feature selection algorithm and as a causal inference method. The results show outstanding efficiency gains for IPC–MB when the G-test is computed with the proposed method, compared to the unoptimized G-test, but also when compared to IPC–MB++, a variant of IPC–MB which is enhanced with an AD–tree, both static and dynamic. Even if this proposed method of computing the G-test is presented here in the context of IPC–MB, it is in fact bound neither to IPC–MB in particular, nor to feature selection or causal inference applications in general, because this method targets the information-theoretic concept that underlies the G-test, namely conditional mutual information. This aspect grants it wide applicability in data sciences.

Causal Discovery of Flight Service Process Based on Event Sequence

The development of the civil aviation industry has continuously increased the requirements for the efficiency of airport ground support services. In the existing ground support research, there has not yet been a process model that directly obtains support from the ground support log to study the causal relationship between service nodes and flight delays. Most ground support studies mainly use machine learning methods to predict flight delays, and the flight support model they are based on is an ideal model. The study did not conduct an in-depth study of the causal mechanism behind the ground support link and did not reveal the true cause of flight delays. Therefore, there is a certain deviation in the prediction of flight delays by machine learning, and there is a certain deviation between the ideal model based on the research and the actual service process. Therefore, it is of practical significance to obtain the process model from the guarantee log and analyze its causality. However, the existing process causal factor discovery methods only do certain research when the assumption of causal sufficiency is established and does not consider the existence of latent variables. Therefore, this article proposes a framework to realize the discovery of process causal factors without assuming causal sufficiency. The optimized fuzzy mining process model is used as the service benchmark model, and the local causal discovery algorithm is used to discover the causal factors. Under this framework, this paper proposes a new Markov blanket discovery algorithm that does not assume causal sufficiency to discover causal factors and uses benchmark data sets for testing. Finally, the actual flight service data are used for causal discovery among flight service nodes. The local causal discovery algorithm proposed in this paper has a certain competitive advantage in accuracy, F1, and other aspects of the existing causal discovery algorithm. It avoids the occurrence of its dimensional disaster. Through the in-depth analysis of the flight safety reason node discovered by this method, it is found that the unreasonable scheduling of flight support personnel is an important reason for frequent flight delays at the airport.

Inferring Gene Regulatory Networks Using the Improved Markov Blanket Discovery Algorithm.

Inferring gene regulatory networks (GRNs) from microarray data can help us understand the mechanisms of life and eventually develop effective therapies. Currently, many computational methods have been used in inferring GRNs. However, owing to high-dimensional data and small samples, these methods often tend to introduce redundant regulatory relationships. Therefore, a novel network inference method based on the improved Markov blanket discovery algorithm, IMBDANET, is proposed to infer GRNs. Specifically, for each target gene, data processing inequality was applied to the Markov blanket discovery algorithm for the accurate differentiation of direct regulatory genes from indirect regulatory genes. Finally, direct regulatory genes were used in constructing GRNs, and the network structure was optimized according to the importance degree score. Experimental results on six public network datasets show that the proposed method can be effectively used to infer GRNs.

Towards Efficient Local Causal Structure Learning

Local causal structure learning aims to discover and distinguish direct causes (parents) and direct effects (children) of a variable of interest from data. While emerging successes have been made, existing methods need to search a large space to distinguish direct causes from direct effects of a target variable T. To tackle this issue, we propose a novel Efficient Local Causal Structure learning algorithm, named ELCS. Specifically, we first propose the concept of N-structures, then design an efficient Markov Blanket (MB) discovery subroutine to integrate MB learning with N-structures to learn the MB of T and simultaneously distinguish direct causes from direct effects of T. With the proposed MB subroutine, ELCS starts from the target variable, sequentially finds MBs of variables connected to the target variable and simultaneously constructs local causal structures over MBs until the direct causes and direct effects of the target variable have been distinguished. Using eight Bayesian networks the extensive experiments have validated that ELCS achieves better accuracy and efficiency than the state-of-the-art algorithms.

RWRNET: A Gene Regulatory Network Inference Algorithm Using Random Walk With Restart.

Inferring gene regulatory networks from expression data is essential in identifying complex regulatory relationships among genes and revealing the mechanism of certain diseases. Various computation methods have been developed for inferring gene regulatory networks. However, these methods focus on the local topology of the network rather than on the global topology. From network optimisation standpoint, emphasising the global topology of the network also reduces redundant regulatory relationships. In this study, we propose a novel network inference algorithm using Random Walk with Restart (RWRNET) that combines local and global topology relationships. The method first captures the local topology through three elements of random walk and then combines the local topology with the global topology by Random Walk with Restart. The Markov Blanket discovery algorithm is then used to deal with isolated genes. The proposed method is compared with several state-of-the-art methods on the basis of six benchmark datasets. Experimental results demonstrated the effectiveness of the proposed method.

Decomposition-based Bayesian network structure learning algorithm using local topology information

Hybrid learning algorithms which integrate the merits of the constraint-based methods and the search-and-score methods are used to cope with Bayesian network (BN) structure estimation problem. However, such simple and crude synthesis techniques always consider the global topology information during the learning process and attempt to directly search for the optimal network structure in the enormous solution space for large-scale BNs, resulting in prohibitive computational cost as well as low learning accuracy. Therefore, we propose a novel hybrid structure learning algorithm based on the idea of model decomposition, which takes into account the knowledge of local neighborhood structures. The proposed method works in four stages. We first draft an undirected independence graph by using an efficient Markov blanket discovery approach, and then split the entire network into a series of subgraphs. After learning the small BNs from the observed data, the resultant topology can be obtained by combining these small BNs. Experiments on different benchmark BNs and the varying data sets demonstrate that the proposed algorithm generally gains the better performance of structure recovery than other representative methods, especially for large-scale BNs.