Causal Structure Learning Algorithms Research Articles

Adversarial Missingness Attacks on Causal Structure Learning

Causality-informed machine learning has been proposed as an avenue for achieving many of the goals of modern machine learning, from ensuring generalization under domain shifts to attaining fairness, robustness, and interpretability. A key component of causal machine learning is the inference of causal structures from observational data; in practice, this data may be incompletely observed. Prior work has demonstrated that adversarial perturbations of completely observed training data may be used to force the learning of inaccurate causal structural models (SCMs). However, when the data can be audited for correctness (e.g., it is cryptographically signed by its source), this adversarial mechanism is invalidated. This work introduces a novel attack methodology wherein the adversary deceptively omits a portion of the true training data to bias the learned causal structures in a desired manner (under strong signed sample input validation, this behavior seems to be the only strategy available to the adversary). Under this model, theoretically sound attack mechanisms are derived for the case of arbitrary SCMs, and a sample-efficient learning-based heuristic is given. Experimental validation of these approaches on real and synthetic data sets, across a range of SCMs from the family of additive noise models (linear Gaussian, linear non-Gaussian, and non-linear Gaussian) demonstrates the effectiveness of adversarial missingness attacks at deceiving popular causal structure learning algorithms.

Modeling the causal mechanism in process safety management (PSM) systems from historical accidents

Process safety management (PSM) underpins reducing major accidents in the process industries. For decades, PSM systems have been implemented globally. However, few studies have modeled the complexities within the PSM system, leading to some underlying chronic problems within organizations that go unnoticed. Here, we investigate causal mechanisms among PSM elements from historical accidents to deliver a practical tool for safety intervention formulation. Specifically, based on the Center for Chemical Process Safety's (CCPS) PSM framework, 100 chemical accident reports derived from the U.S. Chemical Safety and Hazard Investigation Board (CSB) were analyzed and converted into structured information stored in 22 PSM elements. Then, employing two causal structure learning algorithms, the causal structures among 22 elements were learned from the processed accident information. Finally, the causal effects among PSM elements were estimated by extending Rubin's counterfactual causal framework to an approach applicable to multi-treatment scenarios. We demonstrate that the causal structures and effects learned from accident reports reveal the causal mechanism within the PSM system. A total of 20 significant causal relationships among single elements, nine significant causal relationships among element combinations, and several interesting PSM failure paths were identified. According to these causal patterns, the PSM effort will shift from the traditional "what to do" to "how to do it."

Delving into Causal Discovery in Health-Related Quality of Life Questionnaires

Questionnaires on health-related quality of life (HRQoL) play a crucial role in managing patients by revealing insights into physical, psychological, lifestyle, and social factors affecting well-being. A methodological aspect that has not been adequately explored yet, and is of considerable potential, is causal discovery. This study explored causal discovery techniques within HRQoL, assessed various considerations for reliable estimation, and proposed means for interpreting outcomes. Five causal structure learning algorithms were employed to examine different aspects in structure estimation based on simulated data derived from HRQoL-related directed acyclic graphs. The performance of the algorithms was assessed based on various measures related to the differences between the true and estimated structures. Moreover, the Resource Description Framework was adopted to represent the responses to the HRQoL questionnaires and the detected cause–effect relationships among the questions, resulting in semantic knowledge graphs which are structured representations of interconnected information. It was found that the structure estimation was impacted negatively by the structure’s complexity and favorably by increasing the sample size. The performance of the algorithms over increasing sample size exhibited a similar pattern, with distinct differences being observed for small samples. This study illustrates the dynamics of causal discovery in HRQoL-related research, highlights aspects that should be addressed in estimation, and fosters the shareability and interoperability of the output based on globally established standards. Thus, it provides critical insights in this context, further promoting the critical role of HRQoL questionnaires in advancing patient-centered care and management.

FedCSL: A Scalable and Accurate Approach to Federated Causal Structure Learning

As an emerging research direction, federated causal structure learning (CSL) aims at learning causal relationships from decentralized data across multiple clients while preserving data privacy. Existing federated CSL algorithms suffer from scalability and accuracy issues, since they require computationally expensive CSL algorithms to be executed at each client. Furthermore, in real-world scenarios, the number of samples held by each client varies significantly, and existing methods still assign equal weights to the learned structural information from each client, which severely harms the learning accuracy of those methods. To address these two limitations, we propose FedCSL, a scalable and accurate method for federated CSL. Specifically, FedCSL consists of two novel strategies: (1) a federated local-to-global learning strategy that enables FedCSL to scale to high-dimensional data for tackling the scalability issue, and (2) a novel weighted aggregation strategy that does not rely on any complex encryption techniques while preserving data privacy for tackling the accuracy issue. Extensive experiments on benchmark datasets, high-dimensional synthetic datasets and a real-world dataset verify the efficacy of the proposed FedCSL method. The source code is available at https://github.com/Xianjie-Guo/FedCSL.

Towards Privacy-Aware Causal Structure Learning in Federated Setting

Causal structure learning has been extensively studied and widely used in machine learning and various applications. To achieve an ideal performance, existing causal structure learning algorithms often need to centralize a large amount of data from multiple data sources. However, in the privacy-preserving setting, it is impossible to centralize data from all sources and put them together as a single dataset. To preserve data privacy, federated learning as a new learning paradigm has attached much attention in machine learning in recent years. In this paper, we study a privacy-aware causal structure learning problem in the federated setting and propose a novel Federated PC (FedPC) algorithm with two new strategies for preserving data privacy without centralizing data. Specifically, we first propose a novel layer-wise aggregation strategy for a seamless adaptation of the PC algorithm into the federated learning paradigm for federated skeleton learning, then we design an effective strategy for learning consistent separation sets for federated edge orientation. The extensive experiments validate that FedPC is effective for causal structure learning in federated learning setting.

Local causal structure learning with missing data

Local causal structure learning aims to discover and distinguish the direct causes and direct effects of a target variable. However, the state-of-the-art algorithms for local causal structure learning fail to perform well when dealing with missing data. The general approach is to fill in the missing data using imputation techniques before learning the local causal structure, but this method suffers from problems such as low accuracy, low efficiency, and instability. To address these issues, we propose a novel method for local causal structure learning with missing data, named misLCS. Firstly, we design an iterative data imputation method to obtain the complete and correct data from the missing data. Then, misLCS adopts a data subset strategy to get a data subset that variables are closely related to the target variable. Thirdly, within this data subset, misLCS constructs the local causal skeleton of the target variable using a mutual information-based feature selection method and orients the direction of edges using conditional independence tests and Meek rules. Finally, misLCS updates the missing data in preparation for the next iteration. This procedure continues until the direct causes and direct effects of the target variable have been identified. Our experiments on seven benchmark Bayesian networks and a real-world bioinformatics dataset, with a number of variables from 11 to 801, demonstrate that our algorithm achieves better accuracy than the existing local causal structure learning algorithms.

On causal structural learning algorithms: Oracles’ simulations and considerations

This work evaluates the performance of several causal structure learning algorithms, in terms of their effectiveness and efficiency in detecting true causal relations among variables. Constraint-based, score-based and hybrid algorithms are jointly compared and ranked according to the two criteria above and their performance is evaluated when used in either directed or undirected acyclic graphs. Fixing the number of variables considered, a Monte Carlo simulation is run for constructing linear causal effects among variables, both in small and large data samples with different causal network properties. Latent confounding variables are empirically demonstrated to be the main drawback of an algorithms’ performance, independently of the size of the sample.

Causal learner: A toolbox for causal structure and Markov blanket learning

Causal Learner is a toolbox for learning causal structure and Markov blanket (MB) from data. It integrates functions for generating simulated Bayesian network data, a set of state-of-the-art global causal structure learning algorithms, a set of state-of-the-art local causal structure learning algorithms, a set of state-of-the-art MB learning algorithms, and functions for evaluating algorithms. The data generation part of Causal Learner is written in R, and the rest of Causal Learner is written in MATLAB. Causal Learner aims to provide researchers and practitioners with an open-source platform for causal learning from data and for the development and evaluation of new causal learning algorithms. The Causal Learner project is available at http://bigdata.ahu.edu.cn/causal-learner.

A Hybrid Causal Structure Learning Algorithm for Mixed-Type Data

Inferring the causal structure of a set of random variables is a crucial problem in many disciplines of science. Over the past two decades, various approaches have been pro- posed for causal discovery from observational data. How- ever, most of the existing methods are designed for either purely discrete or continuous data, which limit their practical usage. In this paper, we target the problem of causal structure learning from observational mixed-type data. Although there are a few methods that are able to handle mixed-type data, they suffer from restrictions, such as linear assumption and poor scalability. To overcome these weaknesses, we formulate the causal mechanisms via mixed structure equation model and prove its identifiability under mild conditions. A novel locally consistent score, named CVMIC, is proposed for causal directed acyclic graph (DAG) structure learning. Moreover, we propose an efficient conditional independence test, named MRCIT, for mixed-type data, which is used in causal skeleton learning and final pruning to further improve the computational efficiency and precision of our model. Experimental results on both synthetic and real-world data demonstrate that our proposed hybrid model outperforms the other state-of-the-art methods. Our source code is available at https://github.com/DAMO-DI-ML/AAAI2022-HCM.

Understanding unforeseen production downtimes in manufacturing processes using log data-driven causal reasoning

In discrete manufacturing, the knowledge about causal relationships makes it possible to avoid unforeseen production downtimes by identifying their root causes. Learning causal structures from real-world settings remains challenging due to high-dimensional data, a mix of discrete and continuous variables, and requirements for preprocessing log data under the causal perspective. In our work, we address these challenges proposing a process for causal reasoning based on raw machine log data from production monitoring. Within this process, we define a set of transformation rules to extract independent and identically distributed observations. Further, we incorporate a variable selection step to handle high-dimensionality and a discretization step to include continuous variables. We enrich a commonly used causal structure learning algorithm with domain-related orientation rules, which provides a basis for causal reasoning. We demonstrate the process on a real-world dataset from a globally operating precision mechanical engineering company. The dataset contains over 40 million log data entries from production monitoring of a single machine. In this context, we determine the causal structures embedded in operational processes. Further, we examine causal effects to support machine operators in avoiding unforeseen production stops, i.e., by detaining machine operators from drawing false conclusions on impacting factors of unforeseen production stops based on experience.

Mining human preference via self-correction causal structure learning

Spurred by causal structure learning (CSL) ability to reveal the cause–effect connection, significant research efforts have been made to enhance the scalability of CSL algorithms in various artificial intelligence applications. However, less effort has been made regarding the stability and the interpretability of CSL algorithms. Thus, this work proposes a self-correction mechanism that embeds domain knowledge for CSL, improving the stability and accuracy even in low-dimensional but high-noise environments by guaranteeing a meaningful output. The suggested algorithm is challenged against multiple classic and influential CSL algorithms in synthesized and field datasets. Our algorithm achieves a superior accuracy on the synthesized dataset, while on the field dataset, our method interprets the learned causal structure as a human preference for investment, coinciding with domain expert analysis.

Causal Structure Learning Algorithm Based on Partial Rank Correlation under Additive Noise Model

ABSTRACT Aiming at the structural learning problem of the additive noise model in causal discovery and the challenge of massive data processing in the era of artificial intelligence, this paper combines partial rank correlation coefficients and proposes two new Bayesian network causal structure learning algorithms: PRCB algorithm based on threshold selection and PRCS algorithm based on hypothesis testing. We mainly made three contributions. First, we proved that the partial rank correlation coefficient can be used as the standard of independence test, and explored the distribution of corresponding statistics. Second, the partial rank correlation coefficient is associated with the correlation, and a causal discovery algorithm PRCB based on partial rank correlation and an improved PRCS algorithm based on hypothesis testing are proposed. Finally, comparing with the existing technology on seven classic Bayesian networks, it proves the superiority of the algorithm in low-dimensional networks; the processing of millions of data on three high-dimensional Bayesian networks verifies the high-efficiency performance of the algorithm in high-dimensional large sample data; the application performance of the algorithm is tested by performing fault prediction on the real power plant equipment measurement point data set. Theoretical analysis and experimental results have proved the superiority of the algorithm.

Towards Efficient Local Causal Structure Learning

Local causal structure learning aims to discover and distinguish direct causes (parents) and direct effects (children) of a variable of interest from data. While emerging successes have been made, existing methods need to search a large space to distinguish direct causes from direct effects of a target variable T. To tackle this issue, we propose a novel Efficient Local Causal Structure learning algorithm, named ELCS. Specifically, we first propose the concept of N-structures, then design an efficient Markov Blanket (MB) discovery subroutine to integrate MB learning with N-structures to learn the MB of T and simultaneously distinguish direct causes from direct effects of T. With the proposed MB subroutine, ELCS starts from the target variable, sequentially finds MBs of variables connected to the target variable and simultaneously constructs local causal structures over MBs until the direct causes and direct effects of the target variable have been distinguished. Using eight Bayesian networks the extensive experiments have validated that ELCS achieves better accuracy and efficiency than the state-of-the-art algorithms.

Using Feature Selection for Local Causal Structure Learning

Local causal structure learning aims to discover and distinguish the direct causes and direct effects of a target variable. However, the state-of-the-art local causal structure learning algorithms need to perform an exhaustive subset search within the currently selected variables for PC (i.e., parents and children) discovery. In this article, we propose an efficient local causal structure learning algorithm around a target variable, called LCS-FS (Local Causal Structure learning by Feature Selection). First, to construct the local causal skeleton of the target, we employ feature selection for finding PC without searching for conditioning sets to speed up PC discovery, leading to improve the skeleton construction efficiency. Second, to orient edges in this local causal skeleton, we propose an efficient method to find separating sets from the subsets of PC for identifying V-structures. With the integration of feature selection and the new way of finding separating sets, LCS-FS recursively finds the spouses of Markov blankets in local causal skeleton for edge orientations, until the direct causes and direct effects of the target are distinguished. The experiments on five benchmark Bayesian networks with the number of variables from 35 to 801 validate that our algorithm achieves higher efficiency and better accuracy than the state-of-the-art local causal structure learning algorithms.

Distinguish Markov Equivalence Classes from Large-Scale Linear Non-Gaussian Data

In the problem of causal discovery, conditional independence (CI) tests are generally used to detect the causal relationships among observed data. Due to the curse of dimensionality and the limitation of causal direction learning based on $V$ -structure learning, it is difficult for constraint-based methods to distinguish the actual graph from a set of Markov equivalence classes. To alleviate this problem, in this work, a novel regression-based method to test CIs over linear Non-Gaussian data is proposed. The main purpose of this proposal is to relax the CI test of $x\bot y|Z$ to two unconditional independence tests $x-f\left ({Z }\right) \bot y-g\left ({Z }\right) +\varSigma H\left ({Z }\right)$ and $x-f\left ({Z }\right) +\varSigma H\left ({Z }\right) \bot y-g\left ({Z }\right)$ , where $f$ and $g$ can be estimated by linear regression independently. In addition, we further show that $x-f\left ({Z }\right) \bot y-g\left ({Z }\right) +\varSigma H\left ({Z }\right)$ (or $x-f\left ({Z }\right) +\varSigma H\left ({Z }\right) \bot y-g\left ({Z }\right)$ ) can lead to $x\gets Z$ (or $y\gets Z$ ). According to this regression-based method, we design a causal structure learning algorithm to learn the actual graph instead of a set of Markov equivalence classes over the observed data. Experiments indicate that our method can detect much more causal relationships than other existing methods, especially in large-scale cases.

An efficient causal structure learning algorithm for linear arbitrarily distributed continuous data

Aim to the linear arbitrarily distributed continuous data, an causal structure learning algorithm BSEM, which is based on simultaneous equations model, was presented. The algorithm merges together simultaneous equations model and local learning. The contribution of this paper is that for linear arbitrarily distributed datasets, BSEM algorithm can effectively learn the causal structure from the datasets. We used the Sociology data to do experiments, and results demonstrated that BSEM displays good accuracy and time performance.

Causal Discovery Combining K2 with Brain Storm Optimization Algorithm.

Exploring and detecting the causal relations among variables have shown huge practical values in recent years, with numerous opportunities for scientific discovery, and have been commonly seen as the core of data science. Among all possible causal discovery methods, causal discovery based on a constraint approach could recover the causal structures from passive observational data in general cases, and had shown extensive prospects in numerous real world applications. However, when the graph was sufficiently large, it did not work well. To alleviate this problem, an improved causal structure learning algorithm named brain storm optimization (BSO), is presented in this paper, combining K2 with brain storm optimization (K2-BSO). Here BSO is used to search optimal topological order of nodes instead of graph space. This paper assumes that dataset is generated by conforming to a causal diagram in which each variable is generated from its parent based on a causal mechanism. We designed an elaborate distance function for clustering step in BSO according to the mechanism of K2. The graph space therefore was reduced to a smaller topological order space and the order space can be further reduced by an efficient clustering method. The experimental results on various real-world datasets showed our methods outperformed the traditional search and score methods and the state-of-the-art genetic algorithm-based methods.

Streaming feature-based causal structure learning algorithm with symmetrical uncertainty

Most existing causal structure learning algorithms must have access to the entire feature set of a dataset during the learning process. However, in many real-world applications, rather than having access to an entire feature set before learning begins, features are generated in an online manner. Learning and analyzing these dynamic features online is in high demand for effective decision-making. In this paper, by modeling these dynamic features as streaming features, we propose the CSSU algorithm, a streaming feature-based casual structure learning algorithm with symmetrical uncertainty. Specifically, the CSSU algorithm performs online updates of the candidate neighbor nodes of each feature seen so far using proposed definitions of the dependence relationship and pseudo-dependence relationship and adopts a constrained greedy search to obtain the final causal structure when no new features are available. The CSSU algorithm obtains candidate neighbors with symmetrical uncertainty to avoid subset searchs to execute the conditional independence (CI) test, which significantly reduces the time complexity. Using seven benchmark Bayesian networks, the experimental results show that the CSSU algorithm improves on other state-of-the-art causal structure leaning algorithms with regard to learning accuracy and time efficiency.

Learning Bayesian network structure using Markov blanket decomposition

Causal structure learning algorithms construct Bayesian networks from observational data. Using non-interventional data, existing constraint-based algorithms may return I-equivalent partially directed acyclic graphs. However, these algorithms do not fully exploit the graphical properties of Bayesian networks, and require many redundant tests that reduce both speed and accuracy. In this paper, we introduce ideas to exploit such properties to increase the speed and accuracy of causal structure learning for multivariate normal data. In numerical experiments on five benchmarking networks our proposed algorithm was faster and more accurate than recently-developed algorithms.

Using Markov Blankets for Causal Structure Learning

We show how a generic feature-selection algorithm returning strongly relevant variables can be turned into a causal structure-learning algorithm. We prove this under the Faithfulness assumption for...