Conditional Independence Tests Research Articles

A Light Causal Feature Selection Approach to High-Dimensional Data

Causal feature selection has received increasing attention in recent years. However, the state-of-the-art causal feature selection algorithms use the conditional independence tests, which require enumerating conditioning sets, leading to an exponential increase in computational complexity along with an increase in feature space. To address this problem, in this paper, we theoretically analyze the unique performance of causal features in mutual information, and propose a novel <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">C</u> ausal <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">F</u> eature <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">S</u> election algorithm using <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">M</u> utual <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">I</u> nformation, called CFS-MI. Specifically, CFS-MI separately instantiates the pairwise comparison of mutual information in two stages to reduce computational complexity, and thus improves the efficiency on high-dimensional data. Extensive experiments on 5 benchmark Bayesian networks and 16 real-world datasets validate that CFS-MI has comparable accuracy compared to 7 state-of-the-art causal feature selection algorithms, while presenting more superior computational efficiency.

Satellite Telemetry Data Anomaly Detection Using Causal Network and Feature-Attention-Based LSTM

Most of the data-driven satellite telemetry data anomaly detection methods suffer from high false positive rate (FPR) and poor interpretability. To solve the above problems, we propose an anomaly detection framework using causal network and feature-attention-based long short-term memory (CN-FA-LSTM). In our method, a causal network of telemetry parameters is constructed by calculating normalized modified conditional transfer entropy (NMCTE) and optimized by conditional independence tests based on the conditional mutual information (CMI). Then, a CN-FA-LSTM is established to predict telemetry data, and a nonparametric dynamic <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> -sigma threshold updating method is proposed to set thresholds. A case study on a real satellite demonstrates that anomaly detection using the CN-FA-LSTM and nonparametric dynamic <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> -sigma threshold updating has an average F1-score of 0.9462 and an FPR of 0.0021, which are better than the baseline methods. Furthermore, CN-FA-LSTM has a stronger interpretability than other commonly used prediction models. Supplementary experiment on two public datasets verifies the universal applicability of our method.

Additive Noise Model Structure Learning Based on Spatial Coordinates

A new algorithm named SCB (Spatial Coordinates Based) algorithm is presented for structure learning of additive noise model, which can effectively deal with nonlinear arbitrarily distributed data. This paper makes three specific contributions. Firstly, SC (Spatial Coordinates) coefficient is proposed to use as a standard of independence test and CSC (Conditional Spatial Coordinates) coefficient as a standard of conditional independence test. Secondly, it is proved that the CSC coefficient conforms to the standard normal distribution and the HSIC independence test can be regarded as a special case of the SC coefficient. Finally, based on the SC coefficient, the SCB algorithm is proposed, and the experimental comparison with some existing algorithms on seven classical networks shows that the SCB algorithm has better performance. In particular, SCB algorithm can deal with large sample, high dimensional nonlinear data, and maintain good accuracy and time performance.

Stable Prediction with Leveraging Seed Variable

In this paper, we focus on the problem of stable prediction across unknown test data, where the test distribution might be different from the training one and is always agnostic when model training. In such a case, previous machine learning methods might exploit subtly spurious correlations induced by non-causal variables in training data for prediction. Those spurious correlations are changeable across data, leading to instability of prediction across unknown test data. To address this problem, we propose a conditional independence test based algorithm to screen out part of non-causal features and reduce those spurious correlations for a more stable prediction by leveraging a seed variable. We show, both theoretically and with empirical experiments, that our algorithm can precisely screen out the isolated non-causal variables, which have no causal relationship with other variables, and remove the spurious correlations induced by them, increasing the stability of prediction across unknown test data. Extensive experiments on both synthetic and real-world datasets demonstrate that our algorithm outperforms state-of-the-art methods for stable prediction across unknown test data.

On the power of conditional independence testing under model-X

For testing conditional independence (CI) of a response Y and a predictor X given covariates Z, the model-X (MX) framework has been the subject of active methodological research, especially in the context of MX knockoffs and their application to genome-wide association studies. In this paper, we study the power of MX CI tests, yielding quantitative insights into the role of machine learning and providing evidence in favor of using likelihood-based statistics in practice. Focusing on the conditional randomization test (CRT), we find that its conditional mode of inference allows us to reformulate it as testing a point null hypothesis involving the conditional distribution of X. The Neyman-Pearson lemma implies that a likelihood-based statistic yields the most powerful CRT against a point alternative. We obtain a related optimality result for MX knockoffs. Switching to an asymptotic framework with arbitrarily growing covariate dimension, we derive an expression for the power of the CRT against local semiparametric alternatives in terms of the prediction error of the machine learning algorithm on which its test statistic is based. Finally, we exhibit a resampling-free test with uniform asymptotic Type-I error control under the assumption that only the first two moments of X given Z are known.

Analysis and mapping of mining subsidence and underground voids collapse risk using GIS and the weights-of-evidence model for the abandoned Ichmoul mine, Algeria

The present study concerns the assessment of mining subsidence and old underground works collapse hazard of the abandoned Ichmoul mine (Batna, Algeria) using the geographic information system (GIS) and the weight of evidence method (WofE). After the identification of collapse cases on the site and developing a spatial database for old underground works, topography and geology, the weight of evidence model was applied to calculate the weight of each relevant factor. Five main factors controlling or related to subsidence and underground voids collapse were determined from the probability analysis of existing cases; the underground voids depth, the distance between the underground voids, the landforms slope, the lithology and the distance between the fractures. Conditional independence tests were performed for the choice of factors. For the analysis of the subsidence and underground void collapse spatial hazard mapping, the contrast values, W+ and W- of the each factor evaluation were analyzed. The analysis results were validated using the curve ROC (AUC) with a past collapse case. For all the factors used, the area under the (ROC) curve showed 92 % accuracy. The results obtained can be used for the prevention of mining subsidence and underground voids collapse risks and the mine site rehabilitation.

Interventional Fairness with Indirect Knowledge of Unobserved Protected Attributes.

The deployment of machine learning (ML) systems in applications with societal impact has motivated the study of fairness for marginalized groups. Often, the protected attribute is absent from the training dataset for legal reasons. However, datasets still contain proxy attributes that capture protected information and can inject unfairness in the ML model. Some deployed systems allow auditors, decision makers, or affected users to report issues or seek recourse by flagging individual samples. In this work, we examine such systems and consider a feedback-based framework where the protected attribute is unavailable and the flagged samples are indirect knowledge. The reported samples are used as guidance to identify the proxy attributes that are causally dependent on the (unknown) protected attribute. We work under the causal interventional fairness paradigm. Without requiring the underlying structural causal model a priori, we propose an approach that performs conditional independence tests on observed data to identify such proxy attributes. We theoretically prove the optimality of our algorithm, bound its complexity, and complement it with an empirical evaluation demonstrating its efficacy on various real-world and synthetic datasets.

Conditional independence test of failure and truncation times: Essential tool for method selection

Conditional independence assumption of truncation and failure times conditioning on covariates is a fundamental and common assumption in the regression analysis of left-truncated and right-censored data. Testing for this assumption is essential to ensure the correct inference on the failure time, but this has often been overlooked in the literature. With consideration of challenges caused by left truncation and right censoring, tests for this conditional independence assumption are developed in which the generalized odds ratio derived from a Cox proportional hazards model on the failure time and the concept of Kendall's tau are combined. Except for the Cox proportional hazards model, no additional model assumptions are imposed, and the distributions of the truncation time and conditioning variables are unspecified. The asymptotic properties of the test statistic are established and an easy implementation for obtaining its distribution is developed. The performance of the proposed test has been evaluated through simulation studies and two real studies.

A high-dimensional power analysis of the conditional randomization test and knockoffs

Summary In many scientific applications, researchers aim to relate a response variable $Y$ to a set of potential explanatory variables $X = (X_1,\dots,X_p)$, and start by trying to identify variables that contribute to this relationship. In statistical terms, this goal can be understood as trying to identify those $X_j$ on which $Y$ is conditionally dependent. Sometimes it is of value to simultaneously test for each $j$, which is more commonly known as variable selection. The conditional randomization test, CRT, and model-X knockoffs are two recently proposed methods that respectively perform conditional independence testing and variable selection by computing, for each $X_j$, any test statistic on the data and assessing that test statistic’s significance, by comparing it with test statistics computed on synthetic variables generated using knowledge of the distribution of $X$. The main contribution of this article is the analysis of the power of these methods in a high-dimensional linear model, where the ratio of the dimension $p$ to the sample size $n$ converges to a positive constant. We give explicit expressions for the asymptotic power of the CRT, variable selection with CRT $p$-values, and model-X knockoffs, each with a test statistic based on the marginal covariance, the least squares coefficient or the lasso. One useful application of our analysis is direct theoretical comparison of the asymptotic powers of variable selection with CRT $p$-values and model-X knockoffs; in the instances with independent covariates that we consider, the CRT probably dominates knockoffs. We also analyse the power gain from using unlabelled data in the CRT when limited knowledge of the distribution of $X$ is available, as well as the power of the CRT when samples are collected retrospectively.

Automated hyperparameter selection for the PC algorithm

• The performance of the PC algorithm depends heavily on the choice of the Type I error rate α . • We propose to automatically tune α for a user chosen metric using a procedure called AutoPC. • AutoPC double checks the output of PC by rerunning the algorithm on restricting conditioning sets. • AutoPC provably maximizes the chosen metric and recovers the true CPDAG in the sample limit. • The algorithm also easily achieves state-of-the-art accuracy across multiple metrics with both synthetic and real data. The PC algorithm infers causal relations using conditional independence tests that require a pre-specified Type I α level. PC is however unsupervised, so we cannot tune α using traditional cross-validation. We therefore propose AutoPC, a fast procedure that optimizes α directly for a user chosen metric. We in particular force PC to double check its output by executing a second run on the recovered graph. We choose the final output as the one which maximizes stability between the two runs. AutoPC consistently outperforms the state of the art across multiple metrics.

Relaxation of Some Confusions about Confounders

This work is about observational causal discovery for deterministic and stochastic dynamic systems. We explore what additional knowledge can be gained by the usage of standard conditional independence tests and if the interacting systems are located in a geodesic space.

Nonparametric Tests of Conditional Independence for Time Series

We propose consistent nonparametric tests of conditional independence for time series data. Our methods are motivated from the difference between joint conditional cumulative distribution function (CDF) and the product of conditional CDFs. The difference is transformed into a proper conditional moment restriction (CMR), which forms the basis for our testing procedure. Our test statistics are then constructed using the integrated moment restrictions that are equivalent to the CMR. We establish the asymptotic behavior of the test statistics under the null, the alternative, and the sequence of local alternatives converging to conditional independence at the parametric rate. Our tests are implemented with the assistance of a multiplier bootstrap. Monte Carlo simulations are conducted to evaluate the finite sample performance of the proposed tests. We apply our tests to examine the predictability of equity risk premium using variance risk premium for different horizons and find that there exist various degrees of nonlinear predictability at mid-run and long-run horizons.

Exploring nonlinearity on the CO[formula omitted] emissions, economic production and energy use nexus: A causal discovery approach

We examine the interactions between growth in CO2 emissions, economic production, and energy use at the global and multi-regional levels over the period 1990–2014. Methodologically, we use causal discovery that relies on linear and nonlinear tests of conditional independence to study their relationships. At the global scale, we show that energy use is unambiguously a nonlinear causal factor of economic output, making energy central in the economic and decarbonization debate. At the multi-regional scale, causal discovery accurately identifies that three regions are driving global dynamics (East Asia and Pacific, Europe and Central Asia, and North America regions). Energy and climate policies in these regions will nonlinearly influence the degree of global CO2 emission reduction rates and will impact the global economic dynamics. Our results further suggest that policy effectiveness could be gained if a country’s climate actions were coordinated with the other geographies most affected by their consequences, providing valuable information on the implementation of Article 6 of the Paris Agreement.

Minimax optimal conditional independence testing

We consider the problem of conditional independence testing of X and Y given Z where X,Y and Z are three real random variables and Z is continuous. We focus on two main cases—when X and Y are both discrete, and when X and Y are both continuous. In view of recent results on conditional independence testing [Ann. Statist. 48 (2020) 1514–1538], one cannot hope to design nontrivial tests, which control the type I error for all absolutely continuous conditionally independent distributions, while still ensuring power against interesting alternatives. Consequently, we identify various, natural smoothness assumptions on the conditional distributions of X,Y|Z=z as z varies in the support of Z, and study the hardness of conditional independence testing under these smoothness assumptions. We derive matching lower and upper bounds on the critical radius of separation between the null and alternative hypotheses in the total variation metric. The tests we consider are easily implementable and rely on binning the support of the continuous variable Z. To complement these results, we provide a new proof of the hardness result of Shah and Peters [Ann. Statist. 48 (2020) 1514–1538].

Testing conditional independence in supervised learning algorithms

We propose the conditional predictive impact (CPI), a consistent and unbiased estimator of the association between one or several features and a given outcome, conditional on a reduced feature set. Building on the knockoff framework of Candès et al. (J R Stat Soc Ser B 80:551–577, 2018), we develop a novel testing procedure that works in conjunction with any valid knockoff sampler, supervised learning algorithm, and loss function. The CPI can be efficiently computed for high-dimensional data without any sparsity constraints. We demonstrate convergence criteria for the CPI and develop statistical inference procedures for evaluating its magnitude, significance, and precision. These tests aid in feature and model selection, extending traditional frequentist and Bayesian techniques to general supervised learning tasks. The CPI may also be applied in causal discovery to identify underlying multivariate graph structures. We test our method using various algorithms, including linear regression, neural networks, random forests, and support vector machines. Empirical results show that the CPI compares favorably to alternative variable importance measures and other nonparametric tests of conditional independence on a diverse array of real and synthetic datasets. Simulations confirm that our inference procedures successfully control Type I error with competitive power in a range of settings. Our method has been implemented in an R package, cpi, which can be downloaded from https://github.com/dswatson/cpi.

Fast and powerful conditional randomization testing via distillation.

We consider the problem of conditional independence testing: given a response and covariates , we test the null hypothesis that . The conditional randomization test was recently proposed as a way to use distributional information about to exactly and nonasymptotically control Type-I error using any test statistic in any dimensionality without assuming anything about . This flexibility, in principle, allows one to derive powerful test statistics from complex prediction algorithms while maintaining statistical validity. Yet the direct use of such advanced test statistics in the conditional randomization test is prohibitively computationally expensive, especially with multiple testing, due to the requirement to recompute the test statistic many times on resampled data. We propose the distilled conditional randomization test, a novel approach to using state-of-the-art machine learning algorithms in the conditional randomization test while drastically reducing the number of times those algorithms need to be run, thereby taking advantage of their power and the conditional randomization test's statistical guarantees without suffering the usual computational expense. In addition to distillation, we propose a number of other tricks, like screening and recycling computations, to further speed up the conditional randomization test without sacrificing its high power and exact validity. Indeed, we show in simulations that all our proposals combined lead to a test that has similar power to the most powerful existing conditional randomization test implementations, but requires orders of magnitude less computation, making it a practical tool even for large datasets. We demonstrate these benefits on a breast cancer dataset by identifying biomarkers related to cancer stage.

Constraint-based learning for non-parametric continuous bayesian networks

Modeling high-dimensional multivariate distributions is a computationally challenging task. In the discrete case, Bayesian networks have been successfully used to reduce the complexity and to simplify the problem. However, they lack of a general model for continuous variables. In order to overcome this problem, Elidan (2010) proposed the model of copula Bayesian networks that parametrizes Bayesian networks using copula functions. We propose a new learning algorithm for this model based on a PC algorithm and a conditional independence test proposed by Bouezmarni et al. (2009). This test being non-parametric, no model assumptions are made allowing it to be as general as possible. This algorithm is compared on generated data with the parametric method proposed by Elidan (2010) and proves to have better results.

Testing Independence Between Linear Combinations for Causal Discovery

Recently, regression based conditional independence (CI) tests have been employed to solve the problem of causal discovery. These methods provide an alternative way to test for CI by transforming CI to independence between residuals. Generally, it is nontrivial to check for independence when these residuals are linearly uncorrelated. With the ability to represent high-order moments, kernel-based methods are usually used to achieve this goal, but at a cost of considerable time. In this paper, we investigate the independence between two linear combinations under linear non-Gaussian structural equation model (SEM). We show that generally the 1-st to 4-th moments of the two linear combinations contain enough information to infer whether or not they are independent. The proposed method provides a simpler but more effective way to measure CIs, with only calculating the 1-st to 4-th moments of the input variables. When applied to causal discovery, the proposed method outperforms kernel-based methods in terms of both speed and accuracy. which is validated by extensive experiments.

Causality‐based online streaming feature selection

AbstractOnline streaming feature selection, as a well‐known and effective preprocessing approach in machine learning, is an eternal topic. Amount of online streaming feature selection algorithms have achieved a great deal of success in classification and prediction tasks. However, most of these existing algorithms only concentrate on the relevance between features and labels, and neglect the causal relationships between them. Discovering the potential causal relationships between features and labels, that is, the Markov blanket (MB) of class label, which can build a more interpretable and robust classification model. In this paper, we put forward a causality‐based online streaming feature selection algorithm with neighborhood conditional mutual information. First, we apply neighborhood symmetrical uncertainty to discover a candidate Markov blanket (CMB) with causal information. Then, neighborhood conditional mutual information instead of conditional independence test is used to delete the false positives in CMB, which can significantly alleviate the computational cost. Moreover, we utilize the updated CMB to choose the true spouses, which may be mistakenly deleted during the process of removing false positives, and then acquire an optimal MB as the online selected feature subset. Finally, causality‐based online streaming feature selection with neighborhood conditional mutual information is compared with four well‐established online streaming feature selection methods on 13 real‐world datasets. Experiment results show that the proposed algorithm outperforms these online streaming feature selection algorithms.

An order independent algorithm for inferring gene regulatory network using quantile value for conditional independence tests

In recent years, due to the difficulty and inefficiency of experimental methods, numerous computational methods have been introduced for inferring the structure of Gene Regulatory Networks (GRNs). The Path Consistency (PC) algorithm is one of the popular methods to infer the structure of GRNs. However, this group of methods still has limitations and there is a potential for improvements in this field. For example, the PC-based algorithms are still sensitive to the ordering of nodes i.e. different node orders results in different network structures. The second is that the networks inferred by these methods are highly dependent on the threshold used for independence testing. Also, it is still a challenge to select the set of conditional genes in an optimal way, which affects the performance and computation complexity of the PC-based algorithm. We introduce a novel algorithm, namely Order Independent PC-based algorithm using Quantile value (OIPCQ), which improves the accuracy of the learning process of GRNs and solves the order dependency issue. The quantile-based thresholds are considered for different orders of CMI tests. For conditional gene selection, we consider the paths between genes with length equal or greater than 2 while other well-known PC-based methods only consider the paths of length 2. We applied OIPCQ on the various networks of the DREAM3 and DREAM4 in silico challenges. As a real-world case study, we used OIPCQ to reconstruct SOS DNA network obtained from Escherichia coli and GRN for acute myeloid leukemia based on the RNA sequencing data from The Cancer Genome Atlas. The results show that OIPCQ produces the same network structure for all the permutations of the genes and improves the resulted GRN through accurately quantifying the causal regulation strength in comparison with other well-known PC-based methods. According to the GRN constructed by OIPCQ, for acute myeloid leukemia, two regulators BCLAF1 and NRSF reported previously are significantly important. However, the highest degree nodes in this GRN are ZBTB7A and PU1 which play a significant role in cancer, especially in leukemia. OIPCQ is freely accessible at https://github.com/haammim/OIPCQ-and-OIPCQ2.