Conditional Independence Tests Research Articles

Small dataset augmentation with radial basis function approximation for causal discovery using constraint‐based method

AbstractCausal analysis involves analysis and discovery. We consider causal discovery, which implies learning and discovering causal structures from available data, owing to the significance of interpreting causal relationships in various fields. Research on causal discovery has been primarily focused on constraint‐ and score‐based interpretable methods rather than on methods based on complex deep learning models. However, identifying causal relationships in real‐world datasets remains challenging. Numerous studies have been conducted using small datasets with established ground truths. Moreover, constraint‐based methods are based on conditional independence tests. However, such tests have a lower statistical power when applied to small datasets. To solve the small sample size problem, we propose a model that generates a continuous function from available samples using radial basis function approximation. We address the problem by extracting data from the generated continuous function and evaluate the proposed method on both real and synthetic datasets generated by structural equation modeling. The proposed method outperforms constraint‐based methods using only small datasets.

Testing conditional quantile independence with functional covariate.

We propose a new non-parametric conditional independence test for a scalar response and a functional covariate over a continuum of quantile levels. We build a Cramer-von Mises type test statistic based on an empirical process indexed by random projections of the functional covariate, effectively avoiding the "curse of dimensionality" under the projected hypothesis, which is almost surely equivalent to the null hypothesis. The asymptotic null distribution of the proposed test statistic is obtained under some mild assumptions. The asymptotic global and local power properties of our test statistic are then investigated. We specifically demonstrate that the statistic is able to detect a broad class of local alternatives converging to the null at the parametric rate. Additionally, we recommend a simple multiplier bootstrap approach for estimating the critical values. The finite-sample performance of our statistic is examined through several Monte Carlo simulation experiments. Finally, an analysis of an EEG data set is used to show the utility and versatility of our proposed test statistic.

Causal-Driven Skill Prerequisite Structure Discovery

Knowing a prerequisite structure among skills in a subject domain effectively enables several educational applications, including intelligent tutoring systems and curriculum planning. Traditionally, educators or domain experts use intuition to determine the skills' prerequisite relationships, which is time-consuming and prone to fall into the trap of blind spots. In this paper, we focus on inferring the prerequisite structure given access to students' performance on exercises in a subject. Nevertheless, it is challenging since students' mastery of skills can not be directly observed, but can only be estimated, i.e., its latency in nature. To tackle this problem, we propose a causal-driven skill prerequisite structure discovery (CSPS) method in a two-stage learning framework. In the first stage, we learn the skills' correlation relationships presented in the covariance matrix from the student performance data while, through the predicted covariance matrix in the second stage, we consider a heuristic method based on conditional independence tests and standardized partial variance to discover the prerequisite structure. We demonstrate the performance of the new approach with both simulated and real-world data. The experimental results show the effectiveness of the proposed model for identifying the skills' prerequisite structure.

Testing for conditional independence of survival time from covariate

Testing for conditional independence of survival time from covariate

Variation of conditional mean and its application in ultrahigh dimensional feature screening

. A new metric, called variation of conditional mean (VCM), is proposed to measure the dependence of conditional mean of a response variable on a predictor variable. The VCM has several appealing merits. It equals zero if and only if the conditional mean of the response is independent of the predictor; it can be used for both real vector-valued variables and functional data. An estimator of the VCM is given through kernel smoothing, and a test for the conditional mean independence based on the estimated VCM is constructed. The limit distributions of the test statistic under the null hypothesis and alternative hypothesis are deduced, respectively. We further use VCM as a marginal utility to do high-dimensional feature screening to screen out variables that do not contribute to the conditional mean of the response given the predictors and prove the validity of the sure screening property. Furthermore, we find the cross variation of conditional mean (CVCM), a variant of the VCM, has a faster convergence rate than the VCM under conditional mean independence. Numerical comparison shows that the VCM and CVCM performs well in both conditional independence testing and feature screening. We also illustrate their applications to real data sets.

Testing Conditional Independence Between Latent Variables by Independence Residuals.

Conditional independence (CI) testing is an important problem, especially in causal discovery. Most testing methods assume that all variables are fully observable and then test the CI among the observed data. Such an assumption is often untenable beyond applications dealing with, e.g., psychological analysis about the mental health status and medical diagnosing (researchers need to consider the existence of latent variables in these scenarios); and typically adopted latent CI test schemes mainly suffer from robust or efficient issues. Accordingly, this article investigates the problem of testing CI between latent variables. To this end, we offer an auxiliary regression-based CI (AReCI) test by taking the measured variable as the surrogate variable of the latent variables to conduct the regression over the latent variables under the linear causal models, in which each latent variable has some certain measured variables. Specifically, given a pair of latent variables LX and LY , and a corresponding latent variable set LO , [Formula: see text] holds if and only if [Formula: see text] and [Formula: see text] are statistically independent, where A' and A'' are the two disjoint subset of the measured variable for the corresponding latent variables, A'{LO} ∩A''{LO} = ∅ , and ω1 is a parameter vector characterized from the cross covariance between A{LX} and A'{LO} , and ω2 is a parameter vector characterized from the cross covariance between A{LY} and A''{LO} . We theoretically show that the AReCI test is capable of addressing both Gaussian and non-Gaussian data. In addition, we find that the well-known partial correlation test can be seen as a special case of the AReCI test. Finally, we devise a causal discovery method by using the AReCI test as the CI test. The experimental results on synthetic and real-world data illustrate the effectiveness of our method.

Causal Feature Selection With Dual Correction.

Causal feature selection methods aim to identify a Markov boundary (MB) of a class variable, and almost all the existing causal feature selection algorithms use conditional independence (CI) tests to learn the MB. However, in real-world applications, due to data issues (e.g., noisy or small samples), CI tests can be unreliable; thus, causal feature selection algorithms relying on CI tests encounter two types of errors: false positives (i.e., selecting false MB features) and false negatives (i.e., discarding true MB features). Existing algorithms only tackle either false positives or false negatives, and they cannot deal with both types of errors at the same time, leading to unsatisfactory results. To address this issue, we propose a dual-correction-strategy-based MB learning (DCMB) algorithm to correct the two types of errors simultaneously. Specifically, DCMB selectively removes false positives from the MB features currently selected, while selectively retrieving false negatives from the features currently discarded. To automatically determine the optimal number of selected features for the selective removal and retrieval in the dual correction strategy, we design the simulated-annealing-based DCMB (SA-DCMB) algorithm. Using benchmark Bayesian network (BN) datasets, the experimental results demonstrate that DCMB achieves substantial improvements on the MB learning accuracy compared with the existing MB learning methods. Empirical studies in real-world datasets validate the effectiveness of SA-DCMB for classification against state-of-the-art causal and traditional feature selection algorithms.

Gradient-Based Local Causal Structure Learning.

Finding the causal structure from a set of variables given observational data is a crucial task in many scientific areas. Most algorithms focus on discovering the global causal graph but few efforts have been made toward the local causal structure (LCS), which is of wide practical significance and easier to obtain. LCS learning faces the challenges of neighborhood determination and edge orientation. Available LCS algorithms build on conditional independence (CI) tests, they suffer the poor accuracy due to noises, various data generation mechanisms, and small-size samples of real-world applications, where CI tests do not work. In addition, they can only find the Markov equivalence class, leaving some edges undirected. In this article, we propose a GradieNt-based LCS learning approach (GraN-LCS) to determine neighbors and orient edges simultaneously in a gradient-descent way, and, thus, to explore LCS more accurately. GraN-LCS formulates the causal graph search as minimizing an acyclicity regularized score function, which can be optimized by efficient gradient-based solvers. GraN-LCS constructs a multilayer perceptron (MLP) to simultaneously fit all other variables with respect to a target variable and defines an acyclicity-constrained local recovery loss to promote the exploration of local graphs and to find out direct causes and effects of the target variable. To improve the efficacy, it applies preliminary neighborhood selection (PNS) to sketch the raw causal structure and further incorporates an l1 -norm-based feature selection on the first layer of MLP to reduce the scale of candidate variables and to pursue sparse weight matrix. GraN-LCS finally outputs LCS based on the sparse weighted adjacency matrix learned from MLPs. We conduct experiments on both synthetic and real-world datasets and verify its efficacy by comparing against state-of-the-art baselines. A detailed ablation study investigates the impact of key components of GraN-LCS and the results prove their contribution.

Sensitivity analysis of prior distributions in Bayesian graphical modeling: Guiding informed prior choices for conditional independence testing

Bayesian analysis methods provide a significant advancement in network psychometrics, allowing researchers to use the edge inclusion Bayes factor for testing conditional independence between pairs of variables in the network. Using this methodology requires setting prior distributions on the network parameters and on the network’s structure. However, the impact of both prior distributions on the inclusion Bayes factor is underexplored. In this paper, we focus on a specific class of Markov Random Field models for ordinal and binary data. We first discuss the different choices of prior distributions for the network parameters and the network structure, and then perform an extensive simulation study to assess the sensitivity of the inclusion Bayes factor to these distributions. We pay particular attention to the effect of the scale of the prior on the inclusion Bayes factor. To improve the accessibility of the results, we also provide an interactive Shiny app. Finally, we present the R package simBgms, which provides researchers with a user-friendly tool to perform their own simulation studies for Bayesian Markov Random Field models. All of this should help researchers make more informed, evidence-based decisions when preparing to analyze empirical data using network psychometric models.

Machine Learning based on Probabilistic Models Applied to Medical Data: The Case of Prostate Cancer

The growth in the amount of data in companies puts analysts in difficulties when extracting hidden knowledge from data. Several models have emerged that focus on the notion of distances while ignoring the notion of conditional probability density. This research study focuses on segmentation using mixture models and Bayesian networks for medical data mining. As enterprise data becomes large, there is a way to apply data mining methods to make sense of it using classification methods. We designed different models with different architectures and then applied these models to the medical database. The algorithms were implemented for the real data. The objective is to classify individuals according to the conditional probability density of random variables, in addition to identifying causalities between traits from tests of conditional independence and a correlation measure, both based on χ2. After a quick illustration of several models (decision tree, SVM, K-means, Bayes), we applied our method to data from an epidemiological study (done at the University of Kinshasa University clinics) of case-control of prostate cancer. Thus, we found after interpretation of the results followed by discussion that our model allows us to classify a new individual with an accuracy of 96%.

CDSC: Causal decomposition based on spectral clustering

Causal discovery is pivotal across numerous scientific and engineering disciplines. General causal discovery methods often falter in accuracy and efficiency when dealing with high-dimensional data due to the curse of dimensionality. While recent advancements in recursive constraint-based methods offer some improvements for high-dimensional causal discovery, their reliance on high-order conditional independence (CI) tests in causal decomposition hinders efficiency and accuracy. To address this challenge, we introduce a novel causal decomposition method based on spectral clustering, called CDSC, tailored for enhanced high-dimensional causal discovery. Concretely, CDSC first constructs the Laplacian matrix of the original variable set based on correlation distance. Then, CDSC decomposes the original variable set into multiple subsets without CI tests and recovers the causal structure of each subset. After determining these causal structures, CDSC merges these sub-results into a complete causal structure. A notable advantage of CDSC over other causal decomposition methods lies in its use of simplified spectral clustering. CDSC preserves the connectivity of the subsets to avoid destroying the causality in partitioned subsets. Moreover, we analyze the validity and time complexity of CDSC theoretically. Exclusive experiments on various real-world causal structures show that CDSC outperforms the state-of-the-arts both in accuracy and efficiency significantly.

Kernel-Based Independence Tests for Causal Structure Learning on Functional Data.

Measurements of systems taken along a continuous functional dimension, such as time or space, are ubiquitous in many fields, from the physical and biological sciences to economics and engineering. Such measurements can be viewed as realisations of an underlying smooth process sampled over the continuum. However, traditional methods for independence testing and causal learning are not directly applicable to such data, as they do not take into account the dependence along the functional dimension. By using specifically designed kernels, we introduce statistical tests for bivariate, joint, and conditional independence for functional variables. Our method not only extends the applicability to functional data of the Hilbert-Schmidt independence criterion (hsic) and its d-variate version (d-hsic), but also allows us to introduce a test for conditional independence by defining a novel statistic for the conditional permutation test (cpt) based on the Hilbert-Schmidt conditional independence criterion (hscic), with optimised regularisation strength estimated through an evaluation rejection rate. Our empirical results of the size and power of these tests on synthetic functional data show good performance, and we then exemplify their application to several constraint- and regression-based causal structure learning problems, including both synthetic examples and real socioeconomic data.

Design and Validation of Structural Causal Model: A Focus on EGRA Dataset

Designing and validating structural causal model (SCM) correctness from a dataset whose background knowledge is obtained from a research process is not a common phenomenon. Studies have shown that in many critical areas, such as healthcare and education, researchers develop models from direct acyclic graphs (DAG), a component of an SCM, without testing them. This phenomenon is worrisome and is bound to cast a shadow on the inference estimates that may arise from such models. In this study, we have designed a novel application-based SCM for the first time using the background knowledge obtained from the Early Grade Reading Assessment (EGRA) program called the Strengthen Education in Northeast Nigeria (SENSE-EGRA), which is an educational intervention program of the American University of Nigeria (AUN), Yola, on the letter identification subtask. This project was sponsored by the United States Agency for International Development (USAID). We employed the conditional independence test (CIT) criteria for the validation of the SCM’s correctness, and the results show a near-perfect SCM.

Loose-to-strict Markov blanket learning algorithm for feature selection

The Markov blanket (MB) represents a crucial concept in a Bayesian network (BN) and is theoretically the optimal solution to the feature selection problem. Methods based on conditional independence (CI) tests are prevalent for MB discovery. Currently, the main challenge is how to improve both the efficiency and effectiveness of this type of method. In this paper, we propose a novel divide-and-conquer discovery algorithm, loose-to-strict MB (LSMB), to discover MBs faster while maintaining high accuracy. LSMB first discovers the approximate parent–child (PC) and spouse sets of a target variable via the loose CI test strategy, a constraint for the condition set of CI tests. Then, by strict CI tests, i.e., without constraint for the size of the condition set, LSMB first removes non-MB nodes in the discovered approximate PC set and then categorizes and removes non-MB nodes in the discovered approximate spouse set. Finally, LSMB combines the discovered sets to obtain the MB of the target. Experiments on benchmark BN datasets show that LSMB can improve the efficiency of MB discovery while maintaining higher accuracy than the state-of-the-art MB discovery algorithms, and experiments on real-world datasets demonstrate the excellent performance of LSMB in feature selection.

Nonlinear learning method for local causal structures

Recent years have witnessed the proliferation of causal learning techniques, aimed at extracting the abundant causal relationships embedded within observational data. In many scenarios, our primary focus lies in predicting a single target variable. In such cases, it becomes both inefficient and unnecessary to learn an entire causal network through advanced global learning methods. To address this challenge, the concept of local causal learning has been introduced to identify the direct causes and effects of a target variable of interest. However, current algorithms exhibit limitations stemming from their reliance on conditional independence tests, which only consider the linear and pairwise relationships but ignore the ubiquitous nonlinear and multivariate causality, making them lose efficacy in practical scenarios. This paper takes significant strides toward facilitating the real-world applications for local causal learning. To identify the nonlinear relations, this paper discovers the Markov boundary (MB) through calculating the minimal conditional covariance operator in reproducing kernel Hilbert space. This approach establishes a theoretical equivalence between the solution and MB. Subsequently, a nonlinear scoring mechanism for structure learning is employed based on the selected subset, yielding the optimal local causal structure. A series of extensive experiments serves to underscore the superiority of the proposed method.

Feature Selection in the Data Stream Based on Incremental Markov Boundary Learning.

Recent years have witnessed the proliferation of techniques for streaming data mining to meet the demands of many real-time systems, where high-dimensional streaming data are generated at high speed, increasing the burden on both hardware and software. Some feature selection algorithms for streaming data are proposed to tackle this issue. However, these algorithms do not consider the distribution shift due to nonstationary scenarios, leading to performance degradation when the underlying distribution changes in the data stream. To solve this problem, this article investigates feature selection in streaming data through incremental Markov boundary (MB) learning and proposes a novel algorithm. Different from existing algorithms focusing on prediction performance on off-line data, the MB is learned by analyzing conditional dependence/independence in data, which uncovers the underlying mechanism and is naturally more robust against the distribution shift. To learn MB in the data stream, the proposal transforms the learned information in previous data blocks to prior knowledge and employs them to assist MB discovery in current data blocks, where the likelihood of distribution shift and reliability of conditional independence test are monitored to avoid the negative impact from invalid prior information. Extensive experiments on synthetic and real-world datasets demonstrate the superiority of the proposed algorithm.

Local causal structure learning with missing data

Local causal structure learning aims to discover and distinguish the direct causes and direct effects of a target variable. However, the state-of-the-art algorithms for local causal structure learning fail to perform well when dealing with missing data. The general approach is to fill in the missing data using imputation techniques before learning the local causal structure, but this method suffers from problems such as low accuracy, low efficiency, and instability. To address these issues, we propose a novel method for local causal structure learning with missing data, named misLCS. Firstly, we design an iterative data imputation method to obtain the complete and correct data from the missing data. Then, misLCS adopts a data subset strategy to get a data subset that variables are closely related to the target variable. Thirdly, within this data subset, misLCS constructs the local causal skeleton of the target variable using a mutual information-based feature selection method and orients the direction of edges using conditional independence tests and Meek rules. Finally, misLCS updates the missing data in preparation for the next iteration. This procedure continues until the direct causes and direct effects of the target variable have been identified. Our experiments on seven benchmark Bayesian networks and a real-world bioinformatics dataset, with a number of variables from 11 to 801, demonstrate that our algorithm achieves better accuracy than the existing local causal structure learning algorithms.

Testing conditional independence in casual inference for time series data

In this paper, we propose a new procedure to test conditional independence assumption in studying casual inference for time series data. The conditional independence assumption is transformed to a nonparametric conditional moment test with the help of auxiliary variables which are allowed to affect policy choice but the dependence can be fully captured by potential outcomes and observable controls. When the policy choice is binary, a nonparametric statistic test is developed further for testing the conditional independence assumption conditional on policy propensity score. Under some regular conditions, we show that the proposed test statistics are asymptotically normal under the null hypotheses for time series data. In addition, the performances of the proposed methods are illustrated through Monte Carlo simulations and a real example considered in Angrist and Kuersteiner (2011).

Towards the use of Unsupervised Causal Learning in Wireless Networks Operation

The current paradigm in Mobile Wireless Networks (MWNs) operation is being defied by the increasing importance of Machine Learning (ML) and Artificial Intelligence (AI). Nevertheless, another paradigm shift is rising with recent developments in causal inference and causal discovery, which, although having the potential to be applied to MWNs, have been relatively unexplored. This paper aims to develop a data-driven methodology using unsupervised ML and Conditional Independence Tests (CITs), typically used in causal discovery tasks, to identify distinct network performance patterns and pinpoint causal factors to explain them. The proposed methodology was first evaluated with crowdsourcing data from User Equipments (UEs). Afterwards, a dataset from a Long-Term Evolution (LTE) network, composed of a set of arbitrary performance indicators and configuration parameters, was considered. The crowdsourcing dataset, containing multiple network speed tests, revealed that the measured uplink throughput contributed the most to the observed performance patterns due to the used Radio Access Technologies (RATs). Furthermore, the LTE dataset revealed a causal relationship between the number of reserved signalling resources in the Physical Uplink Control Channel (PUCCH) and the UE uplink throughput. Notwithstanding, the key contribution of this paper is the consideration of causal-based concepts and methods for network operations enhancement.

Test of conditional independence in factor models via Hilbert–Schmidt independence criterion

This work is concerned with testing conditional independence under a factor model setting. We propose a novel multivariate test for non-Gaussian data based on the Hilbert–Schmidt independence criterion (HSIC). Theoretically, we investigate the convergence of our test statistic under both the null and the alternative hypotheses, and devise a bootstrap scheme to approximate its null distribution, showing that its consistency is justified. Methodologically, we generalize the HSIC-based independence test approach to a situation where data follow a factor model structure. Our test requires no nonparametric smoothing estimation of functional forms including conditional probability density functions, conditional cumulative distribution functions and conditional characteristic functions under the null or alternative, is computationally efficient and is dimension-free in the sense that the dimension of the conditioning variable is allowed to be large but finite. Further extension to nonlinear, non-Gaussian structure equation models is also described in detail and asymptotic properties are rigorously justified. Numerical studies demonstrate the effectiveness of our proposed test relative to that of several existing tests.