Conditional Independence Research Articles

Learning the structure of multivariate regression chain graphs by testing complete separators in prime blocks

This paper introduces an algorithm to construct a bidirectional causal graph using an augmented graph. The algorithm decomposes the augmented graph, significantly reducing the size of the variable set required for conditional independence testing. Simultaneously, it preserves the fundamental structure of the augmented graph after decomposition, saving time and cost in constructing a global skeleton graph. Through experiments on discrete and continuous datasets, the algorithm demonstrates a clear advantage in runtime compared to traditional methods. In large-scale sparse networks, the training time is only about one-tenth of traditional methods. Additionally, the algorithm shows improvement in terms of construction error. Since the input to the algorithm is an augmented graph, this paper also discusses the impact on construction error when using both real and generated augmented graphs as input. Furthermore, the concept of markov blanket is extended to multivariate regression chain graphs, providing a method for rapidly constructing augmented graphs given certain prior knowledge.

Analyzing asymmetry in brain hierarchies with a linear state-space model of resting-state fMRI data.

This study challenges the traditional focus on zero-lag statistics in resting-state functional magnetic resonance imaging (rsfMRI) research. Instead, it advocates for considering time-lag interactions to unveil the directionality and asymmetries of the brain hierarchy. Effective connectivity (EC), the state matrix in dynamical causal modeling (DCM), is a commonly used metric for studying dynamical properties and causal interactions within a linear state-space system description. Here, we focused on how time-lag statistics are incorporated within the framework of DCM resulting in an asymmetric EC matrix. Our approach involves decomposing the EC matrix, revealing a steady-state differential cross-covariance matrix that is responsible for modeling information flow and introducing time-irreversibility. Specifically, the system's dynamics, influenced by the off-diagonal part of the differential covariance, exhibit a curl steady-state flow component that breaks detailed balance and diverges the dynamics from equilibrium. Our empirical findings indicate that the EC matrix's outgoing strengths correlate with the flow described by the differential cross covariance, while incoming strengths are primarily driven by zero-lag covariance, emphasizing conditional independence over directionality.

A novel constraint-based structure learning algorithm using marginal causal prior knowledge

Causal discovery with prior knowledge is important for improving performance. We consider the incorporation of marginal causal relations, which correspond to the presence or absence of directed paths in a causal model. We propose the Marginal Prior Causal Knowledge PC (MPPC) algorithm to incorporate marginal causal relations into a constraint-based structure learning algorithm. We provide the theorems of conditional independence properties by combining observational data and marginal causal relations. We compare the MPPC algorithm with other structure learning methods in both simulation studies and real-world networks. The results indicate that, compare with other constraint-based structure learning methods, MPPC algorithm can incorporate marginal causal relations and is more effective and more efficient.

Long‐range Ising model for regional‐scale seismic risk analysis

AbstractThis study introduces the long‐range Ising model from statistical mechanics to the performance‐based earthquake engineering (PBEE) framework for regional seismic damage analysis. The application of the PBEE framework at a regional scale involves estimating the damage states () of numerous structures, typically performed using fragility function‐based stochastic simulations. However, these simulations often assume conditional independence or employ simplistic dependency models among the of structures, leading to significant misrepresentation of regional risk. The Ising model addresses this issue by converting the available information on binary (safe or failure) into a joint probability mass function, leveraging the principle of maximum entropy. The Ising model offers two main benefits: (1) it requires only the first‐ and second‐order cross‐moments, enabling seamless integration with the existing PBEE framework, and (2) it provides meaningful physical interpretations of the model parameters, facilitating the uncovering of insights not apparent from data. To demonstrate the proposed method, we applied the Ising model to 156 buildings in Antakya, Turkey, using post‐hazard damage evaluation data, and to 182 buildings in Pacific Heights, San Francisco, using simulated data from the Regional Resilience Determination tool. In both instances, the Ising model accurately reproduces the provided information and generates meaningful insights into regional damage. The study also investigates the change in Ising model parameters under varying earthquake magnitudes, along with the mean‐field approximation, further facilitating the applicability of the proposed approach.

Computational Test for Conditional Independence

Conditional Independence (CI) testing is fundamental in statistical analysis. For example, CI testing helps validate causal graphs or longitudinal data analysis with repeated measures in causal inference. CI testing is difficult, especially when testing involves categorical variables conditioned on a mixture of continuous and categorical variables. Current parametric and non-parametric testing methods are designed for continuous variables and can quickly fall short in the categorical case. This paper presents a computational approach for CI testing suited for categorical data types, which we call computational conditional independence (CCI) testing. The test procedure is based on permutation and combines machine learning prediction algorithms and Monte Carlo cross-validation. We evaluated the approach through simulation studies and assessed the performance against alternative methods: the generalized covariance measure test, the kernel conditional independence test, and testing with multinomial regression. We find that the computational approach to testing has utility over the alternative methods, achieving better control over type I error rates. We hope this work can expand the toolkit for CI testing for practitioners and researchers.

A Unique Conditions Model for Landslide Susceptibility Mapping

Several methods and approaches have been proposed to assess landslide susceptibility. The likelihood of landslides occurring can be determined by applying statistical models to historical landslides, taking into account controlling factors. Popular methods for predicting the probability of landslides are weights-of-evidence and logistic regression. We discuss the assumptions and interpretations of these methods, the relationships between them, and their strengths and weaknesses in case of categorical factors. Of particular interest is the conditional independence of the controlling factors and its effect on model bias. To avoid lack of conditional independence of factors and model bias, we present a unique conditions model that is always unbiased. To illustrate the theoretical developments, a practical application is given using observed landslides and geo-environmental factors from a previous study. The unique conditions model appears superior to the other models.

Construction of pan-cancer regulatory networks based on causal inference

The pan-cancer initiative aims to study the origin patterns of cancer cell, the processes of carcinogenesis, and the signaling pathways from a perspective that spans across different types of cancer. The construction of the pan-cancer related gene regulatory network is helpful to excavate the commonalities in regulatory relationships among different types of cancers. It also aids in understanding the mechanisms behind cancer occurrence and development, which is of great scientific significance for cancer prevention and treatment. In light of the high dimension and large sample size of pan-cancer omics data, a causal pan-cancer gene regulation network inference algorithm based on stochastic complexity is proposed. With the network construction strategy of local first and then global, the stochastic complexity is used in the conditional independence test and causal direction inference for the candidate adjacent node set of the target nodes. This approach aims to decrease the time complexity and error rate of causal network learning. By applying this algorithm to the sample data of seven types of cancers in the TCGA database, including breast cancer, lung adenocarcinoma, and so on, the pan-cancer related causal regulatory networks are constructed, and their biological significance is verified. The experimental results show that this algorithm can eliminate the redundant regulatory relationships effectively and infer the pan-cancer regulatory network more accurately (https://github.com/LindeEugen/CNI-SC).

Beyond conjugacy for chain event graph model selection

Chain event graphs are a family of probabilistic graphical models that generalise Bayesian networks and have been successfully applied to a wide range of domains. Unlike Bayesian networks, these models can encode context-specific conditional independencies as well as asymmetric developments within the evolution of a process. More recently, new model classes belonging to the chain event graph family have been developed for modelling time-to-event data to study the temporal dynamics of a process. However, existing Bayesian model selection algorithms for chain event graphs and its variants rely on all parameters having conjugate priors. This is unrealistic for many real-world applications. In this paper, we propose a mixture modelling approach to model selection in chain event graphs that does not rely on conjugacy. Moreover, we show that this methodology is more amenable to being robustly scaled than the existing model selection algorithms used for this family. We demonstrate our techniques on simulated datasets.

Discovering Causal Models with Optimization: Confounders, Cycles, and Instrument Validity

We propose a new optimization-based method for learning causal structures from observational data, a process known as causal discovery. Our method takes as input observational data over a set of variables and returns a graph in which causal relations are specified by directed edges. We consider a highly general search space that accommodates latent confounders and feedback cycles, which few extant methods do. We formulate the discovery problem as an integer program, and propose a solution technique that exploits the conditional independence structure in the data to identify promising edges for inclusion in the output graph. In the large-sample limit, our method recovers a graph that is (Markov) equivalent to the true data-generating graph. Computationally, our method is competitive with the state-of-the-art, and can solve in minutes instances that are intractable for alternative causal discovery methods. We leverage our method to develop a procedure for investigating the validity of an instrumental variable and demonstrate it on the influential quarter-of-birth and proximity-to-college instruments for estimating the returns to education. In particular, our procedure complements existing instrument tests by revealing the precise causal pathways that undermine instrument validity, highlighting the unique merits of the graphical perspective on causality. This paper was accepted by J. George Shanthikumar, data science. Supplemental Material: The online appendix and data files are available at https://doi.org/10.1287/mnsc.2021.02066 .

A note on incorrect inferences in non-binary qualitative probabilistic networks

Qualitative probabilistic networks (QPNs) combine the conditional independence assumptions of Bayesian networks with the qualitative properties of positive and negative dependence. They formalise various intuitive properties of positive dependence to allow inferences over a large network of variables. However, we will demonstrate in this paper that, due to an incorrect symmetry property, many inferences obtained in non-binary QPNs are not mathematically true. We will provide examples of such incorrect inferences and briefly discuss possible resolutions.

Relational-branchformer: Novel framework for audio-visual speech recognition

This study embraced the state-of-the-art Branchformer series architecture within the realm of automatic speech recognition, supplanting the widely utilized Conformer architecture. This substitution offers an innovative remedy tailored to audio-visual speech recognition tasks. Building upon the Branchformer architecture, enhancements were made, culminating in the proposal of the Relational-Branchformer (R-Branchformer). The convolutional attention relation module was innovatively incorporated to augment the connectivity between the local and global branches by meticulously considering their interrelations and interplays. Consequently, this module facilitates the mutual embedding of local and global contextual information, ultimately leading to a substantial enhancement in model performance. Our model was grounded in the utilization of the connectionist temporal classification (CTC) loss, wherein intermediate CTC losses were incorporated between blocks. Moreover, through the reference and enhancement of the gated interlayer collaboration module, which superseded the inter CTC module, the conditional independence assumption intrinsic to the CTC model was effectively relaxed. As a consequence, this augmentation markedly bolstered the overall performance of our model. Furthermore, the audio-visual output enhancement module was proposed, which adeptly assimilates information from both audio and visual modalities to enrich the representation of audio-visual information. Consequently, the R-Branchformer model achieved remarkable word error rates of 1.7% and 1.5% on the LRS2 and LRS3 test sets, respectively, exemplifying its state-of-the-art performance in audio-visual speech recognition tasks.

Is private education worth it? Evidence from school-to-work transitions in Chile

Using comprehensive longitudinal data from Chile, we examine the impact of attending a private high school on labor market outcomes. The empirical strategy exploits conditional independence assumptions and leverages the effects of self-selection based on ability. We establish that private high schools boost average adult earnings by 99–144 dollars a month (relative to public schools), equivalent to a 15%–22% premium. We then explore potential channels behind these effects. Both academic and non-academic factors emerge as mediators. Our findings highlight the importance of financial resources, as education investments have long-term impacts on private high school students, while yielding negligible effects for those attending public and voucher schools. Finally, we document the prominent role of firms as mediators of the private school advantages during the school-to-work transition. Our analysis provides new insights into the association between school choice and income disparities, even after controlling for pre-labor market academic performance.

Additive dynamic Bayesian networks for enhanced feature learning in soft sensor modeling

Due to the advantages of indicating variable structure and efficient reasoning, Bayesian Networks (BN) have been widely used in data-driven soft sensor applications. However, restricted to linear and conditional Gaussian property, BN-based soft sensors rarely achieve high prediction accuracy. In this paper, an ensemble learning framework – additive dynamic Bayesian networks (ADBN) is proposed for enhanced feature learning, in which Dynamic Bayesian networks are used to learn the conditional independencies among variables and construct feature sets for the following base learners. Additional DBNs are constructed upon the residual information from the past model, to carry out feature learning to fit the residuals. The procedure is repeated and a termination rule from the perspective of feature learning is proposed to end this process, and thus the model complexity can be well restricted. The proposed method is validated on two actual industrial cases. It reveals that the ADBN feature learning method has obtained great improvements. Compared to the single DBN feature engineering method, the root mean square error (RMSE) performance has been improved by 20% and 13% on the two cases, respectively.

Total variation convergence preserves conditional independence

This note establishes that if a sequence Pn,n=1,… of probability measures converges in total variation to the limiting probability measure P, and σ-algebras A and B are conditionally independent given H with respect to Pn for all n, then they are also conditionally independent with respect to the limiting measure P. As a corollary, this also extends to pointwise convergence of densities to a density.

Summary statistics knockoffs inference with family-wise error rate control.

Testing multiple hypotheses of conditional independence with provable error rate control is a fundamental problem with various applications. To infer conditional independence with family-wise error rate (FWER) control when only summary statistics of marginal dependence are accessible, we adopt GhostKnockoff to directly generate knockoff copies of summary statistics and propose a new filter to select features conditionally dependent on the response. In addition, we develop a computationally efficient algorithm to greatly reduce the computational cost of knockoff copies generation without sacrificing power and FWER control. Experiments on simulated data and a real dataset of Alzheimer's disease genetics demonstrate the advantage of the proposed method over existing alternatives in both statistical power and computational efficiency.

Visibility graph-based covariance functions for scalable spatial analysis in non-convex partially Euclidean domains.

We present a new method for constructing valid covariance functions of Gaussian processes for spatial analysis in irregular, non-convex domains such as bodies of water. Standard covariance functions based on geodesic distances are not guaranteed to be positive definite on such domains, while existing non-Euclidean approaches fail to respect the partially Euclidean nature of these domains where the geodesic distance agrees with the Euclidean distances for some pairs of points. Using a visibility graph on the domain, we propose a class of covariance functions that preserve Euclidean-based covariances between points that are connected in the domain while incorporating the non-convex geometry of the domain via conditional independence relationships. We show that the proposed method preserves the partially Euclidean nature of the intrinsic geometry on the domain while maintaining validity (positive definiteness) and marginal stationarity of the covariance function over the entire parameter space, properties which are not always fulfilled by existing approaches to construct covariance functions on non-convex domains. We provide useful approximations to improve computational efficiency, resulting in a scalable algorithm. We compare the performance of our method with those of competing state-of-the-art methods using simulation studies on synthetic non-convex domains. The method is applied to data regarding acidity levels in the Chesapeake Bay, showing its potential for ecological monitoring in real-world spatial applications on irregular domains.

Mining Delay Propagation Causality within an Airport Network from Historical Data

Airport networks are interconnected through flight routes, with delays at upstream airports leading to delays at downstream airports, thus causing delay propagation. Exploring the mechanisms of delay propagation in airport networks provides scientific insights for managing and controlling delays in aviation systems. Existing methods, such as Granger causality tests and transfer entropy, must be revised to address the nonlinear causal relationships of delays in airport networks. So, this paper proposes a causality mining method for delay propagation in airport networks based on partial correlation-based multivariate conditional independence (PCMCI). This method comprehensively considers all airports and causality mining in two stages. The first stage uses conditional independence tests to obtain the parent node set of the target airport, which includes both true and false causal relationships. The second stage employs instantaneous conditional independence tests to eliminate false causal relationships and obtain test statistics representing the strength of causality. Based on historical delay data from US airports over a year, the experimental results show that multiple factors cause delay propagation in airport networks rather than a single causal relationship. The scope of delay propagation is limited, mainly affecting a few airports closely connected to it. Delays at airports with small flight volumes are more likely to propagate. Few airport pairs in the network mutually propagate delays and, often, delays at airports affected by a particular airport’s delay also exhibit causal relationships with each other. This method provides a new perspective for deepening the understanding of delay propagation mechanisms in airport networks.

Quantum circuits for discrete graphical models

Graphical models are useful tools for describing structured high-dimensional probability distributions. The development of efficient algorithms for generating samples thereof remains an active research topic. In this work, we provide a quantum algorithm that enables the generation of unbiased and independent samples from general discrete graphical models. To this end, we identify a coherent embedding of the graphical model based on a repeat-until-success sampling scheme which clearly identifies whether a drawn sample represents the underlying distribution. Furthermore, we show that the success probability for finding a valid sample can be lower bounded with a quantity that depends on the number of maximal cliques and the model parameter norm. Moreover, we rigorously proof that the quantum embedding conserves the key property of graphical models, i.e., factorization over the cliques of the underlying conditional independence structure. The quantum sampling algorithm also allows for maximum likelihood learning as well as maximum a posteriori state approximation for the graphical model. Finally, the proposed quantum method shows potential to realize interesting problems on near-term quantum processors. In fact, illustrative experiments demonstrate that our method can carry out sampling and parameter learning not only with idealized simulations of quantum computers but also on actual quantum hardware solely supported by simple readout error mitigation.

A modeling framework for detecting and leveraging node-level information in Bayesian network inference.

Bayesian graphical models are powerful tools to infer complex relationships in high dimension, yet are often fraught with computational and statistical challenges. If exploited in a principled way, the increasing information collected alongside the data of primary interest constitutes an opportunity to mitigate these difficulties by guiding the detection of dependence structures. For instance, gene network inference may be informed by the use of publicly available summary statistics on the regulation of genes by genetic variants. Here we present a novel Gaussian graphical modeling framework to identify and leverage information on the centrality of nodes in conditional independence graphs. Specifically, we consider a fully joint hierarchical model to simultaneously infer (i) sparse precision matrices and (ii) the relevance of node-level information for uncovering the sought-after network structure. We encode such information as candidate auxiliary variables using a spike-and-slab submodel on the propensity of nodes to be hubs, which allows hypothesis-free selection and interpretation of a sparse subset of relevant variables. As efficient exploration of large posterior spaces is needed for real-world applications, we develop a variational expectation conditional maximization algorithm that scales inference to hundreds of samples, nodes and auxiliary variables. We illustrate and exploit the advantages of our approach in simulations and in a gene network study which identifies hub genes involved in biological pathways relevant to immune-mediated diseases.

Integrating Structural Causal Model Ontologies with LIME for Fair Machine Learning Explanations in Educational Admissions

This study employed knowledge discovery in databases (KDD) to extract and discover knowledge from the Benue State Polytechnic (Benpoly) admission database and used a structural causal model (SCM) ontological framework to represent the admission process in the Nigerian polytechnic education system. The SCM ontology identified important causal relations in features needed to model the admission process and was validated using the conditional independence test (CIT) criteria. The SCM ontology was further employed to identify and constrain input features causing bias in the local interpretable model-agnostic explanations (LIME) framework applied to machine learning (ML) black-box predictions. The ablation process produced more stable LIME explanations devoid of fairness bias compared to LIME without ablation, with higher prediction accuracy (91% vs. 89%) and F1 scores (95% vs. 94%). The study also compared the performance of different ML models, including Gaussian Naïve Bayes, Decision Trees, and Logistic Regression, before and after ablation. The limitation is that the SCM ontology is qualitative and context-specific, so the fair-LIME framework can only be extrapolated to similar contexts. Future work could compare other explanation frameworks like Shapley on the same dataset. Overall, this study demonstrates a novel approach to enforcing fairness in ML explanations by integrating qualitative SCM ontologies with quantitative ML/LIME methods.