Causal Representation Research Articles

Attention-based causal representation learning for out-of-distribution recommendation

Attention-based causal representation learning for out-of-distribution recommendation

Causal-relationship representation enhanced joint extraction model for elements and relationships

The joint extraction of elements such as entities, relations, events, and their arguments, along with their specific interrelationships, is a critical task in natural language processing. Existing research often employs implicit handling of interactions between tasks through techniques like shared encodings or parameter sharing, lacking explicit modeling of the specific relationships between tasks. This limitation hinders the full utilization of inter-task correlation information and impacts effective collaboration between tasks. To address this, we propose a model for joint extraction of elements and relations based on causal relationship representation enhancement (CRE). This model aims to capture specific relationships between tasks in multiple stages, facilitating finer adjustments and optimization of subtasks and thereby enhancing the overall model performance. Specifically, CRE comprises three key modules: feature adaptation, feature interaction, and feature fusion. The feature adaptation module selects and adjusts features from shared encodings based on the requirements of specific tasks to better adapt to semantic differences between different tasks. The feature interaction module employs causal reasoning to comprehensively capture the causal relationships between tasks while mitigating negative transfer brought about by interference from semantic information. The feature fusion module further integrates features to obtain optimized task-specific representations. Ultimately, CRE exhibits a significant improvement in average performance across multiple information extraction tasks.

Causal representation learning through higher-level information extraction

The large gap between the generalization level of state-of-the-art machine learning and human learning systems calls for the development of artificial intelligence (AI) models that are truly inspired by human cognition. In tasks related to image analysis, searching for pixel-level regularities has reached a power of information extraction still far from what humans capture with image-based observations. This leads to poor generalization when even small shifts occur at the level of the observations. We explore a perspective on this problem that is directed to learning the generative process with causality-related foundations, using models capable of combining symbolic manipulation, probabilistic reasoning and pattern recognition abilities. We briefly review and explore connections of research from machine learning, cognitive science and related fields of human behavior to support our perspective for the direction to more robust and human-like artificial learning systems.

Inferring gene regulatory networks with graph convolutional network based on causal feature reconstruction

Inferring gene regulatory networks through deep learning and causal inference methods is a crucial task in the field of computational biology and bioinformatics. This study presents a novel approach that uses a Graph Convolutional Network (GCN) guided by causal information to infer Gene Regulatory Networks (GRN). The transfer entropy and reconstruction layer are utilized to achieve causal feature reconstruction, mitigating the information loss problem caused by multiple rounds of neighbor aggregation in GCN, resulting in a causal and integrated representation of node features. Separable features are extracted from gene expression data by the Gaussian-kernel Autoencoder to improve computational efficiency. Experimental results on the DREAM5 and the mDC dataset demonstrate that our method exhibits superior performance compared to existing algorithms, as indicated by the higher values of the AUPRC metrics. Furthermore, the incorporation of causal feature reconstruction enhances the inferred GRN, rendering them more reasonable, accurate, and reliable.

Information-based Gradient enhanced Causal Learning Graph Neural Network for fault diagnosis of complex industrial processes

By representing the embedded components and their interactions in industrial systems as nodes and edges in a graph, Graph Neural Networks (GNNs) have achieved outstanding results due to their ability to model statistical correlations. However, these correlations may not capture the true causal relationships within the data, thereby impairing the model’s performance in fault diagnosis.To address this issue, an Information-based Gradient enhanced Causal Learning Graph Neural Network (IGCL-GNN) is proposed for fault diagnosis of complex industrial processes. First, the information theory in graph representations is theoretically analyzed and the optimization objectives are derived separately for the causal and non-causal parts of the graph neural network, which decouple it into a multi-objective optimization problem. Then, to optimize such problem, a causal disentanglement layer is designed in the graph network that could effectively separate causal and non-causal information in graph representations. Thirdly, a novel gradient reactivation method is proposed to dynamically filter features from the disentangled layers, thereby capture the causal representations of graph data more accurately. For robust and efficient optimization, the multi-objective gradient descent algorithm is employed in this paper. Finally, comparative experiments were conducted on the three-phase flow facility (TFF) dataset, achieving a fault diagnosis accuracy of 98.42% for our proposed method.

Domain generalization via causal fine-grained feature decomposition and learning

Domain generalization aims to accurately predict unknown data using models trained by known domain data. Learning domain-invariant representations based on causal inference is one of the popular directions in domain generalization. However, existing domain generalization models based on causal inference cannot correctly determine the invariance conditions of the data. It is because of the interdependent relationship between domain-invariant features and confounding factors that ultimately leads to difficulty in obtaining truly invariant representations by the model. In this paper, we propose a novel domain generalization method called causal fine-grained feature decomposition and learning (CFFDL), which aims to eliminate latent confounding factors and learn causal domain-invariant representations in image classification tasks. Specifically, we design a feature decomposition module based on mutual information, which can decompose deep features into fine-grained feature factors and achieve factor independence between different dimensions, thereby helping to eliminate confounding factors. Furthermore, we introduce a causal representations learning module that can effectively filter and extract relevant causal features of the prediction task while eliminating the influence of confounding factors, improving the model’s performance on domain generalization. Extensive experiments on three domain generalization datasets VLCS, PACS and Office–Home show that our method outperforms the current state-of-the-art models, proving its effectiveness and superiority on domain generalization tasks of image classification.

Neural mechanisms of credit assignment for delayed outcomes during contingent learning.

Adaptive behavior in complex environments critically relies on the ability to appropriately link specific choices or actions to their outcomes. However, the neural mechanisms that support the ability to credit only those past choices believed to have caused the observed outcomes remain unclear. Here, we leverage multivariate pattern analyses of functional magnetic resonance imaging (fMRI) data and an adaptive learning task to shed light on the underlying neural mechanisms of such specific credit assignment. We find that the lateral orbitofrontal cortex (lOFC) and hippocampus (HC) code for the causal choice identity when credit needs to be assigned for choices that are separated from outcomes by a long delay, even when this delayed transition is punctuated by interim decisions. Further, we show when interim decisions must be made, learning is additionally supported by lateral frontopolar cortex (FPl). Our results indicate that FPl holds previous causal choices in a "pending" state until a relevant outcome is observed, and the fidelity of these representations predicts the fidelity of subsequent causal choice representations in lOFC and HC during credit assignment. Together, these results highlight the importance of the timely reinstatement of specific causes in lOFC and HC in learning choice-outcome relationships when delays and choices intervene, a critical component of real-world learning and decision making.

Causal Representation Learning for GAN-Generated Face Image Quality Assessment

Causal Representation Learning for GAN-Generated Face Image Quality Assessment

A personal thermal comfort model based on causal artificial intelligence: a physiological sensor-enabled causal identifiability.

Personal thermal comfort affects occupants' health, well-being, and productivity. Its satisfaction is subjective, based on individual characteristics and dynamic environments, and challenging to understand, requiring predicted outcomes and explanations of how and why the outcomes happen. This research fulfills this concern using a personal thermal comfort model based on causal artificial intelligence. It encodes personal thermal comfort satisfaction based on a new causal-and-effect framework to connect the human mind and environmental factors. Random variables encode relevant factors, and the structural causal model performs cause-and-effect relationships. The do-calculus (e.g., d-separated and d-connected) draws the common sense of the model based on causal structure representation, contributing to a human-intelligent understanding. A directed acyclic graph and exact-inference-based variable elimination quantify the model parameters based on real-world observational data. The strength of causal relationships is verified based on causal odd ratio, causal sensitivity, and causal impact. The results highlight that our proposed model can encode physiological and physical factors to predict and explain personal thermal comfort satisfaction. It can predict and explain such satisfaction reasonably and robustly, converging to human-like interpretation. It can be applied to intelligent systems to understand personal thermal comfort.

Diffusion-Based Causal Representation Learning.

Causal reasoning can be considered a cornerstone of intelligent systems. Having access to an underlying causal graph comes with the promise of cause-effect estimation and the identification of efficient and safe interventions. However, learning causal representations remains a major challenge, due to the complexity of many real-world systems. Previous works on causal representation learning have mostly focused on Variational Auto-Encoders (VAEs). These methods only provide representations from a point estimate, and they are less effective at handling high dimensions. To overcome these problems, we propose a Diffusion-based Causal Representation Learning (DCRL) framework which uses diffusion-based representations for causal discovery in the latent space. DCRL provides access to both single-dimensional and infinite-dimensional latent codes, which encode different levels of information. In a first proof of principle, we investigate the use of DCRL for causal representation learning in a weakly supervised setting. We further demonstrate experimentally that this approach performs comparably well in identifying the latent causal structure and causal variables.

Understanding of causality and its mathematical representation in accident modeling

Accident models are a critical element of safety science as they provide a profound understanding of accident causality, which helps develop accident prevention and control strategies. The evolving accident situations and complex engineering systems offer significant challenges in understanding causality. This limits the realistic representation of causality in the accident model, impacting its usefulness. This paper presents a framework to demystify the understanding of causality and its mathematical representation in accident modeling. The framework uses the theory of causality, understanding interdependencies, constructing these elements in mathematical representations to formulate a mathematical accident model, and subsequently utilizing advanced probability theory and machine learning for accident analysis. The methodology is demonstrated in the case of the Champlain Tower South collapse. The ultimate objective of this work is that readers can use this framework for any engineering system accident modeling (i.e., construction, road, or process systems). The framework will help develop accident preventive and control strategies.

CDRM: Causal disentangled representation learning for missing data

Missing data pose significant challenges during representation learning of observational data. The incompleteness of data can result in a deterioration of generative performance in disentangled representation learning. Conventional data imputation solutions, such as regression imputation or multiple imputation, often neglect the underlying causal relationships among the data. To address these issues, the causal disentangled representation learning for missing data (CDRM) framework was proposed. Missing data with graph representations are integrated to construct heterogeneous networks composed of observations, features, and known feature values. To achieve data completeness, an interaction module consisting of a parallel neighbor interaction layer and an embedded update layer are integrated with the heterogeneous network to predict missing values. To recover the true causal relationship of missing data, edge embeddings are further introduced during message passing of heterogeneous networks, which can capture the interaction of different features and enrich the representation of observations. Furthermore, the causal relationship is incorporated into the VAE using two distinct encoders to learn representations of causally related concepts. In a series of experimental evaluations on diverse datasets, CDRM consistently outperforms the state-of-the-art method, namely, CausalVAE, in disentangled representation learning, particularly in scenarios with limited labeled data. Notably, CDRM can generate counterfactual data in response to missing data, further enhancing its utility in machine learning applications. The source code and data are available at https://github.com/Causal-Disentangled/CDRM.

Toward multiloop local renormalization within causal loop-tree duality

Renormalization is a well-known technique to get rid of ultraviolet (UV) singularities. When relying on dimensional regularization, these become manifest as ε poles, allowing one to define counterterms with useful recursive properties. However, this procedure requires one to work at “integral level” and poses difficulties to achieve a smooth combination with seminumerical approaches. This article is devoted to the development of an integrand-level renormalization formalism, better suited for semi- or fully numerical calculations. Starting from the loop-tree duality, we keep the causal representations of the integrands of multiloop Feynman diagrams and explore their UV behavior. Then, we propose a strategy that allows one to build local counterterms, capable of rendering the expressions integrable in the high-energy limit and in four space-time dimensions. Our procedure was tested on diagrams up to three loops, and we found a remarkably smooth cancellation of divergences. The results of this work constitute a powerful step toward a fully local renormalization framework in quantum field theory. Published by the American Physical Society 2024

Disentangled causal representation learning for debiasing recommendation with uniform data

Disentangled causal representation learning for debiasing recommendation with uniform data

A Causality-Aware Perspective on Domain Generalization via Domain Intervention

Most mainstream statistical models will achieve poor performance in Out-Of-Distribution (OOD) generalization. This is because these models tend to learn the spurious correlation between data and will collapse when the domain shift exists. If we want artificial intelligence (AI) to make great strides in real life, the current focus needs to be shifted to the OOD problem of deep learning models to explore the generalization ability under unknown environments. Domain generalization (DG) focusing on OOD generalization is proposed, which is able to transfer the knowledge extracted from multiple source domains to the unseen target domain. We are inspired by intuitive thinking about human intelligence relying on causality. Unlike relying on plain probability correlations, we apply a novel causal perspective to DG, which can improve the OOD generalization ability of the trained model by mining the invariant causal mechanism. Firstly, we construct the inclusive causal graph for most DG tasks through stepwise causal analysis based on the data generation process in the natural environment and introduce the reasonable Structural Causal Model (SCM). Secondly, based on counterfactual inference, causal semantic representation learning with domain intervention (CSRDN) is proposed to train a robust model. In this regard, we generate counterfactual representations for different domain interventions, which can help the model learn causal semantics and develop generalization capacity. At the same time, we seek the Pareto optimal solution in the optimization process based on the loss function to obtain a more advanced training model. Extensive experimental results of Rotated MNIST and PACS as well as VLCS datasets verify the effectiveness of the proposed CSRDN. The proposed method can integrate causal inference into domain generalization by enhancing interpretability and applicability and brings a boost to challenging OOD generalization problems.

Bipartite networks represent causality better than simple networks: evidence, algorithms, and applications.

A network, whose nodes are genes and whose directed edges represent positive or negative influences of a regulatory gene and its targets, is often used as a representation of causality. To infer a network, researchers often develop a machine learning model and then evaluate the model based on its match with experimentally verified "gold standard" edges. The desired result of such a model is a network that may extend the gold standard edges. Since networks are a form of visual representation, one can compare their utility with architectural or machine blueprints. Blueprints are clearly useful because they provide precise guidance to builders in construction. If the primary role of gene regulatory networks is to characterize causality, then such networks should be good tools of prediction because prediction is the actionable benefit of knowing causality. But are they? In this paper, we compare prediction quality based on "gold standard" regulatory edges from previous experimental work with non-linear models inferred from time series data across four different species. We show that the same non-linear machine learning models have better predictive performance, with improvements from 5.3% to 25.3% in terms of the reduction in the root mean square error (RMSE) compared with the same models based on the gold standard edges. Having established that networks fail to characterize causality properly, we suggest that causality research should focus on four goals: (i) predictive accuracy; (ii) a parsimonious enumeration of predictive regulatory genes for each target gene g; (iii) the identification of disjoint sets of predictive regulatory genes for each target g of roughly equal accuracy; and (iv) the construction of a bipartite network (whose node types are genes and models) representation of causality. We provide algorithms for all goals.

JITL-MBN: A Real-Time Causality Representation Learning for Sensor Fault Diagnosis of Traction Drive System in High-Speed Trains.

A traction drive system (TDS) in high-speed trains is composed of various modules including rectifier, intermediate dc link, inverter, and others; the sensor fault of one module will lead to abnormal measurement of sensor in other modules. At the same time, the fault diagnosis methods based on single-operating condition are unsuitable to the TDS under multi-operating conditions, because a fault appears various in different conditions. To this end, a real-time causality representation learning based on just-in-time learning (JITL) and modular Bayesian network (MBN) is proposed to diagnose its sensor faults. In specific, the proposed method tracks the change of operating conditions and learns potential features in real time by JITL. Then, the MBN learns causality representation between faults and features to diagnose sensor faults. Due to the reduction of the nodes number, the MBN alleviates the problem of slow real-time modeling speed. To verity the effectiveness of the proposed method, experiments are carried out. The results show that the proposed method has the best performance than several traditional methods in the term of fault diagnosis accuracy.

Safety-Aware Causal Representation for Trustworthy Offline Reinforcement Learning in Autonomous Driving

Safety-Aware Causal Representation for Trustworthy Offline Reinforcement Learning in Autonomous Driving

Unitary (semi)causal quantum-circuit representation of black hole evaporation

A general structure of unitary evolution (evaporation) of the black hole, respecting causality imposed by the event horizon (semicausality), has been derived and presented in the language of quantum circuits. The resulting consequences for the evolution of the corresponding entanglement entropy and the entropy curve have been determined. As an illustration of the general scheme, two families of qubit toy models have been discussed: tensor product models and controlled non-product models.

Granger causal representation learning for groups of time series

Granger causal representation learning for groups of time series