Discovering causally invariant features for out-of-distribution generalization

Abstract

Out-of-distribution (OOD) generalization aims to generalize a model trained on source domains to unseen target domains. Recently, causality-based generalization methods have focused on learning invariant causal relationships around the label variable, as causal mechanisms are robust across different domains. However, these methods would yield an inaccurate causal variable set due to the lack of heterogeneous domain data or a prior causal structure, which severely weakens their generalization capacity. To address this problem, we propose a Causally Invariant Features Discovery (CIFD) framework, which combines causal structure discovery and causal effect estimation for selecting a high-quality causal variable set and realizing better OOD generalization. Specifically, CIFD first identifies all potential causal variables by learning a double-layer-based local causal structure around the label variable. Secondly, CIFD uses a double-layer causal effect estimator for estimating the causality of potential causal variables and obtaining true causal variables. The comprehensive experiments on both regression and classification tasks clearly demonstrate the superiority of our framework over the state-of-art methods.