GaitSCM: Causal representation learning for gait recognition

Abstract

Gait recognition is a promising biometric technology that aims to identify the target subject via walking pattern. Most existing appearance-based methods focus on learning discriminative spatio-temporal representations from gait silhouettes. However, these methods pay less attention to probing the causality between identity factors and identity labels, which often mislead the model to learn gait representations that are susceptible to identity-irrelevant factors. In this paper, we attribute the cause that leads to the decline of model generalization under different external conditions to identity-irrelevant factors. We formulate the causalities among the identity factors, identity-irrelevant factors, and identity labels as a structural causal model (SCM). We accordingly propose a novel gait recognition framework named GaitSCM to learn covariate invariant gait representations, which is mainly composed of three components, including feature extraction module, feature disentanglement module, and backdoor adjustment. Specifically, we design a feature extractor with regard to the movement patterns of different body parts to learn fine-grained gait motion features, and then present a two-branch feature decoupling module to disentangle identity features and identity-irrelevant features with the aid of the classification confusion loss. To relieve the negative effect of identity-irrelevant factors, we develop a backdoor adjustment strategy to eliminate spurious associations between identity and identity-irrelevant features, which further facilitates the proposed framework to generate more powerful identity representations. Extensive experiments conducted on two public datasets validate the effectiveness of our method. The average Rank-1 can reach 93.2% and 90.4% on CASIA-B and OU-MVLP datasets, respectively, which verifies the superiority of GaitSCM. Source code is released at: https://github.com/HuoweiCode/GaitSCM.