Multi‐branch angle aware spatial temporal graph convolutional neural network for model‐based gait recognition

Abstract

AbstractModel‐based gait recognition with skeleton data input has attracted more attention in recent years. The model‐based gait recognition methods take skeletons constructed by body joints as input, which are invariant to changing carrying and clothing conditions. However, previous methods limitedly model the skeleton information in either spatial or temporal domains and ignore the pose variety under different view angles, which results in poor performance for gait recognition. To solve the above problems, we propose the Multi‐Branch Angle Aware Spatial Temporal Graph Convolutional Neural Network to better depict the spatial‐temporal relationship while minimising the interference from the view angles. The model adopts the legacy Spatial Temporal Graph Neural Network (ST‐GCN) as its backbone and relocates it to create independent ST‐GCN branches. The novel Angle Estimator module is designed to predict the skeletons' view angles, which enables the network robust to the changing views. To balance the weights of different body parts and sequence frames, we build a Part‐Frame‐Importance module to redistribute them. Our experiments on the challenging CASIA‐B dataset have proved the efficacy of the proposed method, which achieves state‐of‐the‐art performance under different carrying and clothing conditions.