Skeleton-based abnormal gait recognition with spatio-temporal attention enhanced gait-structural graph convolutional networks

Abstract

Abnormal gait recognition is of great significance for medical monitoring and clinical diagnosis. Recent progress on skeleton-based abnormal gait recognition using recurrent neural networks (RNNs) and temporal-only convolutional networks (TCNs) has been substantial. These methods usually rely on hand-crafted features or treat the skeleton as a kind of grid-shape structure data, thus resulting in limited representation and difficulties of generalization. To solve this problem, we propose a spatio-temporal attention enhanced gait-structural graph convolutional network (AGS-GCN). First, we construct a gait skeleton graph according to clinical prior knowledges and reliability of deep sensors in the gait analysis. A novel partition strategy is designed for the gait graph to simultaneously extract multi-scale gait features from the raw skeleton data. Moreover, in order to extract discriminative gait representations, a spatio-temporal attention mechanism is proposed to layer-wise enhance the features of key joints. The soft attention mechanism can boost the ability to explore fine-grained gait features, and alleviate the over-smoothing of feature maps in deep graph convolutional networks. Extensive experiments on two abnormal gait datasets with different numbers of gait patterns and examples demonstrate that our system achieves better performance than the state-of-the-art works.