Feature reconstruction graph convolutional network for skeleton-based action recognition

Abstract

Skeleton-based action recognition is an important task in computer vision. Recently, graph convolutional networks (GCNs) have been successfully applied to this task and achieved remarkable results. However, there are still some non-negligible limitations with existing GCN-based methods. First, the artificial predefined skeleton partition lacks the joint modeling for different types of edges. Second, most GCN models use interleaved deployment of spatial-only and temporal-only modules to achieve feature learning, which makes them ineffective in capturing spatiotemporal co-occurrence from action sequences. To tackle the above issues, we propose a novel feature reconstruction graph convolutional network (FR-GCN) for skeleton-based action recognition. The proposed FR-GC combines coarse-grained temporal and spatial features to reconstruct fine-grained spatiotemporal features, realizing simultaneous learning of temporal and spatial representations in a single module and significantly improving the capability of the model for spatiotemporal feature extraction. We also propose a topology partition enhancement module to achieve adaptive complementation among different types of edges. Moreover, an efficient multi-scale dual-domain temporal convolution is used to complete further temporal modeling. Compared with state-of-the-art methods, the proposed FR-GCN achieves competitive results on both NTU RGB+D 60 dataset and NTU RGB+D 120 dataset. Especially under the cross-subject benchmarks of the two datasets, the proposed FR-GCN achieves new state-of-the-art performance.