Skeleton-Based Pedestrian Abnormal Behavior Detection with Spatio-Temporal Model in Public Places

Abstract

In computer vision, there is growing interest in the recognition of pedestrian abnormal behaviors. The abnormal behavior of a person could be the sign of some dangerous activities. However, it’s still challenging to extract the discriminative spatial and temporal features effectively faced with video data. In this paper, we propose skeleton-based pedestrian abnormal behavior detection models. The base model is consisting of ResNet as a spatial feature extractor, LSTM as a global temporal feature extractor, and the ResNet network that use the dual-stream network to extract local temporal features. The proposed model is an improvement of all ResNet into Conv1x1_ResNet, and added a layer of Conv1x1_ResNet after dual-stream Conv1x1_ResNet to extract more accurate global space features. The proposed model achieved the highest accuracy of 89.29%, and the averaged get batch time is 0.3399 ms. The base model achieved 88.12% accuracy, and the averaged get batch time is 0.3174 ms less than the time taken by other models. Both models reach speed of 80 frames/sec. Compared with the models made in previous work, the base model has the shortest training time, and the proposed model provides the highest accuracy in the field of pedestrian detection.