A Deep Generative Adversarial Network (GAN)-enabled Abnormal Pedestrian Behavior Detection at Grade Crossings

Abstract

This paper presents a video processing pipeline based on a deep learning generative adversarial network (GAN) for detecting and localizing abnormal behaviors of pedestrians at grade crossings. The skeleton detection and tracking algorithm captures pedestrian motion patterns of pedestrians and generates temporally-varying trajectories of posture key points for each pedestrian. The trajectories of only normal behaviors are learned by the GAN after being decomposed into local and global motion features. During the testing stage, anomalous behaviors can be readily observed as outliers of the learned features and trained models. Our pipeline can handle multiple pedestrians simultaneously presenting in a video frame by running a single model. It requires no specific location information or (minimal) model retraining, enhancing its robustness and extendibility to different grade crossings. In addition, the entire pipeline is also developed on an edge computing platform to enable salient field deployability of the technology. The experimental results demonstrate the prominent performance of the system in field tests.