A Vision-Based System for Abnormal Behavior Detection and Recognition of Bus Passengers

Abstract

The passenger's safety issue is an emerging problem for autonomous driving, especially for the public transport vehicles. In the unmanned operating environment, it is necessary to be aware of improper and dangerous activities onboard the vehicle. In this paper, we propose a vision system for abnormal passenger behavior detection and recognition in bus application scenarios. Different from the current human activity recognition approaches, the images are acquired from an overhead camera to provide a large viewing area with less occlusion. Hence, it is a more challenging task for feature extraction and classification. An action recognition network for top-view images is proposed with the consideration of spatial and temporal information. For detection and classification of dangerous passenger behavior, a new image dataset, BUS-HAR, is created. Experiments carried out with the real scene images have demonstrated the feasibility of our method compared to the existing approaches. The codes and image dataset are made available publicly at https://github.com/richardkuo1999/passenger-action-recognition.