Semantic segmentation-based system for fall detection and post-fall posture classification

Abstract

Fall is one of the most critical issues faced by elders in their daily life. The consequences of falls range from fatal injuries, severe injuries to no injuries. Therefore, an effective system for detecting falls and treating post-fall injuries is important. Unlike wearable sensors, camera-based systems seem more comfortable and flexible for daily life monitoring. However, monitoring human activities in real settings using cameras is not only challenging but also pose privacy issues. To mitigate this problem, we propose a surveillance camera-based framework for fall detection and post-fall classification where the human silhouette is extracted and used instead of raw images. The human silhouette is obtained using a pixel-level classification based on a Multi-Scale Skip Connection Segmentation Network (MSSkip), which is shown to achieve state-of-the-art performance on the validation set from PASCAL VOC 2012 dataset for the class Person with an IoU of 90%. The temporal and spatial variations of the human poses are fed to a Convolutional Long Short Term Memory (ConvLSTM) network to detect whether or not a fall has occurred. The proposed fall detection method achieved an F1-score of 97.68% on the UP Fall dataset, and state-of-art performance on the UR Fall detection database. For post-fall posture classification, the Xception network is shown to achieve an F1-score of 97.85% on our customized post-fall dataset.