Vision-based toddler physical activity recognition using deep learning

Abstract

Human activity recognition (HAR) is a system for understanding human movements and behaviour. It has been applied in many fields such as video surveillance, behaviour analysis, and human-computer interaction. The stateof- the-art studies on HAR generally focus their attention on public dataset which mostly consist of adults as their subjects. Research on HAR for children especially toddlers is important to facilitate their surveillance by monitoring their activities automatically. Since toddlers possess different anatomical proportions than adults, their unusual movements can be a challenge to infer. In this paper, a vision-based deep learning HAR system for toddlers was developed based on skeleton features. Videos of toddlers' activities in a day-care were obtained through different public sources. 2D skeleton data were then extracted from every frame of these videos using a pre-trained deep learning network. These skeleton data were trained on LSTM and fully connected network to infer the toddler's activities. Results showed that this proposed framework managed to achieve 75% accuracies for three toddlers' activities which are jumping, sitting, and standing.