Periodic Physical Activity Information Segmentation, Counting and Recognition From Video

Abstract

The research on complex human body motion including sports and workout activity recognition is a major challenge and long-lasting problem for the computer vision community. Recent development in deep learning algorithms to track people’s workout activity characteristics based on video sensors can be used to infer the human body pose for further analysis. Specifically, tracking complex body movements while performing multi-pose physical exercise helps individuals provide fine granularity feedback including activity repetition counting and activity recognition. Therefore, this research proposes a system that provides a repetition counter and activity recognition of physical exercise from video frames (extracted 3D human skeleton using VIBE) based on the deep semantic features and repetitive segmentation algorithm. The proposed system locates both ends of the activity’s action and segments the activity into multiple unit actions which improves activity recognition, time intervals, # of sets, and other quantitative values of activity. The proposed system is evaluated on the physical activities dataset named “NOL-18 Exercise” through extensive experiments. The proposed system results show that the accuracy of the repetitive action segmentation is 96.27% with 0.23% time error, and action recognition reaches 99.06%. The system can be employed to fitness or rehabilitation centers and used for treating patients.