A Portable System for ToF Camera based Human Body Detection and Pose Estimation

Abstract

In recent years, human motion detection and analysis has been a research hotspot in the field of machine vision. It has been widely used in sports fitness, sports rehabilitation, sports analysis and virtual reality. Most of the existing human motion detection systems are based on optical equipment using retroreflective markers, which have many defects such as high deployment cost and complex operation. To this end, this article develops a household fitness motion evaluation system based on ToF camera with a mobile phone. The system firstly uses the ToF camera to collect the 3D human body point cloud information. Then Huawei augmented reality engine is employed to track the human skeleton. As all the spatial coordinates of joints are acquired, virtual human model can be reconstructed remotely on the phone. The mobile phone, working as the edge node between the person and the cloud server, transmits the captured human motion model parameters to the cloud platform. Then a hierarchical co-occurrence neural network recognizes and scores the detected user actions online. The experimental results show that the system proposed in this paper is portable and inexpensive. Real-time capabilities and measurement accuracy can also satisfy the requirements of evaluation for physical movements in sports rehabilitation.