New multi‐view human motion capture framework

Abstract

Estimating human pose and shape without markers is a challenging problem. This study proposes a multiple-view markerless human motion capture framework. Firstly, a multi-view camera system is built for capturing real-time images of moving humans on multiple views. Secondly, by employing the OpenPose method, the authors calculate robust 3D key points from 2D key points of the human body, which are estimated from the multi-view images. And dense 3D point cloud is reconstructed from images. Thirdly, they propose a novel SMPL-based method to represent human motion by fitting the SMPL model to 3D key points and 3D point clouds. In order to achieve a more accurate human pose, a penalty term is utilised to solve the problem of error accumulation in the process of human motion capture. In addition, they present a dense mesh template-based SMPL that can be deformed to point cloud to recover a real human body shape. Finally, they map multi-view colour images onto the human mesh model to acquire rendered mesh. The experimental results show that the proposed method improves the accuracy of human pose and realises the 3D human body model more realistic.