Human Skeleton Control with the Face Expression Changed Synchronously

Abstract

In this paper, we proposed a method of real-time human motion detection and skeleton control based on the depth image. For the human body part, we use Kinect to detect the original three-dimensional coordinates of the human joint; then smooth the joint movement; calculate the amount of rotation based on the smoothed data. Finally, the FBX file is used to drive the virtual mannequin. For the human face part, we use 3D modeling software to modify the FBX model and replace the head of the human model. With the help of the Kinect device, we combined the idea of linear blend skinning (LBS) to make the human body action effect more realistic. The experimental results show that the motion capture system can better recover the 3D skeleton motion of the captured real human body. Compared with other motion capture system, the motion control time of the human body is greatly reduced, and the 3D mesh and skeleton can be obtained immediately according to the detected human body motion. The modified FBX model can apply specific expression effects.