Smart-VPoseNet: 3D human pose estimation models and methods based on multi-view discriminant network

Abstract

In the multi-view 3D human pose estimation, when there are some poor-quality views in multi-view data, these views will cause serious errors to multi-view 3D pose estimation. We also found many similar problems in the single-view model. For example, in a motion sequence of the current view, there are some frames in which the human body are occluded or the body joints exceed the camera’s field of view. Therefore, we hope to implement an intelligent method to make the model jump in multiple views during 3D pose modeling, so as to avoid the model using the views that have serious problems and pay more attention to higher quality views. We call this method Smart-VPoseNet. We defined three view selection rules to explain the quality of each view in the dataset, and designed a novel view discriminant network. According to the three rules, the data distribution and characteristics in the dataset are visualized for analyzing and designing the details of the model. In addition, we also introduced a hybrid view selection method based on these rules. In the experiment, we used a single-view pretrained model and Smart-VPoseNet to complete a series of comprehensive comparison experiments. These experiments illustrate that the work of this article has a novel and feasible optimization theory and show the application effect of our work in 3D pose estimation. We believe that our work has opened up a new optimization model based on multiple views.