Dynamic manifold Boltzmann optimization based on self‐supervised learning for human motion estimation

Abstract

It is a challenge work to estimate the 3D human motion from image sequence. There are some problems, such as unsatisfactory estimation error, ambiguous matching and transient occlusion. Although the prior information of learning large-scale samples exists, these problems are still difficult to be solved. How to extract the feature of the high-dimensional (HD) sample of 3D human motion and find the desired one will become the key to solve these problems above. Some dimension reduction methods can extract the sample features and build the low-dimensional (LD) space to view their LD features, but how to search the relevant valid and desired LD samples remains the bottleneck problem, which can be used to reconstruct the 3D human motions denoted by the corresponding high-dimensional samples. Thus, a new method called dynamic manifold Boltzmann optimization (DMBO) is proposed to estimate the 3D human motion from multi-view images. DMBO can find the best matching 3D human motion model by the help of the self-supervised learning from Gaussian incremental dimension reduction model (GIDRM). DMBO can avoid the local optimum during searching and solve the problems above, so that the generation of the accurate 3D human motion corresponding to multi-view images can be achieved.