Human Model Reconstruction Research Articles

Enhanced Multi-Scale Attention-Driven 3D Human Reconstruction from Single Image

Due to the inherent limitations of a single viewpoint, reconstructing 3D human meshes from a single image has long been a challenging task. While deep learning networks enable us to approximate the shape of unseen sides, capturing the texture details of the non-visible side remains difficult with just one image. Traditional methods utilize Generative Adversarial Networks (GANs) to predict the normal maps of the non-visible side, thereby inferring detailed textures and wrinkles on the model’s surface. However, we have identified challenges with existing normal prediction networks when dealing with complex scenes, such as a lack of focus on local features and insufficient modeling of spatial relationships.To address these challenges, we introduce EMAR—Enhanced Multi-scale Attention-Driven Single-Image 3D Human Reconstruction. This approach incorporates a novel Enhanced Multi-Scale Attention (EMSA) mechanism, which excels at capturing intricate features and global relationships in complex scenes. EMSA surpasses traditional single-scale attention mechanisms by adaptively adjusting the weights between features, enabling the network to more effectively leverage information across various scales. Furthermore, we have improved the feature fusion method to better integrate representations from different scales. This enhanced feature fusion allows the network to more comprehensively understand both fine details and global structures within the image. Finally, we have designed a hybrid loss function tailored to the introduced attention mechanism and feature fusion method, optimizing the network’s training process and enhancing the quality of reconstruction results. Our network demonstrates significant improvements in performance for 3D human model reconstruction. Experimental results show that our method exhibits greater robustness to challenging poses compared to traditional single-scale approaches.

3-D Clothed Human Model Reconstruction Based on Single-View In-the-Wild Image Data

3-D Clothed Human Model Reconstruction Based on Single-View In-the-Wild Image Data

Implementation of Computer Aided Dance Teaching Integrating Human Model Reconstruction Technology

Implementation of Computer Aided Dance Teaching Integrating Human Model Reconstruction Technology

3D Information Guided Motion Transfer via Sequential Image Based Human Model Refinement and Face-Attention GAN

Image and video based human motions can be regarded as the deformation processes of person appearances, so motion transfer is usually treated as a pose guided image generation task and implemented in the 2D image plane. However, the 2D plane image generation lacks guidance of the original 3D motion information, which results in blur and shape distortions of the generated motion images. Therefore, we propose to simulate the generation process of real motion images by projecting the 3D human models, which are reconstructed from the training motion images and driven with target poses, into the 2D plane. We then take the 2D projections as the pose representations and input them into the generation model as they naturally inherit the 3D information from the original motions. Considering the unreliability on the invisible surface of the single image based human model reconstruction, we propose a sequential image based human model refinement module which exploits the complementary information between adjacent motion frames to refine the 3D human model. Furthermore, we propose a face-attention GAN model to conduct the final motion transfer, in which we use the Gaussian distribution to match the elliptical face region and design a face enhancement loss function since the faces in the generated motion images influence the performances very much. The generated motion images with reliable depth information, accurate shapes and clear faces demonstrate the effectiveness of the proposed method.

Intelligent Clothing Design and Data Acquisition under the Scientific Graphics Programming Combined with Wearable Multisensor

This research focuses on the improvement of intelligent Hanfu design system’s performance. Henceforth, in the current study, an intelligent Hanfu design data acquisition system based on scientific graphics programming and wearable multisensor is designed for the field of human body size measurement and human model reconstruction. Initially, on the basis of wearable multisensor, a design of the recognition and analysis system of human posture is presented. The error of nine-axis inertial sensor in the process of collecting data, the data of accelerometer, gyroscope, and magnetometer are preprocessed, and the processing results are observed. Moreover, MATLAB programming is used for preprocessing of collected images. It is also used for grey normalization, filtering denoising, and image sharpening. The advantages and disadvantages of different methods are compared experimentally. Experiments show that MATLAB programming is more suitable for grey normalization than histogram equalization, and local brightness will appear in histogram equalization. After denoising the image with salt-and-pepper noise and Gaussian noise, the median filter and mean filter have defects. Wiener’s adaptive filter leads to the increase of noise, and wavelet denoising has the best effect. The data acquisition system of intelligent Hanfu design established in this work provides a certain direction for the development of intelligent Hanfu design.

DeepProfile: Accurate Under-the-Clothes Body Profile Estimation

Accurate human body profiles have many potential applications. Image-based human body profile estimation can be regarded as a fine-grained semantic segmentation problem, which is typically used to locate objects and boundaries in images. However, existing image segmentation methods, such as human parsing, require significant amounts of annotation and their datasets consider clothes as part of the human body profile. Therefore, the results they generate are not accurate when the human subject is dressed in loose-fitting clothing. In this paper, we created and labeled an under-the-clothes human body contour keypoint dataset; we utilized a convolutional neural network (CNN) to extract the contour keypoints, then combined them with a body profile database to generate under-the-clothes profiles. In order to improve the precision of keypoint detection, we propose a short-skip multi-scale dense (SMSD) block in the CNN to keep the details of the image and increase the information flow among different layers. Extensive experiments were conducted to show the effectiveness of our method. We demonstrate that our method achieved better results—especially when the person was dressed in loose-fitting clothes—than and competitive quantitative performance compared to state-of-the-art methods, while requiring less annotation effort. We also extended our method to the applications of 3D human model reconstruction and body size measurement.

Experimental Study of a Deep-Learning RGB-D Tracker for Virtual Remote Human Model Reconstruction.

Tracking movements of the body in a natural living environment of a person is a challenging undertaking. Such tracking information can be used as a part of detecting any onsets of anomalies in movement patterns or as a part of a remote monitoring environment. The tracking information can be mapped and visualized using a virtual avatar model of the tracked person. This paper presents an initial novel experimental study of using a commercially available deep-learning body tracking system based on an RGB-D sensor for virtual human model reconstruction. We carried out our study in an indoor environment under natural conditions. To study the performance of the tracker, we experimentally study the output of the tracker which is in the form of a skeleton (stick-figure) data structure under several conditions in order to observe its robustness and identify its drawbacks. In addition, we show and study how the generic model can be mapped for virtual human model reconstruction. It was found that the deep-learning tracking approach using an RGB-D sensor is susceptible to various environmental factors which result in the absence and presence of noise in estimating the resulting locations of skeleton joints. This as a result introduces challenges for further virtual model reconstruction. We present an initial approach for compensating for such noise resulting in a better temporal variation of the joint coordinates in the captured skeleton data. We explored how the extracted joint position information of the skeleton data can be used as a part of the virtual human model reconstruction.

Seamless images stitching for 3D human models.

Realistic 3D human model reconstruction is an important component in computer graphics and computer vision. In particular, texturing on the surface of models is a key stage of reconstruction. In this paper, we dispose the texture mapping on the model’s surface as an optimization of image stitching, and present an effective method to generate a seamless, integrated and smooth texture on the surface of 3D human model. First, we build a corresponding Markov Random Field model with respect to color images and triangular meshes of the surface. On the basis of alpha-expansion optimization for this Markov Random Field model, a 2D translation coordinate of color image, as an adaptive iterative factor, is introduced into the optimization to match the color content at the boundary of adjacent meshes. That compensates for the misalignment of adjacent color images, which caused by the inaccuracy of depth data and multi-view misregistration. Then we apply Poisson blending to a composite vector field in gradient domain, to resolve the small but noticeable illumination variations between different color images. To repair the blank regions, we parameterize the model’s surface and project it onto a 2D plane. Then the K-Nearest Neighbor algorithm is applied to fill up the blank regions with texture contents. Finally, we evaluate our method by comparison with another three advanced methods on some human models, and the results demonstrate that our method of images stitching creates a best texture on the surface of 3D human model both in visual effect and quantitative analysis.

Optimizing human model reconstruction from RGB-D images based on skin detection

This paper reconstructs human model from multi-view RGB-D images of an Xbox One Kinect. We preprocess the depth images by implicit surface de-noising and then part-wisely register them into a point cloud. A template model is selected from the human model database to fit the registered point cloud of a human body by Laplacian deformation. Skin detection of RGB-D images helps to tightly constrain the skin parts of human body in template fitting step in order to get more precise and lifelike human model. We propose a robust skin detection method that is not affected by clothing pattern and background. Experiments demonstrate the effectiveness of our method.

3D Human Model Reconstruction from Sparse Uncalibrated Views.

Using a two-stage algorithm, the proposed technique can tackle the challenges of reconstructing high-quality 3D models of humans wearing regular clothes from sparse uncalibrated cameras. The proposed algorithm based on nonrigid dense correspondences (NRDC) requires fewer images than previous methods because it does not require an initial sparse matching. The authors validated the proposed algorithm using images from an existing dataset and images captured by a cell phone camera.

Reconstructing 3D human models with a Kinect

AbstractThree‐dimensional human model reconstruction has wide applications due to the rapid development of computer vision. The appearance of cheap depth camera, such as Kinect, opens up new horizons for home‐oriented 3D human reconstructions. However, the resolution of Kinect is relatively low, making it difficult to build accurate human models. In this paper, we improve the accuracy of human model reconstruction from two aspects. First, we improve the depth data quality by registering the depth images captured from multi‐views with a single Kinect. The part‐wise registration method and implicit‐surface‐based de‐noising method are proposed. Second, we utilize a statistical human model to iteratively augment and complete the human body information by fitting the statistical human model to the registered depth image. Experimental results and several applications demonstrate the applicability and quality of our system, which can be potentially used in virtual try‐on systems. Copyright © 2015 John Wiley & Sons, Ltd.

Depth video based human model reconstruction resolving self-occlusion

Generation of a 3D model has been actively researched in computer vision area. Especially, modeling of a realistic human is a topic of great interest due to its use in many application areas such as animation and 3DTV. In this paper, a novel method is presented to generate a 3D human model using a sequence of depth images from a monocular TOF (time-of-flight) camera. Mainly, we solve a 'self-occlusion' problem which occurs in the depth data where a human model is partially occluded by other parts of a human body. This becomes very critical when we want to develop a human model for various applications. We demonstrate the capability of our method by showing experimental results that resolves a multiple occlusion problem.

Low-cost 3D human reconstruction from multi-image

A method for low-cost 3D specific human model reconstruction from multi-image was proposed.First,the feature points on each image were matched with their generic model.Then a deformation from radial basic function was performed to obtain specific model.After that,three kinds of filters were used to refine the specific model.Finally the surface of model was texture mapped by viewpoint-independent algorithm,and skin-skeleton system by vertex blending algorithm as well.The experimental results show that this method is accurate,easy to implement and of low-cost.