Body2vec: 3D Point Cloud Reconstruction for Precise Anthropometry with Handheld Devices.

Magda Alexandra Trujillo-Jiménez,Soledad De Azevedo,Orlando Pérez,Claudio Delrieux,Pablo Navarro,Anahí Ruderman,Virginia Ramallo,Bruno Pazos,Leonardo Morales,Carolina Paschetta,Rolando Gonzalez-José

doi:10.3390/jimaging6090094

Abstract

Current point cloud extraction methods based on photogrammetry generate large amounts of spurious detections that hamper useful 3D mesh reconstructions or, even worse, the possibility of adequate measurements. Moreover, noise removal methods for point clouds are complex, slow and incapable to cope with semantic noise. In this work, we present body2vec, a model-based body segmentation tool that uses a specifically trained Neural Network architecture. Body2vec is capable to perform human body point cloud reconstruction from videos taken on hand-held devices (smartphones or tablets), achieving high quality anthropometric measurements. The main contribution of the proposed workflow is to perform a background removal step, thus avoiding the spurious points generation that is usual in photogrammetric reconstruction. A group of 60 persons were taped with a smartphone, and the corresponding point clouds were obtained automatically with standard photogrammetric methods. We used as a 3D silver standard the clean meshes obtained at the same time with LiDAR sensors post-processed and noise-filtered by expert anthropological biologists. Finally, we used as gold standard anthropometric measurements of the waist and hip of the same people, taken by expert anthropometrists. Applying our method to the raw videos significantly enhanced the quality of the results of the point cloud as compared with the LiDAR-based mesh, and of the anthropometric measurements as compared with the actual hip and waist perimeter measured by the anthropometrists. In both contexts, the resulting quality of body2vec is equivalent to the LiDAR reconstruction.

Highlights

Reconstruction of 3D objects is among the many potential applications of Computer Vision models working together with Deep Learning techniques
Removal Network), a human body identification and segmentation model that is applied for video background removal; (iii) we generate structure from motion (SfM) point clouds from the raw and clean videos, generate a registration thereof to the meshed LiDAR acquisition, and measure the respecive registration errors; and (iv) we evaluate anthropometric measurements from the LiDAR mesh, and the raw and clean point clouds, and compare them to the measurements performed by anthropometrists
We developed BRemNet, a further refinement of Mask R-CNN with the aim of pre-processing per frame the videos taken for photogrammetric 3D body reconstruction

Summary

Introduction

Reconstruction of 3D objects is among the many potential applications of Computer Vision models working together with Deep Learning techniques This combined approach can contribute to solve common problems regarding the analysis of human body shape. J. Imaging 2020, 6, 94 measurements (i.e., the shape, form, size, and several perimeters and volumes of the human body) [1,2,3]. Imaging 2020, 6, 94 measurements (i.e., the shape, form, size, and several perimeters and volumes of the human body) [1,2,3] This is the case in many clinical applications ranging from diagnostic, treatment, and follow-up of overweight-related conditions, to less frequent but important skeletal pathologies, such as scoliosis [4].

Methods

Results

Conclusion