Abstract
This paper presents a sequential non-rigid reconstruction method that recovers the 3D shape and the camera pose of a deforming object from a video sequence and a previous shape model of the object. We take PTAM (Parallel Mapping and Tracking), a state-of-the-art sequential real-time SfM (Structure-from-Motion) engine, and we upgrade it to solve non-rigid reconstruction. Our method provides a good trade-off between processing time and reconstruction error without the need for specific processing hardware, such as GPUs. We improve the original PTAM matching by using descriptor-based features, as well as smoothness priors to better constrain the 3D error. This paper works with perspective projection and deals with outliers and missing data. We evaluate the tracking algorithm performance through different tests over several datasets of non-rigid deforming objects. Our method achieves state-of-the-art accuracy and can be used as a real-time method suitable for being embedded in portable devices.
Highlights
The problem of 3D reconstruction and camera localization from images is known as Structure-from-Motion (SfM). 3D awareness from visual cues is a natural task for a human being, but it is still a very challenging problem in computer vision
This paper proposes a sequential solution to Shape from Template (SfT) able to be run in real time (e.g., 15–30 frames per second) using a low-cost hardware based on a CPU
After a sequence is processed by the algorithm, the output results are post-processed to compare the performance of the algorithm using the ground truth
Summary
The problem of 3D reconstruction and camera localization from images is known as Structure-from-Motion (SfM). 3D awareness from visual cues is a natural task for a human being, but it is still a very challenging problem in computer vision. The problem of 3D reconstruction and camera localization from images is known as Structure-from-Motion (SfM). During the last few decades, SfM has been widely studied [1,2,3,4,5,6,7]. The general assumption in SfM is based on the rigidity of the scene, where changes in the images are caused by the relative motion between the camera and the scene. Rigidity strongly links camera motion with image motion, making SfM a well-posed problem. Rigid SfM fails in scenarios where the rigidity assumption is violated. It fails to reconstruct scenes with multiple objects that move independently or with deformable objects, such as the human body, articulated objects, wires, flags, sheets, flesh, fabric, etc
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
