Abstract
In this paper, we present a novel approach for reconstructing 3D geometry from a stream of images captured by a consumer-grade mobile RGB-D sensor. In contrast to previous real-time online approaches that process each incoming image in acquisition order, we show that applying a carefully selected order of (possibly a subset of) frames for pose estimation enables the performance of robust 3D reconstruction while automatically filtering out error-prone images. Our algorithm first organizes the input frames into a weighted graph called the similarity graph. A maximum spanning tree is then found in the graph, and its traversal determines the frames and their processing order. The basic algorithm is then extended by locally repairing the original spanning tree and merging disconnected tree components, if they exist, as much as possible, enhancing the result of 3D reconstruction. The capability of our method to generate a less error-prone stream from an input RGB-D stream may also be effectively combined with more sophisticated state-of-the-art techniques, which further increases their effectiveness in 3D reconstruction.
Highlights
The reconstruction of 3D worlds from 2D images has been a fundamental challenge in computer graphics and vision for decades
Its importance is apparent in the fields of virtual, augmented, and mixed reality where real geometry must be naturally mingled with virtual geometry
To demonstrate its effectiveness and applicability, the presented similarity graph scheme was tested with several RGB-D sequences, where all the test datasets, including those shown in the previous sections, were produced by storing into files the live RGB-D streams of 320 × 180 pixels, captured using a Lenovo Phab 2 Pro smartphone
Summary
The reconstruction of 3D worlds from 2D images has been a fundamental challenge in computer graphics and vision for decades. If the real-time online camera tracking is not mandatory, there is no need to process the RGB-D stream in its given order or for all frames to participate in reproducing the 3D geometry despite any possible defects in the images. In addition to the generation of an effective set of input frames and their registration order, the presented similarity graph scheme automatically removes from consideration those frames that may introduce intolerable errors in the results of pose estimation and 3D reconstruction. This may disconnect the similarity graph, resulting in multiple separate components. This does not mean that the presented similarity graph technique is orthogonal to these sophisticated mechanisms because our method may be combined with them to enhance their effectiveness by providing them with fewer error-inducing input streams
Sign-in/Register to view highlights & summary 
Published Version (
Free)
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call