3D Reconstruction Method Using A Pendulum RGB-D Camera

Abstract

3D models are used in various fields to reproduce the real environment in a virtual world, such as assessing conditions at construction sites, sharing blueprints and other data, and disaster experiences using Virtual Reality. Various methods have been developed to create 3D models of real environments, including methods using laser scanners and cameras. The method using a laser scanner can create an accurate 3D model, but the equipment is expensive and not easy to use. On the other hand, the method of using a camera is less accurate than a laser scanner, but it is less expensive and easier to use. In the camera-based method, the pose, which encompasses the 3D coordinates of the camera and its orientation at the moment the image was acquired, is estimated through image processing. Then, a 3D model can be created based on the estimated camera poses. In this method, it is difficult to estimate the camera poses from images with few image features. Therefore, in regions with few features, such as flat walls of a single color, it is difficult to reconstruct the 3D model.In this study, camera poses are estimated using the camera moving as a pendulum and using these poses to create a 3D model. The camera is attached at the end of a pendulum so that the camera motion can be physically modeled, and the trajectory of the camera can be accurately estimated. In this method, the environment is first photographed using a pendulum camera in which an RGB-D camera is used for the weight of the pendulum. Then, tracking is performed to estimate the part of the pendulum motion using clear images that represents the area with enough features. Next, the fulcrum position and length of the pendulum are estimated using the tracking results. The fulcrum position and the length are then used to estimate the camera poses for all images by the pendulum physical model. Finally, the camera poses are used to perform a 3D reconstruction of the captured environment. To evaluate the effectiveness of this method, the 3D models created with and without the proposed method were compared from two perspectives: the size of the area that could be reconstructed and the accuracy of the 3D models created. As a result, the proposed method was able to reconstruct the 3D model even in regions where enough features cannot be obtained from the images. However, the model created with the proposed method displays the subject shifted from frame to frame compared to the model created without the proposed method. This is supposed to be due to the fact that the calculated camera poses were not accurate enough because the estimated length of the pendulum included an error. Future work is needed to further improve the accuracy of the camera pose estimation in order to improve the accuracy of the model. Other possibilities include extending the shooting range by conducting cooperative photography with multiple cameras.