3D Scanning of High Dynamic Scenes Using an RGB-D Sensor and an IMU on a Mobile Device

Abstract

With the development of RGB-D sensors and mobile devices, 3D scanning has witnessed great progress in recent years. KinectFusion opens up an era of RGB-D 3D reconstruction, which integrates captured depth images into a voxel-based representation. A number of improvements have been applied to the KinectFusion to reduce large footprint and make the 3D reconstruction on mobile devices possible. However, these methods are designed to handle static scenes. In this paper, we propose a method which can perform the 3D scanning of high dynamic scenes using an RGB-D sensor and an inertial measurement unit (IMU) on a mobile device. We first introduce a novel method to segment the depth images into static and dynamic elements with the use of sparse optical flow and the rotational part of the IMU. Then, we determine the camera pose with pixels labeled as static. At last depth, images are integrated into a voxel-based representation. The truncated signed distance function values of static voxels are updated while dynamic voxels are set to free-space. The experiments show that compared with some state-of-the-art systems, our method has better results when scanning high dynamic scenes and has comparable results when scanning low dynamic and static scenes. Besides, our method processes eight frames/s on an Apple iPad Air 2.