Robust 3D street-view reconstruction using sky motion estimation

Abstract

We introduce a robust 3D reconstruction system that uses a combination of the structure-from-motion (SfM) filter and the bundle adjustment. The local bundle adjustment provides an initial depth of a newly introduced feature to the SfM filter, and the filter enables to predict the motion of the camera while performing the reconstruction process. In addition, we increase the robustness of the rotation estimation by estimating the motion of the sky from cylindrical panoramas of street views. The sky region is segmented by a robust estimating algorithm based on a translational motion model in the cylindrical panoramas. We show that the combination of the SfM filter and the bundle adjustment with sky motion estimation algorithms produces a robust 3D reconstruction from the street view images, compared to running each method separately.