3-D Metric Reconstruction and Registration of Images of Near-planar Surfaces

Abstract

In this study, we address the problem of 3-D dense metric reconstruction and registration from multiple images, given that the observed surface is nearly planar. This is difficult, as classical methods work well only if the scene is truly planar (mosaicing) or the scene has certain significant depth variations (classical Structure-from- Motion (SfM)). One domain in which this problem occurs is image analysis of the retinal fundus. Our approach is to first assume planarity, and perform 2-D global registration. A first bundle adjustment is applied to find the camera positions in metric space. We then select two images and compute the epipolar geometry between them using plane+parallax approach. These images are matched to generate a dense disparity map using mutual information. A second bundle adjustment is applied to transform the disparity map into a dense metric depth map, fixing the 2 camera positions. A third bundle adjustment is performed to refine both camera positions and a 3-D structure. All images are back-projected to the 3-D structure for the final registration. The entire process is fully automatic. In addition, a clear definition of "near-planarity " is provided. 3-D reconstruction is shown visually. The method is general, and can be applied to other domains, as shown in the experiments.