3D reconstruction of non-textured surface by combining shape from shading and stereovision

Abstract

For the improvement of robot-assisted minimally invasive surgery, 3D reconstruction of the visual field in surgery is demanding nowadays. This task faces the difficulty that the tissue surfaces in the visual field usually appear to be non-textured. How to reconstruct such kind of surfaces well under this environment is unsolved. Among many vision techniques, Shape From Shading is a unique method which infers surface shape (normal field) based on the reflectance model without requiring any texture. One major limitation is that the accurate surface depth (not shape) is unable to compute when illumination parameters are not calibrated. Stereovision is a long-standing method which estimates depth by establishing the pixel correspondences between two camera images. But it fails to find correspondences when encountered with non-textured surfaces. It is intuitive to combine these two methods to recover depth of non-textured surfaces and calibrate the illumination parameters as well. However, how to bridge these two methods efficiently is challenging. In this paper, the combination of these two methods is accomplished by a 3D matching framework based on Fast Point Feature Histogram (FPFH). We first reconstruct initial surface point clouds of left image and right image using SFS method. The pixel correspondences can be built by aligning SFS calculated point clouds using FPFH 3D descriptors. Therefore, stereo method can reconstruct depth of the surface. Furthermore, we employ a classic variational fusion algorithm which refines SFS-shape and stereovision-depth. Results of the recovered depth are shown through experiments.