Disparity estimation of stereo-endoscopic images using deep generative network

Abstract

A novel disparity estimation pipeline is proposed for 3D reconstruction of dynamic soft tissues in minimally invasive surgery (MIS), which uses a deep generative network to learn manifold distributions of reasonable disparity maps from past stereo images in the training phase, and transforms stereo matching into an optimization problem with respect to the low-dimensional latent vector of the learned generator in the application phase. The proposed pipeline is particularly suitable for dynamic MIS scenarios with insufficient training data, as the photometric loss is explicitly used in the application phase and the scenario priors is introduced via a deep generative network.