Training a Convolutional Neural Network for Disparity Optimization in Stereo Matching

Abstract

In this paper, we describe an efficient stereo matching algorithm which is inspired by the excellent performances of convolutional neural network (CNN) on vision problems in recent years. Our algorithm applies adaptive smoothness constraints making use of disparity discontinuous information to optimize the overall disparity map. First, we define a CNN architecture called DD-CNN to classify whether disparities of pixels in the image is continuous or not. The training data set is constructed from Middlebury stereo data sets. Once we obtain the disparity discontinuous map, different penalizes are applied to the energy function which takes the whole disparity map as argument. The algorithm imposes large penalizes to disparity differences between pixels and their neighborhoods when disparities of the center pixels are predicted to be discontinuous and small penalizes otherwise. Experiments show that the proposed algorithm performs better than the state-of-art algorithm.