A study of match cost functions and colour use in global stereopsis

Abstract

Stereopsis is the process of inferring the distance to objects from two or more images. It has applications in areas such as: novel-view rendering, motion capture, autonomous navigation, and topographical mapping from remote sensing data. Although it sounds simple, in light of the effortlessness with which we are able to perform the task with our own eyes, a number of factors that make it quite challenging become apparent once one begins delving into computational methods of solving it. For example, occlusions that block part of the scene from being seen in one of the images, and changes in the appearance of objects between the two images due to: sensor noise, view dependent effects, and/or differences in the lighting/camera conditions between the two images. Global stereopsis algorithms aim to solve this problem by making assumptions about the smoothness of the depth of surfaces in the scene, and formulating stereopsis as an optimization problem. As part of their formulation, these algorithms include a function that measures the similarity between pixels in different images to detect possible correspondences. Which of these match cost functions work better, when, and why is not well understood. Furthermore, in areas of computer vision such as segmentation, face detection, edge detection, texture analysis and classification, and optical flow, it is not uncommon to use colour spaces other than the well known RGB space to improve the accuracy of algorithms. However, the use of colour spaces other than RGB is quite rare in stereopsis research. In this dissertation we present results from two, first of their kind, large scale studies on global stereopsis algorithms. In the first we compare the relative performance of a structured set of match cost cost functions in five different global stereopsis frameworks in such a way that we are able to infer some general rules to guide the choice of which match cost functions to use in these algorithms. In the second we investigate how much accuracy can be gained by simply changing the colour representation used in the input to global stereopsis algorithms.