Constraint, Optimization, and Hierarchy: Reviewing Stereoscopic Correspondence of Complex Features

Abstract

To extract three dimensional data from a pair of images, it is essential to solve the correspondence problem. In the literature, a large number of algorithms have been implemented which differ in the token type, match constraints, and search methods employed. Recently, hierarchical matching schemes have utilized multiple token types of increasing complexity. In previous reviews of stereopsis, no general framework has emerged within which to evaluate all the different contributions. This paper breaks down the correspondence problem into its general components: token type, match constraints, and method employed to encode and search match information. In common with other reported work, matching is cast as an optimization problem, and the definition of match functionals may be separated from the method employed to search the solution space. Within this very general framework, hierarchical matching is discussed at some length including suggestions on how hierarchical constraints may be formally embedded within the matching algorithm. The benefits of the hierarchical approach are illustrated with some examples.