Self-Consistency and MDL: A Paradigm for Evaluating Point-Correspondence Algorithms, and Its Application to Detecting Changes in Surface Elevation

Abstract

The self-consistency methodology is a new paradigm for evaluating certain vision algorithms without relying extensively on ground truth. We demonstrate its effectiveness in the case of point-correspondence algorithms and use our approach to predict their accuracy. For point-correspondence algorithms, our methodology consists in applying independently the algorithm to subsets of images obtained by varying the camera geometry while keeping 3-D object geometry constant. Matches that should correspond to the same surface element in 3-D are collected to create statistics that are then used as a measure of the accuracy and reliability of the algorithm. These statistics can then be used to predict the accuracy and reliability of the algorithm applied to new images of new scenes. An effective representation for these statistics is a scatter diagram along two dimensions: A normalized distance and a matching score. The normalized distance make the statistics invariant to camera geometry, while the matching score allows us to predict the accuracy of individual matches. We introduce a new matching score based on Minimum Description Length (MDL) theory, which is shown to be a better predictor of the quality of a match than the traditional Sum of Squared Distance (SSD) score. We demonstrate the potential of our methodology in two different application areas. First, we compare different point-correspondence algorithms, matching scores, and window sizes. Second, we detect changes in terrain elevation between 3-D terrain models reconstructed from two sets of images taken at a different time. We finish by discussing the application of self-consistency to other vision problems.