Segmentation from motion: Combining Gabor- and Mallat-wavelets to overcome aperture and correspondence problem

Abstract

A segmentation-from-motion algorithm is presented, which is designed to be part of a general object recognition system. The key idea is to integrate information from Gabor- and Mallat-wavelet transform to overcome the aperture and the correspondence problem. The assumption is made that objects move fronto-parallel. Gabor-wavelet responses allow precise estimation of image flow vectors with low spatial resolution. A histogram over this image flow field is evaluated, its local maxima providing motion hypotheses. These serve to reduce the correspondence problem on the Mallat-wavelet transform, which provides the required high resolution. The segmentation reliability is improved by integration over time. The system can segment even small, disconnected, and openworked objects of arbitrary number, such as dot patterns. Several examples demonstrate the performance of the system and show, that the algorithm behaves reasonably even if the assumption of fronto-parallel motion is strongly violated.