Segmentation from motion: combining Gabor- and Mallat-wavelets to overcome the aperture and correspondence problems

Abstract

A new method for segmentation from motion is presented, which is designed to be part of a general object-recognition system. The key idea is to integrate information from Gabor- and Mallat-wavelet transforms of an image sequence to overcome the aperture and the correspondence problem. It is assumed that objects move fronto-parallel. Gabor-wavelet responses allow accurate estimation of image flow vectors with low spatial resolution. A histogram over this image flow field is evaluated and its local maxima provide a set of motion hypotheses. These serve to reduce the correspondence problem occurring in utilizing the Mallat-wavelet transform, which provides the required high spatial resolution in segmentation. Segmentation reliability is improved by integration over time. The system can segment several small, disconnected, and openworked objects, such as dot patterns. Several examples demonstrate the performance of the system and show that the algorithm behaves reasonably well, even if the assumption of fronto-parallel motion is not met.