Scalable Region-matching Motion Estimation Based on an Unsupervised Spatial Segmentation

Abstract

In a video scene, motion estimation (ME) can be studied on a dense field (optical flow) or on image structures or regions. Image structures can be deduced from the motion itself or formerly deduced by a segmentation. A scheme of ME, funded on a Bayesian segmentation using a Potts‐Markov model, has lead to a “region‐matching” ME scheme. Bayesian segmentation has been operated, with the same model, in the wavelet domain and has shown an interesting gain in segmentation speed. In the present work we have synthesized both approaches to demonstrate a new scheme of region‐matching ME which uses the hierarchical property of the multiscale segmentation scheme. A bottom‐top ME is built from the hierarchical segmentation in the wavelet domain. We show that a hierarchical, region‐based, ME, can provide an interesting approach w.r.t. the necessity of ME robustness as well as its scalability, in a region (or object) ‐based compression scheme. This approach is to be compared with recent developments like the “structure from motion” (SfM) in [15], based on Bayesian inference and sequential Monte Carlo methods, and the “trace model” for object (face) detection and tracking in [12] (see also [13]).