Differential methods for the identification of 2D and 3D motion models in image sequences

Abstract

Abstract This paper presents a detailed analysis of the differential approach for motion estimation in video image sequences. The models considered are defined either by a parametric approach, or by a physical approach (in terms of parameters of the pick-up equipment, movement and object structures). The relationships between the 2D and the 3D approaches are examined. The ambiguities inherent in a physical interpretation of a set of descriptors identified from image sequences are underlined. The critical points when using differential estimators are discussed, in particular, we study several classical image processing tools which improve the convergence rate of these estimators (hierarchical analysis, multiresolution, spatial interpolation of the luminance…). Definition and tuning of gains, initialization stage, cross-dependence between image segmentation and identification of the parameters associated to each region (as well as the duality between top-down and bottom-up approaches), which partly condition the behavior of the algorithms, are studied too. Results in terms of motion and segmentation maps, images predicted from one or several previous images by motion compensation, convergence curves of some of the proposed iterative algorithms illustrate this paper. Finally, we draw from these theoretical developments and the associated simulations some key elements for an effective implementation of a complete motion estimation scheme. We conclude by some perspectives for future work.