<title>Joint multigrid estimation of 2D motion and object boundaries using boundary patterns</title>

Abstract

ABSTRACT For the improvement ofjoint estimation of 2D motion and 2D object boundaries an optimization procedure based on multignd andpattern processing is presented. In this procedure an iterative optimization is applied using grids of different resolution, startingwith a global estimation of the motion in the coarsest grid. Improvements are achieved by the introduction of boundary patterns,which are obtained from boundary estimates from coarser grids to be used in the finer ones. It is shown that the developed techniquecompared to a hierarchical blockmatching technique leads to a significant reduction of the mean square error between compensatedand original images at object boundaries. 1 INTRODUCTION The 2D motion in an image sequence originates from 3D motion of objects or from 3D motion of the camera within the 3D sceneprojected into the image plane. It is represented as a displacement vector field. If, during the image sequence, objects are movingin front of others or a camera is moving through a static 3D scene containing object parts covering others, discontinuities in thedisplacement vector field occur. In the case that the camera is moving, the estimation of discontinuities in the displacement vectorfield is equivalent to the estimation of depth discontinuities in the 3D scene. In the case that the real scene consists of differentlymoving objects, discontinuities in the displacement vector field originate also from the boundaries of the moving objects. Theestimation of the displacement vector field and its discontinuities mutually depends on each other. Therefore the estimation mustbe performed simultaneously.Basic work for the estimation of the 2D motion in image sequences was done by Horn and Schunck [1]. Introducing a smoothnessconstraint for the 2D motion, they proposed a cost function for the motion estimation, which leads to smooth motion estimates,but does not consider object boundaries. This cost function has been extended and adapted by several researchers in order to copewith object boundaries [2],[3],[4]. Line processes, first proposed by Geman and Geman [51forimage restoration, have been provento be suitable for the estimation of discontinuities. The insertion of line processes into the cost function allows discontinuities inthe displacement vector field, but leads to a non—convex optimization problem. Both, stochastic and deterministic optimizationstrategies using local iterative optimization have been applied and compared [3],[4},[6],[7]. It turned out that stochastic optimiza-tion procedures like simulated annealing [8],[9],[1O] lead to better results, but with a computational effort of an order of magnitudelarger than deterministic ones [4]. Deterministic procedures often get stuck in local minima of the cost function. The use of initialmotion estimates obtained by blockmatching, which is a widely used displacement estimation technique in image coding, furtherreduces the computational effort and improves the 2D motion estimates, but can still not prevent deterministic procedures to get