Object oriented coding using 3D motion estimation

Abstract

In this work, we apply 3D motion estimation to the problem of motion compensation for video coding. We model the video sequence as the perspective projection of a collection of rigid bodies which undergo a roto-translational motion. Motion compensation of the sequence frames can be performed once the shape of the objects and the motion parameters are determined. The motion equations of a rigid body can be formulated as a non linear dynamic system whose state is represented by the motion parameters and by the scaled depths of the object feature points. An extended Kalman filter is then used to estimate both the motion and the object shape parameters simultaneously. We found that the inclusion of the shape parameters in the estimation procedure is essential for reliable motion estimation. Our experiments show that the proposed approach gives the following advantages: the filter gives more reliable estimates in the presence of measurement noise in comparison with other motion estimators that separately compute motion and structure; the filter can effectively track abrupt motion changes; the structure imposed by the model implies that the reconstructed motion is very natural as opposed to more common block-based schemes; the parametrization of the model allows for a very efficient coding of motion information.