3D-model-based nonrigid motion estimation for multiview image sequence compression

Abstract

ABSTRACT This paper describes a procedure for model-based coding of all channels of a multiview image sequence. The 3Dmodel is initialized by accurate adaptation of a 2D wireframe model to the foreground object of one of the views.The rigid 3D motion is estimated for each triangle, and spatial homogeneity neighbourhood constraints are used toimprove the reliability of the estimation efficiency and to smooth the motion field produced. A novel technique isused to estimate flexible motion of the nodes of the wireframe from the rigid 3D motion vectors of the wireframetriangles containing each node. Kalman filtering is used to track both rigid 3D motion of each triangle and flexible deformation of each node of the wireframe. The performance of the resulting 3D flexible motion estimation methodis evaluated experimentally.Keywords: Multiview image sequence compression, Model-based coding, Non-rigid 3D motion estimation. 1. INTRODUCTION Stereoscopic and multiview video processing has been the focus of considerable attention in recent literature.'3 Ina multiview image sequence, each different view is recorded with a difference in the observation angle, creating anenhanced 3D feeling to the observer, and increased telepresence in teleconferencing and several other (medical,entertainment, etc.) applications. New international video coding standards such as MPEG-4, are in the process of