Motion and disparity field estimation using rate-distortion optimization

Abstract

A rate-distortion framework is used to define a displacement vector field estimation technique for use in video coding. This technique achieves maximum reconstructed image quality under the constraint of a target bit rate for the coding of the vector sequence. The technique may be adapted so as to limit its smoothing effect to homogeneous areas and avoid highly textured areas and edges. Use of this technique is evaluated for two application areas in which the need for high compression of displacement vector fields is particularly acute. The first is motion field coding for very-low-bit-rate image sequence transmission, as in video-phone applications. The second application area is coding for the transmission of dense disparity fields. This is needed for the generation at the receiver of intermediate viewpoints through spatial interpolation. It is also needed in a number of other applications requiring accurate depth knowledge, including three-dimensional medical data transmission and transmission of scenes to be postprocessed using depth-keyed segmentation. Experimental results illustrating the performance of the proposed technique in these application areas are presented and evaluated.