Hierarchical 2-D mesh representation, tracking, and compression for object-based video

Abstract

This paper proposes methods for designing, tracking and coding hierarchical two-dimensional (2-D) content-based mesh representations. The design procedure consists of constructing a fine-to-coarse hierarchy of Delaunay meshes, using image- and shape-based criteria for mesh geometry simplification. Hierarchical tracking employs a coarse-to-fine strategy with mesh-based motion vector optimization. We introduce new techniques to maintain the initial mesh hierarchy and topology during tracking by imposing certain constraints at each stage of the procedure. The hierarchical compression technique is based on a nearest neighbor ordering of mesh node points. This ordering serves to identify the mesh boundary nodes as well as establish spatial predictors for differential coding of node coordinates and motion vectors. The proposed hierarchical mesh representation, which has applications in object-based video manipulation, investing, and compression, provides improved tracking performance (compared to a nonhierarchical representation) and allows progressive (scalable) transmission of the object geometry (including shape) and motion information, as well as variable level-of-detail rendering. Experimental results are presented to compare the tracking and compression performance of hierarchical versus nonhierarchical mesh representations and to demonstrate the tradeoff between image quality and mesh bit rate for 2-D mesh-based video object rendering.