Use of object articulation and motion estimation for video production

Abstract

This paper describes a procedure for stereo video production using model-based processing of both left and right channels of a stereoscopic image sequence. The proposed scheme uses an original video and the first / basic frame of a video to be produced according to the original one. A 3D model is created first for both the original and the basic images using consistent depth information. Robust classification techniques are then used to obtain an articulated description of the foreground of the scene (head, neck, shoulders). The object articulation procedure is based on a novel scheme for the segmentation of the rigid 3D motion fields of the triangle patches of the 3D model object. This procedure is applied for both the original and basic images resulting to corresponding sub-objects of the two images. The rigid 3D motion is then re-computed for each sub-object of the original image sequence and finally, the motion parameters produced for each sub-object of the original video are applied to the corresponding sub-objects of the first image of the video to be produced. The model of the produced image is moved according to the original one and the result is a new video with the same motion but different shape, size and texture than the original one. The described method is demonstrated over two stereoscopic videoconference image sequences.