Abstract
The author presents a new face module for MASSY, the Modular Audiovisual Speech SYnthesizer [1]. Within this face module the system combines two approaches of visual speech synthesis. Although the articulation space is parameterized in terms of movements of the articulators, the visual synthesis is image based (video-realistic). The high-level visual speech synthesis generates a sequence of control commands for the visible articulation. The video synthesis searches an image database for appropriate video frames. If no image with facial properties according to the control commands is found, the missing image is generated by deforming a neutral image. MPEG-4 facial definition parameters (FDPs) [2] and additional points in the mouth opening area and around the lower jaw are defined in the neutral image as feature points. A twodimensional displacement vector is defined for each feature point. For the image deformation a mesh of triangles connecting the feature points is used. The displacement vector of a point in a triangle is interpolated from the displacement vectors of the vertices. Hence, the video synthesis algorithm is capable to use either a database of appropriately annotated video frames or a single neutral image with specified feature points and displacement vectors. A simple software tool for marking the feature points in the image was developed. Other well known data (image) based audio-visual speech synthesis systems like MIKETALK [3] and VIDEO REWRITE [4] concatenate prerecorded video sequences. The presented system demonstrates the compatibility of parametric and data based visual speech synthesis approaches.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.