Realistic speech animation of synthetic faces

Abstract

We combined physically based modeling and parameterization to generate realistic speech animation on synthetic faces. We used physically based modeling for muscles. Muscles are modeled as forces deforming the mesh of polygons. A parameterization technique is used for generating mouth shapes for speech animation. Each meaningful part of a text, which is a letter in our case, corresponds to a specific mouth shape and the mouth shape is generated by setting a set of parameters used for representing the muscles and jaw rotation. We also developed a mechanism to generate and synchronize facial expressions while speaking. Some tags specifying the facial expressions are inserted into the input text together with the degree of the expression. In this way, the facial expression with the specified degree is generated and synchronized with speech animation.