Animating speech: an automated approach using speech synthesised by rules

Abstract

This paper is concerned with the problem of animating computer drawn images of speaking human characters, and particularly with the problem of reducing the cost of adequate lip synchronisation. Since the method is based upon the use of speech synthesis by rules, extended to manipulate facial parameters, and there is also a need to gather generalised data about facial expressions associated with speech, these problems are touched upon as well. Useful parallels can be drawn between the problems of speech synthesis and those of facial expression synthesis. The paper outlines the background to the work, as well as the problems and some approaches to solution, and goes on to describe work in progress in the authors' laboratories that has resulted in one apparently successful approach to low-cost animated speaking faces. Outstanding problems are noted, the chief ones being the difficulty of selecting and controlling appropriate facial expression categories: the lack or naturalness of the synthetic speech; and the need to consider the body movements and speech of all characters in an animated sequence during the animation process.