Image Talk: a real time synthetic talking head using one single image with Chinese text-to-speech capability

Abstract

Image Talk uses a single image to automatically create talking sequences in real time. The image can be acquired from a photograph, video clip, or hand drawn characters. This interactive system accepts Chinese text and talks back in Mandarin Chinese, generating facial expression in real time. Image Talk analyzes Chinese text by converting it to a standard Pinyin system used in Taiwan and fetches the associated facial expressions from an expression pool. The expressions are synchronized with the sound and played back in a talking sequence. Image Talk also incorporates eye blinking, small scale head rotation and translation perturbations, to make the resulting sequence more natural. It is also easy to switch to any other face images. The result is quite entertaining, and can easily be used as a new human machine interface, as well as for lip sync in computer animated characters.