Realistic head motion synthesis for an image-based talking head

Abstract

In this paper, we present a novel approach to add flexible head motions to talking heads. First, head motion patterns are collected from original recordings. These head motion patterns are recorded video segments with different head motions, like nod and shake. The head motion is synthesized by selecting and concatenating appropriate head motion patterns according to the input text or head motion tags. In order to join these patterns, optical flow based morphing is used to smooth transitions without introducing noticeable discontinuities. Experimental results show that head motion synthesis is realistic, and animations with flexible head motions are rated with a higher average mean opinion score than the ones with repeated head motions.