Brand-new Speech Animation Technology based on First Order Motion Model and MelGAN-VC

Abstract

Speech animation has huge application potential in instant messaging and entertainment media fields such as videophones, virtual meetings, audio and video chats. The traditional voice-driven speech animation has the problem of a single adaptation language, and the performance-driven speech animation has the problem of high cost of capture equipment and difficult mass production. Based on the above existing problems, we propose a new method of speech animation generation, that is, given a static portrait of a person and a face-driven video, finally generate a face animation video of the character in the given portrait. The conversion system consists of two parts: face conversion and voice conversion. We noticed that the final generated face animation video has problems such as low definition, not smooth playback, and metallic sound. On this basis, this article proposes to increase the animation enhancement experiment and replace the encoder measures for improvement. Through comparative experiments, the above measures are proved to be effective.