Virtual Kabuki Theater: towards the realization of human metamorphosis systems

Abstract

This paper describes the Virtual Kabuki Theater the authors have recently developed. In the Virtual Kabuki Theater, people at different locations can be Kabuki actors in a Kabuki scene. In our system, Kabuki actors' 3D models are created in advance. Facial expressions of a person are detected in real-time in the face images from the small camera fired to the helmet worn by the person. Body movements of the person are estimated in real-time from the thermal images acquired by the infrared camera that observes the person. The detected expressions and body movements are reproduced in the Kabuki actor's model. Our implementation shows good performance. The Virtual Kabuki Theater is a first step towards human metamorphosis systems, in which anyone can change (metamorphose) his/her form into any other characters.