Unreal Interactive Puppet Game Development Using Leap Motion

Abstract

This paper proposed a novel puppet play method utilizing recent technology. An interactive puppet game has been developed based on the theme of a famous Chinese classical novel. This project was implemented using Unreal Engine, which is a leading software of integrated tools for developers to design and build games. On the other hand, Leap Motion Controller is a sensor device for recognizing hand movements and gestures. It is commonly used in systems which require close-range finger-based user interaction. In order to manipulate puppets’ movements, the developed program employs the Leap Motion SDK, which provides a friendly way to add motion-controlled 3D hands to an Unreal game. The novelty of our project is to replace 3D model of rigged hands by two 3D humanoid rigged characters. The challenges of this task are two folds. First, the skeleton structure of a human hand and a humanoid character (i.e., puppets) are totally different. Making the puppets to follow the hand poses of the user and yet ensuring reasonable puppets’ movements has not been discussed in the literatures nor in the developer forums. Second, there are only a limited number of built-in recognizable hand gestures. More recognizable hand gestures need to be created for the interactive game. This paper reports the proposed solutions to these challenges.