PuppetTalk: Conversation Between Glove Puppetry and Internet of Things

Yi-Bing Lin,Chen-Chi Liao,Yu-Fen Huang,Helin Luo

doi:10.1109/access.2020.3048697

Yi-Bing Lin, Chen-Chi Liao + Show 2 more

Open Access

https://doi.org/10.1109/access.2020.3048697

Copy DOI

Journal: IEEE Access	Publication Date: Jan 1, 2021
Citations: 19	License type: CC BY 4.0

Affiliation: National Yang Ming Chiao Tung University

Abstract

Glove Puppetry is popular in Chinese societies. Glove puppet shows have been performed in a traditional (manual) way for over 300 years. Due to the advances in information and communications technology (ICT), we can extend glove puppet art in a very different dimension. However, existing ICT solutions for puppet control have not demonstrated the time complexities for their control mechanisms nor how they can effectively control the puppets. This paper proposes PuppetTalk, an Internet of Things (IoT) application platform for puppet control. We allow multiple input IoT devices such as smart gloves to control a puppet robot, and smartphones can be used to control the sequences of actions on that robot at the same time. In particular, the PuppetTalk Graphical User Interface (GUI) allows users to quickly configure multiple distributed devices to control a puppet robot. Also, controlling puppet legs using a second glove and puppet movements with hand gestures in real time through smart phones are both innovations that have not appeared in the literature so far. When multiple input devices are used to control a puppet robot, their instructions must be received by the robot in a correct sequence. We have conducted measurements, simulation and analytic modeling to show that PuppetTalk can correctly control a puppet robot by multiple input devices with the probability higher than 99.7%.

Highlights

Glove puppetry, a Quanzhou folk performance theater tradition that originated in the 17th century, has been very popular in Chinese societies
We proposed PuppetTalk, an Internet of Things (IoT) application platform for puppet control
PuppetTalk allows distributed input IoT devices such as smart gloves to control a puppet robot, and smartphones can be used to control the sequences of actions on that robot at the same time

Summary

INTRODUCTION

A Quanzhou folk performance theater tradition that originated in the 17th century, has been very popular in Chinese societies. In PuppetTalk, the sensors are grouped as an input device that can record the hand gestures of a puppeteer, e.g., a smart glove or a camera, and transform them into IoT control messages. The received gesture data are synchronized with the skeleton data of every cyber puppet (an output device) in the cloud theater Performance of this approach has not been investigated, and no physical puppets are considered. The study in [4] proposed X-puppet that robotizes the glove puppet (an output device) through a camera-based motion capture system to extract the puppeteer’s motion from the video (an input device) and controls the robot to simulate the puppet gestures. There is a delay (i.e., the think time tT ) between consecutive actions of two hands of a puppeteer (observed from puppeteer behavior), which is used to design the IoT communications mechanism to make sure that the messages are in sequence.

THE KINEMATIC MODEL FOR HAND AND INPUT PUPPET DEVICES

PUPPETTALK CONFIGURATION

TIME COMPLEXITY ANALYSIS

Findings

CONCLUSION