A Grasping Device to Sense Hand Gesture for Expressive Sound Generation

Abstract

This article introduces a new device driven by emotional human twiddling gestures, and its application as a musical instrument. In traditional musical instruments, the relationship between the action and the generated sound is determined by the physical structures of the instruments. Although various computerized musical instruments have been actively developed and reported, the development of a human-computer interface is not sufficient for the translation of emotional feelings into sounds. We have so far paid attention to emotional feelings presented in gestures and music, and have developed a gesture acquisition device that measures grasping forces caused by user's twiddling actions by using compact pressure sensors and acceleration sensors. For the association of gesture patterns with sound parameters, a SOM algorithm is applied. The approach is considered to provide not only a new sound generation technique in the music scene, but also a flexible human-computer interface.