Guidophone: A handheld virtual music instrument combining vocal tract geometry and hand gestures

Abstract

Vocal sounds provide an intuitive and appealing basis for virtual instruments. The former aspect allows most people to engage in a musical activity simply by repetition of a heard sound. Physical models in speech synthesis attempt, among other objectives, to approximate the geometry of the vocal tract required to produce specific sounds. Cook has developed one such model [Perry Cook, Identification of Control Parameters in an Articulatory Vocal Tract Model with Applications to the Synthesis of Singing, Stanford University, 1990], where the vocal tract is divided into tube sections that govern the transmission and reflection of acoustic energy at the junctions between sections. Tube section radii provide the model parameters. This paper explores the feasibility of using hand gestures to control those parameters. To that end, the problem of mapping hand gestures to the parameters of Cook’s vocal synthesis model, with attention paid to transitions between successive phonemes, is explored. The physical basis of the model is reviewed. Next the relationship between the geometries implied by Cook’s tract model and those determined by experiment is investigated. A means for mapping a number of parameters less than the degrees of freedom inherent in the model is presented. Finally, the details of a virtual instrument, controlled by a mechanical device recently developed for the work, are presented.