Developing vowel mappings for an interactive voice synthesis system controlled by hand motions.

Abstract

This study investigates vowel mappings for a voice synthesizer controlled by hand gestures for artistic performance. The vowel targets are on a horizontal plane navigated by the movement of the right hand in front of the performer. Two vowel mappings were explored. In one mapping, the vowels were evenly distributed in a circle to make the vowel targets easier for the performer to find. In the other mapping, the vowels were arranged according to the F2 versus F1 space. Linear hand motions were then made through the vowel space while plotting the formant trajectories. The evenly distributed mapping resulted in formant trajectories that were not monotonic; the F1 and F2 pitch contours varied up and down as the hand carried out the linear motions. This had the unintended result of producing multiple diphthongs. In contrast, the F2 versus F1 mapping enabled the performer to create monotonic formant trajectories and the perception of a single diphthong. The performer found it easier to speak and sing through the system when a single linear hand motion resulted in a single diphthong. [This project was supported by Canada Council for the Arts, Natural Sciences and Engineering Council of Canada, and Media and Graphics Interdisciplinary Centre.]