Lexical Gesture Interface

Abstract

Gesture interfaces have long been pursued in the context of portable computing and immersive environments. However, such interfaces have been difficult to realize, in part due to a lack of frameworks for their design and implementation. This paper presents a framework for automatically producing a gesture interface based on a simple interface description. Rather than defining hand positions in a low-level high-dimensional joint angle space, we describe and recognize gestures in a "lexical" space, in which each hand pose is decomposed into elements in a finger-pose alphabet. The alphabet and underlying rules are defined as a gesture notation system called GeLex. By implementing a generic hand pose recognition algorithm, and a mechanism to adapt it to a specific application based on a general interface description, developing a gesture interface becomes straightforward.