Freehand pose-based Gestural Interaction: Studies and implications for interface design

Abstract

Most work on Freehand Gestural Interaction has focused on high-energy expressive interfaces for expert user. In this work, we examine the use of hand poses in laid-back freehand gestural interactions for novice users and examine the factors that impact gesture performance. Through two Wizard-of-Oz studies, one leading to the other, we observe how novice users behave under relaxed conditions. The first study explores the ease of use of a pose-based hand gesture vocabulary in the context of a photo-browsing task, and examines some of the key factors that impact the performance of such pose based gestures. The second explores pose-based interaction techniques for widget manipulation tasks. These studies reveal that while hand poses have the potential to expand the vocabulary of gestures and are easy to recall and use, there are a number of issues that show up in actual performance related to inadvertent modifications in hand pose and hand trajectories. We summarize the implications of these findings for the design of pose-based freehand gestural interfaces, which we believe would be useful for both interaction designers and gesture recognition researchers.