Gesture analysis in a case study with a tangible user interface for collaborative problem solving

Abstract

This paper describes a case study that took place at the Public Research Centre Henri Tudor, Luxembourg in November 2012. A tangible user interface (TUI) was used in the context of collaborative problem solving. The task of participants was to explore the relation of external parameters on the production of electricity of a windmill presented on a tangible tabletop; these parameters were represented through physical objects. The goal of the study was to observe, analyze, and understand the interactions of multiple participants with the table while collaboratively solving a task. In this paper we focus on the gestures that the users performed during the experiment and the reaction of the other users to those gestures. Gestures were categorized into deictic/pointing, iconic, emblems, adaptors, and TUI-related. TUI-related/manipulative gestures, such as tracing and rotating, represented the biggest part, followed by the pointing gestures. In addition, we evaluated how active was the participation of the participants and whether gesture was accompanied by speech during the user study. Our case study can be described as a collaborative, problem solving, and cognitive activity, which showed that gesturing facilitates group focus, enhances collaboration among the participants, and encourages the use of epistemic actions.