Mid-air interaction with optical tracking for 3D modeling

Abstract

Compared to common 2D interaction done with mouse and other 2D tracking devices, 3D hand tracking with low-cost optical cameras can provide more degrees of freedom, as well as natural gestures, when shape modeling is done in virtual spaces. However, though quite precise, the optical tracking devices cannot avoid problems intrinsic to hand interaction, such as hand tremor and jump release, and they also introduce an additional problem of hand occlusion. We investigate how to minimize the negative impact of these problems, and eventually propose to use hands in a way similar to how it is done when playing the Theremin – an electronic musical instrument controlled without physical contact by hands of the performer. We suggest that the dominant hand controls manipulation and deformation of objects while the non-dominant hand controls grasping, releasing and precision of interaction. Based on this method, we describe a generic set of reliable and precise interaction gestures for various manipulation and deformation tasks. We then prove with the user study that for the tasks involving 3D manipulations and deformations, hand interaction is faster than common 2D interaction done with mouse.