"It's Natural to Grab and Pull": Retrieving Content from Large Displays Using Mid-Air Gestures

Abstract

Mid-air gestures have been largely overlooked for transferring content between large displays and personal mobile devices. To fully utilize the ubiquitous nature of mid-air gestures for this purpose, the authors developed SimSense, a smart space system that automatically pairs users with their mobile devices based on location data. Users can then interact with a gesture-controlled large display and move content onto their handheld devices. In a user study, the authors investigated two mid-air gestures for content transfer: grab-and-pull and grab-and-drop. Their results show that mid-air gestures are well suited for content-retrieval scenarios and offer an impressive user experience; grab-and-pull is the preferred for scenarios when content is transferred to the user, whereas grab-and-drop is presumably ideal when the recipient is another person or a device; and distinct gestures can be successfully combined with common point-and-dwell mechanics prominent in many gesture-controlled applications.