Motor-Intuitive Interactions Based on Image Schemas: Aligning Touchless Interaction Primitives with Human Sensorimotor Abilities

Abstract

Elicitation and evaluation studies investigated intuitiveness of touchless gestures but did not operationalizeintuitiveness.Forexample,studiesfoundthatusersfailtomakeaccurate3Dstrokesas interactioncommands.Butthisphenomenonremainsunexplained.Inthispaper,wefirstexplainhow making accurate 3D strokes is generally unintuitive, because it exceeds our sensorimotor knowledge. Wethenintroducemotor-intuitive,touchlessinteractionthatusessensorimotorknowledgebyrelying on image schemas. Specifically, we propose an interaction primitive—mid-air, directional strokes— based on space schemas up‐down and left‐right. In a controlled study with large displays, we found that biomechanical factors affected directional strokes. Strokes were efficient (0.2s) and effective (12.5 ◦ angular error), but affected by directions and length. Our work operationalized intuitive touchless interaction using the continuum of knowledge in intuitive interaction, and demonstrated how user performance of a motor-intuitive, touchless primitive based on sensorimotor knowledge (image schemas) is affected by biomechanical factors.