Grasping at ‘thin air’: multimodal contact cues for reaching and grasping

Abstract

Two experiments investigated the effects of haptic, auditory and graphic contact cues on reaching to grasp augmented objects (physical and graphic) and virtual objects (graphic only) of various sizes. In Experiment 1, auditory contact cues were presented either to enhance or to replace natural haptic contact cues in grasping. In Experiment 2, graphic contact cues were presented alone or in combination with auditory cues, and were provided either to enhance or to replace haptic contact information. Visual information of the hand was not available. Experiment 1 showed that enhancing haptic contact information with redundant auditory cues (augmented object) led to faster movement times than haptic cues alone. When haptic information was not available (virtual object), it could be replaced to some extent by auditory contact cues. In Experiment 2 movement times were fastest when both auditory and graphic cues were provided, and slowest when no contact cues were provided. Further, movement times were scaled to target width when reaching to grasp augmented objects, thus following Fitts' law. In contrast, movement times showed a less pronounced decrease with increasing object size for virtual objects. However, even in the absence of haptic information, movement times showed a more pronounced scaling to object size when auditory contact cues were provided. These results emphasize the importance of contact information, especially haptic and auditory information, for planning and control of reaching and grasping.