Integrating multisensory information across external and motor-based frames of reference

Abstract

In the mirror box illusion, participants often report that their hand is located where they see it, even when the position of the reflected hand differs from the actual position of their hand. This illusory shift (an index of multisensory integration) is stronger when the two hands engage in synchronous bimanual movement, in which visual and proprioceptive information is congruent in both motor-based (i.e. coordinate centered on the effector) and external (i.e. coordinates centered on elements external to the effector) frames of reference. To investigate the separate contributions of external and motor-based congruence in multisensory integration, we instructed participants to make synchronous or asynchronous tapping movements in either the same (i.e. both hands palms up) or opposing (palm up, palm down) postures. When in opposing postures, externally congruent movements were incongruent in a motor-based frame of reference, and vice versa. Across three experiments, participants reported more illusory shift and stronger ownership of the viewed hand in the mirror for external versus motor-based congruence trials regardless of motor outflow or motor effort, indicating that information from an externally-based representation is more strongly weighted in multisensory integration. These findings provide evidence that not only information across sensory modalities, but also information regarding crossmodal congruence represented in different spatial frames of reference, is differentially weighted in multisensory integration. We discuss how our findings can be incorporated into current computational models on multisensory integration.