Twists and Turns in Visuomotor Time Perception and Temporal Recalibration

Abstract

In our perceptual world, voluntary actions (kinesthetically sensed) are temporally aligned with visual feedback about these actions. This requires compensation for existing latencies between the senses, for example due to neural processing. By training participants to the presence of a longer lag (e.g., Stetson et al., 2006) or shorter lag (Rohde & Ernst, 2013) between movement and visual feedback this subjective temporal alignment of vision and motion can also be recalibrated. Similar delay compensation and recalibrated have often been reported in multisensory time perception, including many cases where sensing is passive (e.g., audiovisual, visuotactile: Fujisaki et al., 1994; Keetels & Vroomen, 2008; Di Luca, Machulla, & Ernst, 2009). However, compared to these cases of passive sensing, active sensing, i.e., the processing of sensorimotor signals during intentional action, is influenced by additional factors: (1) the temporal asymmetry of agency and (2) the influence of action on sensation in the closed sensorimotor loop. We here summarize some of our recent results on temporal recalibration in sensorimotor control.For a visual event and an intentional movement event to be perceived as causally linked, the voluntary movement has to happen first (cause has to come before effect). This is not the case if sensory cues are sensed passively. To investigate the role of this temporal asymmetry in visuomotor time perception, we conducted a series of psychophysical experiments (Rohde & Ernst, 2013; Rohde, Greiner & Ernst,; Rohde, Scheller, & Ernst, 2012), where participants were presented with visual events that were timed around a voluntary button press. To be able to time visual stimuli also before the voluntary button press, we monitored participants finger movement and predicted the moment of the button press in real-time from the (stereotypical) movement trajectory. Perceptual judgments about relative visuomotor timing and perceived agency in response to visuomotor discrepancies were recorded.The window of perceived simultaneity resulting from relative timing judgments is asymmetrically skewed to tolerate larger visual lag temporal discrepancies than visual lead discrepancies (simultaneity judgments: Rohde, Scheller, & Ernst, 2012; interval estimation: Rohde, Greiner, & Ernst,). The sense of agency has been previously shown to distort our time perception of vision and intentional movement (Haggard, Clark, & Kalogeras, 2001), and these results suggest that the temporal asymmetry of agency appears to constrain the directionality of this effect. In line with this interpretation, we found that the temporal window of perceived agency is even more temporally asymmetrical (Rohde, Scheller, & Ernst, 2012). Furthermore, if participants’ perception of simultaneity is recalibrated, the window of perceived simultaneity is shrunk or extended on the side of visual lag stimuli only. The perceived timing of visual lead discrepancies that violate the underlying causal structure does not change (Rohde, Greiner, & Ernst,). The temporal asymmetry of intentional action distorts visuomotor time perception in a number of ways.Additionally, in closed-loop motor control tasks, sensory cues are partially determined by participants’ behavioral reactions to alterations in sensorimotor mappings. This implies that motor behavior can play a role in determining the outcome of perceptual recalibration in the closed loop. For instance, we found that target predictability in continuous manual tracking determines whether motor adaptation also involves perceptual temporal recalibration (Rohde, van Dam, & Ernst,). We trained participants in a visually guided manual tracking task with a 200ms visual feedback delay, varying the task predictability between conditions. Only training in the predictable condition led to a wide range of adaptation aftereffects in behavior as well as to perceptual temporal recalibration, which we measured by testing for transfer to synchronization and interval estimation tasks. This shows that the mere exposure to delayed visual feedback in manual tracking is not sufficient to induce temporal recalibration. Delay adaptation on the perceptual level requires specific behavioral adjustments to feedback delays that in turn depend on task predictability. This result can also explain controversies in the literature on closed-loop feedback delay adaptation: Some previous studies supported delay adaptation also on the level of perception (e.g., Cunningham, Billock, & Tsou, 2001; Cunningham et al., 2001a), others report little or no evidence for delay adaptation beyond a mere stabilization of motor strategy (e.g., Miall & Jackson, 2006).These two strands of evidence provide examples of how sensorimotor time perception in the context of intentional action is modulated by additional factors not usually deemed relevant in the study of multisensory temporal processing: the temporal order of modalities, the sense of agency, and the perceptual consequences of motor learning. This distinguishes sensorimotor time perception from multisensory time perception scenarios where stimuli are passively sensed.