Visual Remapping Research Articles

Functional Dynamics of Deafferented Early Visual Cortex in Glaucoma.

In advanced retinitis pigmentosa with retinal lesions, the lesion projection zone (LPZ) in the early visual cortex can be driven during visual tasks, while it remains unresponsive during passive viewing. We tested whether this finding translates to advanced glaucoma, a major cause of acquired blindness. During visual stimulation, 3T fMRI scans were acquired for participants with advanced glaucoma (n = 4; age range: 51–72) and compared to two reference groups, i.e., advanced retinitis pigmentosa (n = 3; age range: 46–78) and age-matched healthy controls with simulated defects (n = 7). The participants viewed grating patterns drifting in 8 directions (12 s) alternating with uniform gray (12 s), either during passive viewing (PV), i.e., central fixation, or during a one-back task (OBT), i.e., reports of succeeding identical motion directions. As another reference, a fixation-dot task condition was included. Only in glaucoma and retinitis pigmentosa but not in controls, fMRI-responses in the lesion projection zone (LPZ) of V1 shifted from negative for PV to positive for OBT (p = 0.024 and p = 0.012, respectively). In glaucoma, these effects also reached significance in V3 (p = 0.006), while in V2 there was a non-significant trend (p = 0.069). The general absence of positive responses in the LPZ during PV underscores the lack of early visual cortex bottom-up plasticity for acquired visual field defects in humans. Trends in our exploratory analysis suggesting the task-dependent LPZ responses to be inversely related to visual field loss, indicate the benefit of patient stratification strategies in future studies with greater sample sizes. We conclude that top-down mechanisms associated with task-elicited demands rather than visual cortex remapping appear to shape LPZ responses not only in retinitis pigmentosa, but also in glaucoma. These insights are of critical importance for the development of schemes for treatment and rehabilitation in glaucoma and beyond.

Chapter 13 - Bálint syndrome

This chapter starts by reviewing the various interpretations of Bálint syndrome over time. We then develop a novel integrative view in which we propose that the various symptoms, historically reported and labeled by various authors, result from a core mislocalization deficit. This idea is in accordance with our previous proposal that the core deficit of Bálint syndrome is attentional (Pisella et al., 2009, 2013, 2017) since covert attention improves spatial resolution in visual periphery (Yeshurun and Carrasco, 1998); a deficit of covert attention would thus increase spatial uncertainty and thereby impair both visual object identification and visuomotor accuracy. In peripheral vision, we perceive the intrinsic characteristics of the perceptual elements surrounding us, but not their precise localization (Rosenholtz et al., 2012a,b), such that without covert attention we cannot organize them to their respective and recognizable objects; this explains why perceptual symptoms (simultanagnosia, neglect) could result from visual mislocalization. The visuomotor symptoms (optic ataxia) can be accounted for by both visual and proprioceptive mislocalizations in an oculocentric reference frame, leading to field and hand effects, respectively. This new pathophysiological account is presented along with a model of posterior parietal cortex organization in which the superior part is devoted to covert attention, while the right inferior part is involved in visual remapping. When the right inferior parietal cortex is damaged, additional representational mislocalizations across saccades worsen the clinical picture of peripheral mislocalizations due to an impairment of covert attention.

The Detached Self: Investigating the Effect of Depersonalisation on Self-Bias in the VisualRemappingof Touch.

There is a growing consensus that our most fundamental sense of self is structured by the ongoing integration of sensory and motor information related to our own body. Depersonalisation (DP) is an intriguing form of altered subjective experience in which people report feelings of unreality and detachment from their sense of self. The current study used the visual remapping of touch (VRT) paradigm to explore self-bias in visual-tactile integration in non-clinical participants reporting high and low levels of depersonalisation experiences. We found that the high-DP group showed an increased overall VRT effect but a no-self-face bias, instead showing a greater VRT effect when observing the face of another person. In addition, across all participants, self-bias was negatively predicted by the occurrence of anomalous body experiences. These results indicate disrupted integration of tactile and visual representations of the bodily self in those experiencing high levels of DP and provide greater understanding of how disruptions in multisensory perception of the self may underlie the phenomenology of depersonalisation.

Differential parietal activations for spatial remapping and saccadic control in a visual memory task

Remapping is a process that updates visual information in internal spatial representations across eye movements, allowing for stable perception of the environment. Previous work has demonstrated visual remapping activity in parietal cortex during saccades, but it remains unclear whether remapping is triggered by overt saccades only (or by attentional shifts also), and whether it engages parietal areas only (or other cortical areas). Here, we used fMRI to investigate spatial remapping during two visuospatial memory tasks requiring either overt (accompanied by a saccade) or covert (with central fixation) attention shifts to peripheral distracters. Participants had to remember the position and color of a lateralized dot during a saccade or attention shift, requiring them to update the dot position in memory, and then indicate if a second dot matched the first. Differential activation patterns were observed within parietal cortex as a function of the different visual, motor, and interhemispheric remapping demands in the saccade task, presumably mediating the maintenance of spatial position in perceptual and motor maps. Remapping engaged parietal areas adjacent to, but not overlapping with, those activated by saccade execution, while it did not engage the frontal eye fields, pointing to distinct neural substrates for ocular motor and spatial updating processes. No differential activation related to remapping was found during the covert attention shift task, suggesting that this condition did not necessitate the same remapping as the saccade condition. Overall these results further elucidate the mechanisms of spatial remapping in human parietal cortex and their relationship with attention processing and ocular motor behavior, with implications for understanding visuospatial attention deficits in hemispatial neglect.

Fear-specific enhancement of tactile perception is disrupted after amygdala lesion.

Tactile perception on one's own face is enhanced when viewing a fearful face being touched - as opposed to just approached - by fingers, compared to viewing other expressions, a phenomenon known as the emotional modulation of Visual Remapping of Touch (eVRT). This effect seems to be related to a preferential activation of the somatosensory system in response to threat. To test the contribution of the amygdala to this mechanism, a group of patients with unilateral lesions to the amygdala, a control group of patients with lesions in the extra-temporal regions, and a group of healthy participants completed an eVRT paradigm. They were required to detect bilateral tactile stimulation on their own cheeks, while viewing fearful, happy, or neutral faces being touched or just approached by fingers. Healthy participants and control patients confirmed that viewing a neutral face being touched - as opposed to just approached - by fingers increases tactile detection on one's own face (i.e., the typical VRT effect) and that this effect is enhanced for fearful faces, compared to neutral and happy faces. However, in patients with amygdala lesion, although the standard VRT effect was preserved for neutral faces, this was disrupted for fearful faces. This result indicates that the preferential activation of the somatosensory cortices in response to threat relies on structural integrity of the amygdala.

Corollary Discharge Contributions to Perceptual Continuity Across Saccades.

Our vision depends upon shifting our high-resolution fovea to objects of interest in the visual field. Each saccade displaces the image on the retina, which should produce a chaotic scene with jerks occurring several times per second. It does not. This review examines how an internal signal in the primate brain (a corollary discharge) contributes to visual continuity across saccades. The article begins with a review of evidence for a corollary discharge in the monkey and evidence from inactivation experiments that it contributes to perception. The next section examines a specific neuronal mechanism for visual continuity, based on corollary discharge that is referred to as visual remapping. Both the basic characteristics of this anticipatory remapping and the factors that control it are enumerated. The last section considers hypotheses relating remapping to the perceived visual continuity across saccades, including remapping's contribution to perceived visual stability across saccades.

Neural Network Evidence for the Coupling of Presaccadic Visual Remapping to Predictive Eye Position Updating.

As we look around a scene, we perceive it as continuous and stable even though each saccadic eye movement changes the visual input to the retinas. How the brain achieves this perceptual stabilization is unknown, but a major hypothesis is that it relies on presaccadic remapping, a process in which neurons shift their visual sensitivity to a new location in the scene just before each saccade. This hypothesis is difficult to test in vivo because complete, selective inactivation of remapping is currently intractable. We tested it in silico with a hierarchical, sheet-based neural network model of the visual and oculomotor system. The model generated saccadic commands to move a video camera abruptly. Visual input from the camera and internal copies of the saccadic movement commands, or corollary discharge, converged at a map-level simulation of the frontal eye field (FEF), a primate brain area known to receive such inputs. FEF output was combined with eye position signals to yield a suitable coordinate frame for guiding arm movements of a robot. Our operational definition of perceptual stability was “useful stability,” quantified as continuously accurate pointing to a visual object despite camera saccades. During training, the emergence of useful stability was correlated tightly with the emergence of presaccadic remapping in the FEF. Remapping depended on corollary discharge but its timing was synchronized to the updating of eye position. When coupled to predictive eye position signals, remapping served to stabilize the target representation for continuously accurate pointing. Graded inactivations of pathways in the model replicated, and helped to interpret, previous in vivo experiments. The results support the hypothesis that visual stability requires presaccadic remapping, provide explanations for the function and timing of remapping, and offer testable hypotheses for in vivo studies. We conclude that remapping allows for seamless coordinate frame transformations and quick actions despite visual afferent lags. With visual remapping in place for behavior, it may be exploited for perceptual continuity.

A State Space Model for Spatial Updating of Remembered Visual Targets during Eye Movements.

In the oculomotor system, spatial updating is the ability to aim a saccade toward a remembered visual target position despite intervening eye movements. Although this has been the subject of extensive experimental investigation, there is still no unifying theoretical framework to explain the neural mechanism for this phenomenon, and how it influences visual signals in the brain. Here, we propose a unified state-space model (SSM) to account for the dynamics of spatial updating during two types of eye movement; saccades and smooth pursuit. Our proposed model is a non-linear SSM and implemented through a recurrent radial-basis-function neural network in a dual Extended Kalman filter (EKF) structure. The model parameters and internal states (remembered target position) are estimated sequentially using the EKF method. The proposed model replicates two fundamental experimental observations: continuous gaze-centered updating of visual memory-related activity during smooth pursuit, and predictive remapping of visual memory activity before and during saccades. Moreover, our model makes the new prediction that, when uncertainty of input signals is incorporated in the model, neural population activity and receptive fields expand just before and during saccades. These results suggest that visual remapping and motor updating are part of a common visuomotor mechanism, and that subjective perceptual constancy arises in part from training the visual system on motor tasks.

Perisaccadic Updating of Visual Representations and Attentional States: Linking Behavior and Neurophysiology.

During natural vision, saccadic eye movements lead to frequent retinal image changes that result in different neuronal subpopulations representing the same visual feature across fixations. Despite these potentially disruptive changes to the neural representation, our visual percept is remarkably stable. Visual receptive field remapping, characterized as an anticipatory shift in the position of a neuron’s spatial receptive field immediately before saccades, has been proposed as one possible neural substrate for visual stability. Many of the specific properties of remapping, e.g., the exact direction of remapping relative to the saccade vector and the precise mechanisms by which remapping could instantiate stability, remain a matter of debate. Recent studies have also shown that visual attention, like perception itself, can be sustained across saccades, suggesting that the attentional control system can also compensate for eye movements. Classical remapping could have an attentional component, or there could be a distinct attentional analog of visual remapping. At this time we do not yet fully understand how the stability of attentional representations relates to perisaccadic receptive field shifts. In this review, we develop a vocabulary for discussing perisaccadic shifts in receptive field location and perisaccadic shifts of attentional focus, review and synthesize behavioral and neurophysiological studies of perisaccadic perception and perisaccadic attention, and identify open questions that remain to be experimentally addressed.

Dissociation between Emotional Remapping of Fear and Disgust in Alexithymia.

There is growing evidence that individuals are able to understand others’ emotions because they “embody” them, i.e., re-experience them by activating a representation of the observed emotion within their own body. One way to study emotion embodiment is provided by a multisensory stimulation paradigm called emotional visual remapping of touch (eVRT), in which the degree of embodiment/remapping of emotions is measured as enhanced detection of near-threshold tactile stimuli on one’s own face while viewing different emotional facial expressions. Here, we measured remapping of fear and disgust in participants with low (LA) and high (HA) levels of alexithymia, a personality trait characterized by a difficulty in recognizing emotions. The results showed that fear is remapped in LA but not in HA participants, while disgust is remapped in HA but not in LA participants. To investigate the hypothesis that HA might exhibit increased responses to emotional stimuli producing a heightened physical and visceral sensations, i.e., disgust, in a second experiment we investigated participants’ interoceptive abilities and the link between interoception and emotional modulations of VRT. The results showed that participants’ disgust modulations of VRT correlated with their ability to perceive bodily signals. We suggest that the emotional profile of HA individuals on the eVRT task could be related to their abnormal tendency to be focalized on their internal bodily signals, and to experience emotions in a “physical” way. Finally, we speculated that these results in HA could be due to a enhancement of insular activity during the perception of disgusted faces.

What is visual remapping to saccade adaptation, a cause or a consequence?

What is visual remapping to saccade adaptation, a cause or a consequence?

I feel what you feel if I like you: the effect of attractiveness on visual remapping of touch.

Observing touch being applied to another human's face enhances tactile perception for touch being applied to one's own face. This effect, termed the Visual Remapping of Touch (VRT), is maximal the greater the physical or conceptual similarity between observer and observed. An interesting possibility, however, is that even though the basic nature of the VRT is multisensory, a high cognitive level affinity from the observer toward the observed could modulate the VRT even in the face of decreased physical similarity. In the present study we manipulate the level of attractiveness of the avatars that participants observed being touched. By doing so, we either increased (attractive) or decreased (unattractive) the interpersonal judgment value toward the avatar, while always decreasing the physical semblance between the avatar shown and the original image. Results revealed that both for an avatar depicting oneself or a stranger, the VRT is present when touch is applied to an attractive, but not to an unattractive avatar. These findings suggest that basic multisensory effects, such as visuo-tactile interaction, are modulated by higher-level cognitive representations of the self and of others.

A new perceptual paradigm to investigate the visual remapping of others’ tactile sensations onto one’s own body shows “mirror touch” for the hands

The last two decades have seen a multitude of publications showing the activation of an observer’s somatosensory cortical system during the observation of touch on another person. Behavioral demonstrations of “mirror touch,” however, have been slow in coming forward, and have so far primarily been shown as “visual remapping of touch” on the face. The present study uses a new paradigm to investigate the mirroring of others’ tactile sensations: a 2-AFC task of intensity judgment for touch on the observer’s left and right index finger pads. Observers viewed a left and right hand in an egocentric position, which were either touched passively (pencil moving to touch index finger pad) or actively sought touch (index finger moving to touch pencil). Touch and no-touch events for the two viewed hands were designed to eliminate confounding effects of spatial attention. Felt touches were either concurrent with viewed touch or no-touch events, or were delayed in time to assess potential response bias. The findings demonstrate visual remapping of touch for touch on the hands. If touch was shown on one of the hands only (e.g., left), observers were more likely to perceive touch on the same hand (i.e., their own left hand) as more intense than touch on the other hand even if tactile intensities did not differ, compared to touch shown on both or neither hand. These remapping effects occurred only when viewed and felt touches were concurrent, they were strongly modulated by the way in which viewed touch was incurred, and they were more reliable for touch on the left hand. A second, control experiment, in which touch observation was replaced by bright dots shown on or next to the finger pads, confirmed that these effects were largely due to genuine tactile mirroring rather than to somatotopic cueing. This 2-AFC tactile intensity judgment task may be a useful paradigm to investigate the remapping of others’ tactile sensations onto an observer’s own body.

Emotional modulation of touch in alexithymia.

Alexithymia refers to difficulties in recognizing one's own emotions, but difficulties have also been found in the recognition of others' emotions, particularly when the task is not easy. Previous research has demonstrated that, in order to understand other peoples' feelings, observers remap the observed emotion onto their own sensory systems. The aim of the present study was to investigate the ability of high and low alexithymic subjects to remap the emotional expressions of others onto their own somatosensory systems using an indirect task. We used the emotional Visual Remapping of Touch (eVRT) paradigm, in which seeing a face being touched improves detection of near-threshold tactile stimulation concurrently delivered to one's own face. In eVRT, subjects performance is influenced by the emotional content of the stimuli, while they were required to distinguish between unilateral or bilateral tactile stimulation on their own cheeks. The results show that tactile perception was enhanced when viewing touch on a fearful face compared with viewing touch on other expressions in low but not in high alexithymic participants. A negative correlation between TAS-20 alexithymia subscale ("difficulty in identify feelings") and the magnitude of the eVRT effect was also found. Conversely, arousal and valence ratings of emotional faces did not vary as a function of the degree of alexithymia. The results provide evidence that alexithymia is associated with difficulties in remapping seen emotions, particularly fear, onto one's own sensory system. This impairment could be due to an inability to modulate somatosensory system activity according to the observed emotional expression.

Observed touch on a non-human face is not remapped onto the human observer's own face.

Visual remapping of touch (VRT) is a phenomenon in which seeing a human face being touched enhances detection of tactile stimuli on the observer's own face, especially when the observed face expresses fear. This study tested whether VRT would occur when seeing touch on monkey faces and whether it would be similarly modulated by facial expressions. Human participants detected near-threshold tactile stimulation on their own cheeks while watching fearful, happy, and neutral human or monkey faces being concurrently touched or merely approached by fingers. We predicted minimal VRT for neutral and happy monkey faces but greater VRT for fearful monkey faces. The results with human faces replicated previous findings, demonstrating stronger VRT for fearful expressions than for happy or neutral expressions. However, there was no VRT (i.e. no difference between accuracy in touch and no-touch trials) for any of the monkey faces, regardless of facial expression, suggesting that touch on a non-human face is not remapped onto the somatosensory system of the human observer.

Multisensory integration across exteroceptive and interoceptive domains modulates self-experience in the rubber-hand illusion

Identifying with a body is central to being a conscious self. The now classic “rubber hand illusion” demonstrates that the experience of body-ownership can be modulated by manipulating the timing of exteroceptive (visual and tactile) body-related feedback. Moreover, the strength of this modulation is related to individual differences in sensitivity to internal bodily signals (interoception). However the interaction of exteroceptive and interoceptive signals in determining the experience of body-ownership within an individual remains poorly understood. Here, we demonstrate that this depends on the online integration of exteroceptive and interoceptive signals by implementing an innovative “cardiac rubber hand illusion” that combined computer-generated augmented-reality with feedback of interoceptive (cardiac) information. We show that both subjective and objective measures of virtual-hand ownership are enhanced by cardio-visual feedback in-time with the actual heartbeat, as compared to asynchronous feedback. We further show that these measures correlate with individual differences in interoceptive sensitivity, and are also modulated by the integration of proprioceptive signals instantiated using real-time visual remapping of finger movements to the virtual hand. Our results demonstrate that interoceptive signals directly influence the experience of body ownership via multisensory integration, and they lend support to models of conscious selfhood based on interoceptive predictive coding.

It feels like it’s me: Interpersonal multisensory stimulation enhances visual remapping of touch from other to self.

Understanding other people's feelings in social interactions depends on the ability to map onto our body the sensory experiences we observed on other people's bodies. It has been shown that the perception of tactile stimuli on the face is improved when concurrently viewing a face being touched. This Visual Remapping of Touch (VRT) is enhanced the more similar others are perceived to be to the self and is strongest when viewing one's face. Here, we ask whether altering self-other boundaries can in turn change the VRT effect. We used the enfacement illusion, which relies on synchronous interpersonal multisensory stimulation (IMS), to manipulate self-other boundaries. Following synchronous, but not asynchronous, IMS, the self-related enhancement of the VRT extended to the other individual. These findings suggest that shared multisensory experiences represent one key way to overcome the boundaries between self and others, as evidenced by changes in somatosensory processing of tactile stimuli on one's own face when concurrently viewing another person's face being touched.

Seeing and feeling for self and other: Proprioceptive spatial location determines multisensory enhancement of touch

We have investigated the relation between visuo-tactile interactions and the self-other distinction. In the Visual Enhancement of Touch (VET) effect, non-informative vision of one’s own hand improves tactile spatial perception. Previous studies suggested that looking at another person’s hand could also enhance tactile perception, but did not systematically investigate the basis of this effect. In Experiment 1 we manipulated the spatial location where one’s own or another person’s hand was seen. Viewing one’s own hand enhanced tactile orientation discrimination relative to viewing a neutral object, but only when the visual image of the hand was spatially aligned with the actual location of the participant’s unseen hand, as signaled by proprioception. In contrast, viewing another person’s hand produced enhanced tactile perception irrespective of spatial location.In Experiment 2, we used a multisensory stimulation technique, known as Visual Remapping of Touch (VRT), to reduce the spatial misalignment between the visual and proprioceptive locations of the hand. Participants saw an image of their own hand being touched at the same time as the tactile stimulation, which reduces perceived misalignment. This spatial adaptation procedure caused the VET effect to return. Our results suggest that multisensory modulation of touch depends on a representation of one’s own body that is fundamentally spatial in nature. In contrast, representation of others is free from this spatial constraint.

Embodying an outgroup: the role of racial bias and the effect of multisensory processing in somatosensory remapping

We come to understand other people's physical and mental states by re-mapping their bodily states onto our sensorimotor system. This process, also called somatosensory resonance, is an essential ability for social cognition and is stronger when observing ingroup than outgroup members. Here we investigated, first, whether implicit racial bias constrains somatosensory resonance, and second, whether increasing the ingroup/outgroup perceived physical similarity results in an increase in the somatosensory resonance for outgroup members. We used the Visual Remapping of Touch effect as an index of individuals' ability in resonating with the others, and the Implicit Association Test to measure racial bias. In Experiment 1, participants were asked to detect near-threshold tactile stimuli delivered to their own face while viewing either an ingroup or an outgroup face receiving a similar stimulation. Our results showed that individuals' tactile accuracy when viewing an outgroup face being touched was negatively correlated to their implicit racial bias. In Experiment 2, participants received the interpersonal multisensory stimulation (IMS) while observing an outgroup member. IMS has been found to increase the perceived physical similarity between the observer's and the observed body. We tested whether such increase in ingroup/outgroup perceived physical similarity increased the remapping ability for outgroup members. We found that after sharing IMS experience with an outgroup member, tactile accuracy when viewing touch on outgroup faces increased. Interestingly, participants with stronger implicit bias against the outgroup showed larger positive change in the remapping. We conclude that shared multisensory experiences might represent one key way to improve our ability to resonate with others by overcoming the boundaries between ingroup and outgroup categories.

It feels like it’s me: Interpersonal multisensory stimulation enhances visual remapping of touch from other to self

We constantly feel, see and move our body, and have no doubt that it is our own, distinct from the body of other people. At the same time, understanding other people’s feelings in social interactions depends on the ability to map onto one’s own body the observed experiences on the bodies of others. It has been shown that the more similar others are perceived to be to our self, the stronger this mapping is. While previous studies have focused on existing similarities or differences between self and other, we ask whether the experimental change of the self-other boundaries can lead to changes in somatosensory processing. It has been shown that the perception of tactile stimuli on the face is enhanced if participants concurrently observe a face being touched. This Visual Remapping of Touch (VRT; Ladavas and Serino, 2010) is enhanced the more similar others are perceived to be to the self, and is strongest when viewing one’s face (Serino et al., 2008, 2009). We used the enfacement illusion that relies on synchronous interpersonal multisensory stimulation (IMS; Sforza et al., 2010; Tsakiris, 2008) to manipulate self-other boundaries. Following synchronous, but not asynchronous IMS, the self-related enhancement of the VRT effect was extended to the other individual. These findings suggest that shared multi-sensory experiences represent one key way in which the boundaries and perceived similarity between self and others can be overcome, as evidenced by changes in somatosensory processing of shared tactile stimuli.