Receptive Field Shifts Research Articles

Spike-timing dependent plasticity partially compensates for neural delays in a multi-layered network of motion-sensitive neurons.

The ability of the brain to represent the external world in real-time is impacted by the fact that neural processing takes time. Because neural delays accumulate as information progresses through the visual system, representations encoded at each hierarchical level are based upon input that is progressively outdated with respect to the external world. This 'representational lag' is particularly relevant to the task of localizing a moving object-because the object's location changes with time, neural representations of its location potentially lag behind its true location. Converging evidence suggests that the brain has evolved mechanisms that allow it to compensate for its inherent delays by extrapolating the position of moving objects along their trajectory. We have previously shown how spike-timing dependent plasticity (STDP) can achieve motion extrapolation in a two-layer, feedforward network of velocity-tuned neurons, by shifting the receptive fields of second layer neurons in the opposite direction to a moving stimulus. The current study extends this work by implementing two important changes to the network to bring it more into line with biology: we expanded the network to multiple layers to reflect the depth of the visual hierarchy, and we implemented more realistic synaptic time-courses. We investigate the accumulation of STDP-driven receptive field shifts across several layers, observing a velocity-dependent reduction in representational lag. These results highlight the role of STDP, operating purely along the feedforward pathway, as a developmental strategy for delay compensation.

Tuning curves vs. population responses, and perceptual consequences of receptive-field remapping.

Sensory processing is often studied by examining how a given neuron responds to a parameterized set of stimuli (tuning curve) or how a given stimulus evokes responses from a parameterized set of neurons (population response). Although tuning curves and the corresponding population responses contain the same information, they can have different properties. These differences are known to be important because the perception of a stimulus should be decoded from its population response, not from any single tuning curve. The differences are less studied in the spatial domain where a cell's spatial tuning curve is simply its receptive field (RF) profile. Here, we focus on evaluating the common belief that perrisaccadic forward and convergent RF shifts lead to forward (translational) and convergent (compressive) perceptual mislocalization, respectively, and investigate the effects of three related factors: decoders' awareness of RF shifts, changes of cells' covering density near attentional locus (the saccade target), and attentional response modulation. We find that RF shifts alone produce either no shift or an opposite shift of the population responses depending on whether or not decoders are aware of the RF shifts. Thus, forward RF shifts do not predict forward mislocalization. However, convergent RF shifts change cells' covering density for aware decoders (but not for unaware decoders) which may predict convergent mislocalization. Finally, attentional modulation adds a convergent component to population responses for stimuli near the target. We simulate the combined effects of these factors and discuss the results with extant mislocalization data. We speculate that perisaccadic mislocalization might be the flash-lag effect unrelated to perisaccadic RF remapping but to resolve the issue, one has to address the question of whether or not perceptual decoders are aware of RF shifts.

Muscles that move the retina augment compound eye vision in Drosophila.

Most animals have compound eyes, with tens to thousands of lenses attached rigidly to the exoskeleton. A natural assumption is that all of these species must resort to moving either their head ortheir body to actively change their visual input. However, classic anatomy has revealed that flies have muscles poised to move their retinas under the stable lenses of each compound eye1-3. Here we show that Drosophila use their retinal muscles to smoothly track visual motion, which helps to stabilize the retinal image, and also to perform small saccades when viewing a stationary scene. We show that when the retina moves, visual receptive fields shift accordingly, and that even the smallest retinal saccades activate visual neurons. Using a head-fixed behavioural paradigm, we find that Drosophila perform binocular, vergence movements of their retinas-which could enhance depth perception-when crossing gaps, and impairing the physiology of retinal motor neurons alters gap-crossing trajectories during free behaviour. That flies evolved an ability to actuate their retinas suggests that moving the eye independently of the head is broadly paramount for animals. The similarities of smooth and saccadic movements of the Drosophila retina and the vertebrate eye highlight a notable example of convergent evolution.

Microsaccades, Drifts, Hopf Bundle and Neurogeometry

The first part of the paper contains a short review of the image processing in early vision is static, when the eyes and the stimulus are stable, and in dynamics, when the eyes participate in fixation eye movements. In the second part, we give an interpretation of Donders’ and Listing’s law in terms of the Hopf fibration of the 3-sphere over the 2-sphere. In particular, it is shown that the configuration space of the eye ball (when the head is fixed) is the 2-dimensional hemisphere , called Listing hemisphere, and saccades are described as geodesic segments of with respect to the standard round metric. We study fixation eye movements (drift and microsaccades) in terms of this model and discuss the role of fixation eye movements in vision. A model of fixation eye movements is proposed that gives an explanation of presaccadic shift of receptive fields.

Behavioral benefits of spatial attention explained by multiplicative gain, not receptive field shifts, in a neural network model.

Behavioral benefits of spatial attention explained by multiplicative gain, not receptive field shifts, in a neural network model.

Mexican Hat Modulation of Visual Acuity Following an Exogenous Cue.

Classical models of exogenous attention suggest that attentional enhancement at the focus of attention degrades gradually with distance from the attended location. On the other hand, the Attentional Attraction Field (AAF) model (Baruch and Yeshurun, 2014) suggests that the shift of receptive fields toward the attended location, reported by several physiological studies, leads to a decreased density of RFs at the attentional surrounds and hence the model predicts that the modulation of performance by spatial attention may have the shape of a Mexican Hat. Motivated by these theories, this study presents behavioral evidence in support of a Mexican Hat shaped modulation in exogenous spatial tasks that appears only at short latencies. In two experiments participants had to decide the location of a small gap in a target circle that was preceded by a non-informative attention capturing cue. The distance between cue and target and the latency between their onsets were varied. At short SOAs the performance curves were cubic and only at longer SOAs- this trend turned linear. Our results suggest that a rapid Mexican Hat modulation is an inherent property of the mechanism underlying exogenous attention and that a monotonically degrading trend, such as advocated by classical models, develops only at later stages of processing. The involvements of bottom-up processes such as the attraction of RFs to the focus of attention are further discussed.

Spatial receptive field shift by preceding cross-modal stimulation in the cat superior colliculus.

It has been known for some time that sensory information of one type can bias the spatial perception of another modality. However, there is a lack of evidence of this occurring in individual neurons. In the present study, we found that the spatial receptive field of superior colliculus multisensory neurons could be dynamically shifted by a preceding stimulus in a different modality. The extent to which the receptive field shifted was dependent on both temporal and spatial gaps between the preceding and following stimuli, as well as the salience of the preceding stimulus. This result provides a neural mechanism that could underlie the process of cross-modal spatial calibration. Psychophysical studies have shown that the different senses can be spatially entrained by each other. This can be observed in certain phenomena, such as ventriloquism, in which a visual stimulus can attract the perceived location of a spatially discordant sound. However, the neural mechanism underlying this cross-modal spatial recalibration has remained unclear, as has whether it takes place dynamically. We explored these issues in multisensory neurons of the cat superior colliculus (SC), a midbrain structure that involves both cross-modal and sensorimotor integration. Sequential cross-modal stimulation showed that the preceding stimulus can shift the receptive field (RF) of the lagging response. This cross-modal spatial calibration took place in both auditory and visual RFs, although auditory RFs shifted slightly more. By contrast, if a preceding stimulus was from the same modality, it failed to induce a similarly substantial RF shift. The extent of the RF shift was dependent on both temporal and spatial gaps between the preceding and following stimuli, as well as the salience of the preceding stimulus. A narrow time gap and high stimulus salience were able to induce larger RF shifts. In addition, when both visual and auditory stimuli were presented simultaneously, a substantial RF shift toward the location-fixed stimulus was also induced. These results, taken together, reveal an online cross-modal process and reflect the details of the organization of SC inter-sensory spatial calibration.

Two Types of Receptive Field Dynamics in Area V4 at the Time of Eye Movements?

How we perceive the world as stable despite the frequent disruptions of the retinal image caused by eye movements is one of the fundamental questions in sensory neuroscience. Seemingly convergent evidence points towards a mechanism which dynamically updates representations of visual space in anticipation of a movement (Wurtz, 2008). In particular, receptive fields (RFs) of neurons, predominantly within oculomotor and attention related brain structures (Duhamel et al., 1992; Walker et al., 1995; Umeno and Goldberg, 1997), are thought to “remap” to their future, post-movement location prior to an impending eye movement. New studies (Neupane et al., 2016a,b) report observations on RF dynamics at the time of eye movements of neurons in area V4. These dynamics are interpreted as being largely dominated by a remapping of RFs. Critically, these observations appear at odds with a previous study reporting a different type of RF dynamics within the same brain structure (Tolias et al., 2001), consisting of a shrinkage and shift of RFs towards the movement target. Importantly, RFs have been measured with different techniques in those studies. Here, we measured V4 RFs comparable to Neupane et al. (2016a,b) and observe a shrinkage and shift of RFs towards the movement target when analyzing the immediate stimulus response (Zirnsak et al., 2014). When analyzing the late stimulus response (Neupane et al., 2016a,b), we observe RF shifts resembling remapping. We discuss possible causes for these shifts and point out important issues which future studies on RF dynamics need to address.

Plastic changes along auditory pathway during salicylate-induced ototoxicity: Hyperactivity and CF shifts

High dose of salicylate, the active ingredient in aspirin, has long been known to induce transient hearing loss, tinnitus and hyperacusis making it a powerful experimental tool. These salicylate-induced perceptual disturbances are associated with a massive reduction in the neural output of the cochlea. Paradoxically, the diminished neural output of the cochlea is accompanied by a dramatic increase in sound-evoked activity in the auditory cortex (AC) and several other parts of the central nervous system. Exactly where the increase in neural activity begins and builds up along the central auditory pathway are not fully understood. To address this issue, we measured sound-evoked neural activity in the cochlea, cochlear nucleus (CN), inferior colliculus (IC), and AC before and after administering a high dose of sodium salicylate (SS, 300 mg/kg). The SS-treatment abolished low-level sound-evoked responses along the auditory pathway resulting in a 20–30 dB threshold shift. While the neural output of the cochlea was substantially reduced at high intensities, the neural responses in the CN were only slightly reduced; those in the IC were nearly normal or slightly enhanced while those in the AC considerably enhanced, indicative of a progress increase in central gain. The SS-induced increase in central response in the IC and AC was frequency-dependent with the greatest increase occurring in the mid-frequency range the putative pitch of SS-induced tinnitus. This frequency-dependent hyperactivity appeared to result from shifts in the frequency receptive fields (FRF) such that the response areas of many FRF shifted/expanded toward the mid-frequencies. Our results suggest that the SS-induced threshold shift originates in the cochlea. In contrast, enhanced central gain is not localized to one region, but progressively builds up at successively higher stage of the auditory pathway either through a loss of inhibition and/or increased excitation.

Saccade Preparation Reshapes Sensory Tuning

Human observers make large rapid eye movements-saccades-to bring behaviorally relevant information into the fovea, where spatial resolution is high. In some visual tasks [1-4], performance at the location of a saccade target improves before the eyes move. Although these findings provide evidence that extra-retinal signals evoked by saccades can enhance visual perception, it remains unknown whether and how presaccadic modulations change the processing of feature information and thus modulate visual representations. To answer this question, one must go beyond the use of methods that only probe performance accuracy (d') in different tasks. Here, using a psychophysical reverse correlation approach [5-8], we investigated how saccade preparation influences the processing of orientation and spatial frequency-two building blocks of early vision. We found that saccade preparation selectively enhanced the gain of high spatial frequency information and narrowed orientation tuning at the upcoming saccade landing position. These modulations were time locked to saccade onset, peaking right before the eyes moved (-50-0ms). Moreover, merely deploying covert attention within the same temporal interval without preparing a saccade did not alter performance. The observed presaccadic tuning changes may correspond to the presaccadic enhancement [9-11] and receptive field shifts reported in neurophysiological studies [12-14]. Saccade preparation may support transaccadic integration by reshaping the representation of the saccade target to be more fovea-like just before theeyes move. The presaccadic modulations on spatial frequency and orientation processing illustrate a strong perception-action coupling by revealing that the visual system dynamically reshapes feature selectivity contingent upon eye movements.

A Probabilistic Approach to Receptive Field Mapping in the Frontal Eye Fields.

Studies of the neuronal mechanisms of perisaccadic vision often lack the resolution needed to determine important changes in receptive field (RF) structure. Such limited analytical power can lead to inaccurate descriptions of visuomotor processing. To address this issue, we developed a precise, probabilistic technique that uses a generalized linear model (GLM) for mapping the visual RFs of frontal eye field (FEF) neurons during stable fixation (Mayo et al., 2015). We previously found that full-field RF maps could be obtained using 1–8 dot stimuli presented at frame rates of 10–150 ms. FEF responses were generally robust to changes in the number of stimuli presented or the rate of presentation, which allowed us to visualize RFs over a range of spatial and temporal resolutions. Here, we compare the quality of RFs obtained over different stimulus and GLM parameters to facilitate future work on the detailed mapping of FEF RFs. We first evaluate the interactions between the number of stimuli presented per trial, the total number of trials, and the quality of RF mapping. Next, we vary the spatial resolution of our approach to illustrate the tradeoff between visualizing RF sub-structure and sampling at high resolutions. We then evaluate local smoothing as a possible correction for situations where under-sampling occurs. Finally, we provide a preliminary demonstration of the usefulness of a probabilistic approach for visualizing full-field perisaccadic RF shifts. Our results present a powerful, and perhaps necessary, framework for studying perisaccadic vision that is applicable to FEF and possibly other visuomotor regions of the brain.

A Feedback Model of Attention Explains the Diverse Effects of Attention on Neural Firing Rates and Receptive Field Structure.

Visual attention has many effects on neural responses, producing complex changes in firing rates, as well as modifying the structure and size of receptive fields, both in topological and feature space. Several existing models of attention suggest that these effects arise from selective modulation of neural inputs. However, anatomical and physiological observations suggest that attentional modulation targets higher levels of the visual system (such as V4 or MT) rather than input areas (such as V1). Here we propose a simple mechanism that explains how a top-down attentional modulation, falling on higher visual areas, can produce the observed effects of attention on neural responses. Our model requires only the existence of modulatory feedback connections between areas, and short-range lateral inhibition within each area. Feedback connections redistribute the top-down modulation to lower areas, which in turn alters the inputs of other higher-area cells, including those that did not receive the initial modulation. This produces firing rate modulations and receptive field shifts. Simultaneously, short-range lateral inhibition between neighboring cells produce competitive effects that are automatically scaled to receptive field size in any given area. Our model reproduces the observed attentional effects on response rates (response gain, input gain, biased competition automatically scaled to receptive field size) and receptive field structure (shifts and resizing of receptive fields both spatially and in complex feature space), without modifying model parameters. Our model also makes the novel prediction that attentional effects on response curves should shift from response gain to contrast gain as the spatial focus of attention drifts away from the studied cell.

Perisaccadic Updating of Visual Representations and Attentional States: Linking Behavior and Neurophysiology.

During natural vision, saccadic eye movements lead to frequent retinal image changes that result in different neuronal subpopulations representing the same visual feature across fixations. Despite these potentially disruptive changes to the neural representation, our visual percept is remarkably stable. Visual receptive field remapping, characterized as an anticipatory shift in the position of a neuron’s spatial receptive field immediately before saccades, has been proposed as one possible neural substrate for visual stability. Many of the specific properties of remapping, e.g., the exact direction of remapping relative to the saccade vector and the precise mechanisms by which remapping could instantiate stability, remain a matter of debate. Recent studies have also shown that visual attention, like perception itself, can be sustained across saccades, suggesting that the attentional control system can also compensate for eye movements. Classical remapping could have an attentional component, or there could be a distinct attentional analog of visual remapping. At this time we do not yet fully understand how the stability of attentional representations relates to perisaccadic receptive field shifts. In this review, we develop a vocabulary for discussing perisaccadic shifts in receptive field location and perisaccadic shifts of attentional focus, review and synthesize behavioral and neurophysiological studies of perisaccadic perception and perisaccadic attention, and identify open questions that remain to be experimentally addressed.

Adult cortical plasticity studied with chronically implanted electrode arrays.

The functional architecture of adult cerebral cortex retains a capacity for experience-dependent change. This is seen after focal binocular lesions as rapid changes in receptive field (RF) of the lesion projection zone (LPZ) in the primary visual cortex (V1). To study the dynamics of the circuitry underlying these changes longitudinally, we implanted microelectrode arrays in macaque (Macaca mulatta) V1, eliminating the possibility of sampling bias, which was a concern in previous studies. With this method, we observed a rapid initial recovery in the LPZ and, during the following weeks, 63-89% of the sites in the LPZ showed recovery of visual responses with significant position tuning. The RFs shifted ∼3° away from the scotoma. In the absence of a lesion, visual stimulation surrounding an artificial scotoma did not elicit visual responses, suggesting that the postlesion RF shifts resulted from cortical reorganization. Interestingly, although both spikes and LFPs gave consistent prelesion position tuning, only spikes reflected the postlesion remapping.

Attentional attraction of receptive fields can explain spatial and temporal effects of attention

A multitude of attentional effects were found at both the neural and behavioural levels of perception; however, the nature of the attentional mechanism is an unsettled issue. It is typically assumed that the fundamental impact of attention is enhancement of neuronal responses, but some attentional effects are difficult to explain by such gain modulations. Here we offer a different view: We suggest that the most fundamental impact of attention is the attraction (shift) of receptive fields towards the attended location. We further show, both conceptually and by model simulations, that this attraction of receptive fields can explain a diverse range of attentional effects, spatial as well as temporal, linking physiological measurements at the unit level with psychophysical observations (e.g., enhanced contrast sensitivity, enhanced spatial resolution, the Mexican-hat profile of attention, prolonged perceived duration, prior entry, degraded temporal resolution).

Brain circuits underlying visual stability across eye movements—converging evidence for a neuro-computational model of area LIP

The understanding of the subjective experience of a visually stable world despite the occurrence of an observer's eye movements has been the focus of extensive research for over 20 years. These studies have revealed fundamental mechanisms such as anticipatory receptive field (RF) shifts and the saccadic suppression of stimulus displacements, yet there currently exists no single explanatory framework for these observations. We show that a previously presented neuro-computational model of peri-saccadic mislocalization accounts for the phenomenon of predictive remapping and for the observation of saccadic suppression of displacement (SSD). This converging evidence allows us to identify the potential ingredients of perceptual stability that generalize beyond different data sets in a formal physiology-based model. In particular we propose that predictive remapping stabilizes the visual world across saccades by introducing a feedback loop and, as an emergent result, small displacements of stimuli are not noticed by the visual system. The model provides a link from neural dynamics, to neural mechanism and finally to behavior, and thus offers a testable comprehensive framework of visual stability.

Visual Stability During Saccades is Achieved through Transient Changes in Perceptual Space and Time

To interact rapidly and effectively with our environment, our brain needs access to a dynamic neural representation – or map – of the spatial layout of the external world. One of the major challenges to constructing such a map is the frequent rapid movements of the eyes, which displace the images on our retinae, up to three times per second. Much neurophysiological research shows that just before saccades, the receptive fields of neurons in many visual areas are subject to large shifts, in the direction of the saccade (Duhamel, Colby, & Goldberg, 1992). While the neural mechanisms of this dynamic process are now well understood (Sommer & Wurtz, 2006), it is less clear how a shift in the direction of the saccade benefits perception.We suggest that the purpose of the shifts in receptive fields is to prepare for the effects of saccades before they commence, and that the real work of stabilizing occurs when they relax back into their standard position during the actual saccade (Burr & Morrone, 2011, 2012; Cicchini, Binda, Burr, & Morrone, 2012). It is this relaxation that compensates for the saccadic displacement. The relaxation process is best described as a shift in both space and in time, and this can be well described by spatiotemporal receptive field that is slanted in space-time (see Figure 9 from Cicchini, Binda, Burr, & Morrone, 2012).We have accumulated a wealth of psychophysical evidence supporting the notion of the spatiotemporal shift. Perhaps the strongest evidence is the temporal lag that occurs during saccades, a clear signature of a spatiotemporal transformation. During saccades, visual flashes are perceived to be delayed by 100ms compared with normal viewing (Binda, Cicchini, Burr, & Morrone, 2009), and it is this delay that creates the spatiotemporal tuning (as do motion perception mechanisms, on a different timescale; Burr, Ross, & Morrone, 1986). There is also strong temporal compression, so stimuli 100ms long are perceived as 50ms (Morrone, Ross, & Burr, 2005); we have developed a simple model that explains how the temporal compression results from a spatiotemporal shift (Cicchini & Morrone, 2012). Recently, neurophysiological studies find similar delays and compression in neurones of cortical area FEF (Joiner, Cavanaugh, & Wurtz, 2013). Finally, we have explicitly measured the spatiotemporal interaction field of stimuli flashed around the time of the saccade, and report a field like that shown in Figure 9 of Cicchini, Binda, Burr, and Morrone (2012).All up these studies show clear evidence for a spatiotemporal mechanism that creates transient spatiotopy at the time of saccades, allowing perceptual systems to momentarily bridge the transition between one fixation and another. This process cannot be understood as only a spatial transform, as it is inherently spatiotemporal, resulting in strong and measureable delays and compression of time.

Frequency-specific plasticity of the auditory cortex elicited by thalamic stimulation in the rat

In a process known as frequency-specific plasticity, electrical stimulation of the ventral division of the medial geniculate body (MGBv) in the thalamus evokes a shift in the frequency-tuning curves of auditory cortical (AC) neurons toward the best frequency (BF) of stimulated MGBv neurons. However, the underlying synaptic mechanisms of this process are uncharacterized. To investigate whether this dynamic change depends on thalamocortical (TC) synaptic plasticity, we studied frequency-specific changes in synaptic transmission efficacy in TC pathways evoked by thalamic stimulation. Specifically, we induced cortical plasticity by repetitive focal electrical stimulation of the MGBv in rats and measured receptive field shifts and local field potentials in AC neurons. Our data show that focal electrical stimulation of the MGBv induced receptive field shifts as well as long-term potentiation or depression of the local field potentials in AC neurons. The evoked potentiation and depression depended on the frequency of the electrical stimulation of the MGBv synchronized with the BF of MGBv and AC neurons. Receptive field shifts were produced by inhibition of responses at the BF of the recorded AC neurons and facilitation of responses at the BF of the stimulated MGBv neurons. These results suggest that MGBv neurons play a decisive role in the expression of AC synaptic plasticity and that activation of different frequency-specific TC pathways may be the synaptic mechanism underlying this plasticity.

Robot studies on saccade-triggered visual prediction

Three robot studies on visual prediction are presented. In all of them, a visual forward model is used, which predicts the visual consequences of saccade-like camera movements. This forward model works by remapping visual information between the pre- and postsaccadic retinal images; at an abstract modeling level, this process is closely related to neurons whose visual receptive fields shift in anticipation of saccades. In the robot studies, predictive remapping is used (1) in the context of saccade adaptation, to reidentify target objects after saccades are carried out; (2) for a model of grasping, in which both fixated and non-fixated target objects are processed by the same foveal mechanism; and (3) in a computational architecture for mental imagery, which generates “gripper appearances” internally without real sensory inflow. The robotic experiments and their underlying computational models are discussed with regard to predictive remapping in the brain, transsaccadic memory, and attention. The results confirm that visual prediction is a mechanism that has to be considered in the design of artificial cognitive agents and the modeling of information processing in the human visual system.

Anticipatory Saccade Target Processing and the Presaccadic Transfer of Visual Features

As we shift our gaze to explore the visual world, information enters cortex in a sequence of successive snapshots, interrupted by phases of blur. Our experience, in contrast, appears like a movie of a continuous stream of objects embedded in a stable world. This perception of stability across eye movements has been linked to changes in spatial sensitivity of visual neurons anticipating the upcoming saccade, often referred to as shifting receptive fields (Duhamel et al., 1992; Walker et al., 1995; Umeno and Goldberg, 1997; Nakamura and Colby, 2002). How exactly these receptive field dynamics contribute to perceptual stability is currently not clear. Anticipatory receptive field shifts toward the future, postsaccadic position may bridge the transient perisaccadic epoch (Sommer and Wurtz, 2006; Wurtz, 2008; Melcher and Colby, 2008). Alternatively, a presaccadic shift of receptive fields toward the saccade target area (Tolias et al., 2001) may serve to focus visual resources onto the most relevant objects in the postsaccadic scene (Hamker et al., 2008). In this view, shifts of feature detectors serve to facilitate the processing of the peripheral visual content before it is foveated. While this conception is consistent with previous observations on receptive field dynamics and on perisaccadic compression (Ross et al., 1997; Morrone et al., 1997; Kaiser and Lappe, 2004), it predicts that receptive fields beyond the saccade target shift toward the saccade target rather than in the direction of the saccade. We have tested this prediction in human observers via the presaccadic transfer of the tilt-aftereffect (Melcher, 2007).