Coding In Visual Cortex Research Articles

A cortical zoom-in operation underlies covert shifts of visual spatial attention

Shifting the focus of attention without moving the eyes poses challenges for signal coding in visual cortex in terms of spatial resolution, signal routing, and cross-talk. Little is known how these problems are solved during focus shifts. Here, we analyze the spatiotemporal dynamic of neuromagnetic activity in human visual cortex as a function of the size and number of focus shifts in visual search. We find that large shifts elicit activity modulations progressing from highest (IT) through mid-level (V4) to lowest hierarchical levels (V1). Smaller shifts cause those modulations to start at lower levels in the hierarchy. Successive shifts involve repeated backward progressions through the hierarchy. We conclude that covert focus shifts arise from a cortical coarse-to-fine process progressing from retinotopic areas with larger toward areas with smaller receptive fields. This process localizes the target and increases the spatial resolution of selection, which resolves the above issues of cortical coding.

Predictive coding of visual object position ahead of moving objects revealed by time-resolved EEG decoding

Due to the delays inherent in neuronal transmission, our awareness of sensory events necessarily lags behind the occurrence of those events in the world. If the visual system did not compensate for these delays, we would consistently mislocalize moving objects behind their actual position. Anticipatory mechanisms that might compensate for these delays have been reported in animals, and such mechanisms have also been hypothesized to underlie perceptual effects in humans such as the Flash-Lag Effect. However, to date no direct physiological evidence for anticipatory mechanisms has been found in humans. Here, we apply multivariate pattern classification to time-resolved EEG data to investigate anticipatory coding of object position in humans. By comparing the time-course of neural position representation for objects in both random and predictable apparent motion, we isolated anticipatory mechanisms that could compensate for neural delays when motion trajectories were predictable. As well as revealing an early neural position representation (lag 80–90 ms) that was unaffected by the predictability of the object's trajectory, we demonstrate a second neural position representation at 140–150 ms that was distinct from the first, and that was pre-activated ahead of the moving object when it moved on a predictable trajectory. The latency advantage for predictable motion was approximately 16 ± 2 ms. To our knowledge, this provides the first direct experimental neurophysiological evidence of anticipatory coding in human vision, revealing the time-course of predictive mechanisms without using a spatial proxy for time. The results are numerically consistent with earlier animal work, and suggest that current models of spatial predictive coding in visual cortex can be effectively extended into the temporal domain.

Transformation from independent to integrative coding of multi-object arrangements in human visual cortex

To optimize processing, the human visual system utilizes regularities present in naturalistic visual input. One of these regularities is the relative position of objects in a scene (e.g., a sofa in front of a television), with behavioral research showing that regularly positioned objects are easier to perceive and to remember. Here we use fMRI to test how positional regularities are encoded in the visual system. Participants viewed pairs of objects that formed minimalistic two-object scenes (e.g., a “living room” consisting of a sofa and television) presented in their regularly experienced spatial arrangement or in an irregular arrangement (with interchanged positions). Additionally, single objects were presented centrally and in isolation. Multi-voxel activity patterns evoked by the object pairs were modeled as the average of the response patterns evoked by the two single objects forming the pair. In two experiments, this approximation in object-selective cortex was significantly less accurate for the regularly than the irregularly positioned pairs, indicating integration of individual object representations. More detailed analysis revealed a transition from independent to integrative coding along the posterior-anterior axis of the visual cortex, with the independent component (but not the integrative component) being almost perfectly predicted by object selectivity across the visual hierarchy. These results reveal a transitional stage between individual object and multi-object coding in visual cortex, providing a possible neural correlate of efficient processing of regularly positioned objects in natural scenes.

Linking pattern completion in the hippocampus to predictive coding in visual cortex.

Models of predictive coding frame perception as a generative process in which expectations constrain sensory representations. These models account for expectations about how a stimulus will move or change from moment to moment, but do not address expectations about what other, distinct stimuli are likely to appear based on prior experience. We show that such memory-based expectations in human visual cortex are related to the hippocampal mechanism of pattern completion.

Linking predictive coding in visual cortex to object representations in the medial temporal lobe

Linking predictive coding in visual cortex to object representations in the medial temporal lobe

Action-specific predictive coding of object states

Our actions can determine visual input by changing the state of objects in the environment. Because the future state of an object is often predictable based on its current state and a planned action, such actions provide a rich source of perceptual expectation. We used fMRI and a novel training paradigm to test whether associative learning can induce predictive coding in visual cortex of what an object will look like after an action. Action-outcome training consisted of an exploratory phase and a directed phase. Each trial of the exploratory training phase began with a stimulus cue (fractal pattern) in the middle of a computer screen, and the subject chose to press the left or right button in order to replace the cue with an outcome fractal. For each cue, a particular outcome appeared when the left button was pressed and a different outcome appeared when the right button was pressed. The directed training phase was used to balance the frequency of specific cue-outcome transitions, with arrows appearing below fractal cues to prompt left and right responses. After both training phases, we used fMRI to assess the neural correlates and consequences of predictive coding for outcomes based on actions in the scanner. As during directed training, each trial in the scanner included three parts: a cue fractal, an arrow prompt to quickly press the corresponding left or right button, and then an outcome fractal. We compared trials in which the outcome was predictable given the cue and action combination, to trials in which the outcome was associated with the cue but only expected after the other action, and found that the BOLD response in object-selective visual cortex was affected by learned action contingencies. Actions and intentions may thus adaptively influence visual perception of objects via predictive coding of forthcoming object states. Meeting abstract presented at VSS 2013

More Time to Taste. Focus on “Variability in Responses and Temporal Coding of Tastants of Similar Quality in the Nucleus of the Solitary Tract of the Rat”

The taste system is ideal for investigations of temporal coding, because neural taste responses are time extensive and often difficult to differentiate by magnitude alone. In a recent article, [Roussin et al. 2008][1] show that spike-time patterning provides a necessary and efficient source of

Network Variability Limits Stimulus-Evoked Spike Timing Precision in Retinal Ganglion Cells

Visual, auditory, somatosensory, and olfactory stimuli generate temporally precise patterns of action potentials (spikes). It is unclear, however, how the precision of spike generation relates to the pattern and variability of synaptic input elicited by physiological stimuli. We determined how synaptic conductances evoked by light stimuli that activate the rod bipolar pathway control spike generation in three identified types of mouse retinal ganglion cells (RGCs). The relative amplitude, timing, and impact of excitatory and inhibitory input differed dramatically between On and Off RGCs. Spikes evoked by repeated somatic injection of identical light-evoked synaptic conductances were more temporally precise than those evoked by light. However, the precision of spikes evoked by conductances that varied from trial to trial was similar to that of light-evoked spikes. Thus, the rod bipolar pathway modulates different RGCs via unique combinations of synaptic input, and RGC temporal variability reflects variability in the input this circuit provides.

Nonlinear and higher-order approaches to the encoding of natural scenes

Linear operations can only partially exploit the statistical redundancies of natural scenes, and nonlinear operations are ubiquitous in visual cortex. However, neither the detailed function of the nonlinearities nor the higher-order image statistics are yet fully understood. We suggest that these complicated issues can not be tackled by one single approach, but require a range of methods, and the understanding of the crosslinks between the results. We consider three basic approaches: (i) State space descriptions can theoretically provide complete information about statistical properties and nonlinear operations, but their practical usage is confined to very low-dimensional settings. We discuss the use of representation-related state-space coordinates (multivariate wavelet statistics) and of basic nonlinear coordinate transformations of the state space (e.g., a polar transform). (ii) Indirect methods, like unsupervised learning in multi-layer networks, provide complete optimization results, but no direct information on the statistical properties, and no simple model structures. (iii) Approximation by lower-order terms of power-series expansions is a classical strategy that has not yet received broad attention. On the statistical side, this approximation amounts to cumulant functions and higher-order spectra (polyspectra), on the processing side to Volterra–Wiener systems. In this context we suggest that an important concept for the understanding of natural scene statistics, of nonlinear neurons, and of biological pattern recognition can be found in AND-like combinations of frequency components. We investigate how the different approaches can be related to each other, how they can contribute to the understanding of cortical nonlinearities such as complex cells, cortical gain control, end-stopping and other extraclassical receptive field properties, and how we can obtain a nonlinear perspective on overcomplete representations and invariant coding in visual cortex.