Higher Visual Areas Research Articles

V1 neurons are tuned to perceptual borders in natural scenes

The visual system needs to identify perceptually relevant borders to segment complex natural scenes. The primary visual cortex (V1) is thought to extract local borders, and higher visual areas are thought to identify the perceptually relevant borders between objects and the background. To test this conjecture, we used natural images that had been annotated by human observers who marked the perceptually relevant borders. We assessed the effect of perceptual relevance on V1 responses using human neuroimaging, macaque electrophysiology, and computational modeling. We report that perceptually relevant borders elicit stronger responses in the early visual cortex than irrelevant ones, even if simple features, such as contrast and the energy of oriented filters, are matched. Moreover, V1 neurons discriminate perceptually relevant borders surprisingly fast, during the early feedforward-driven activity at a latency of ~50 ms, indicating that they are tuned to the features that characterize them. We also revealed a delayed, contextual effect that enhances the V1 responses that are elicited by perceptually relevant borders at a longer latency. Our results reveal multiple mechanisms that allow V1 neurons to infer the layout of objects in natural images.

Brightness illusions drive a neuronal response in the primary visual cortex under top-down modulation

Brightness illusions are a powerful tool in studying vision, yet their neural correlates are poorly understood. Based on a human paradigm, we presented illusory drifting gratings to mice. Primary visual cortex (V1) neurons responded to illusory gratings, matching their direction selectivity for real gratings, and they tracked the spatial phase offset between illusory and real gratings. Illusion responses were delayed compared to real gratings, in line with the theory that processing illusions requires feedback from higher visual areas (HVAs). We provide support for this theory by showing a reduced V1 response to illusions, but not real gratings, following HVAs optogenetic inhibition. Finally, we used the pupil response (PR) as an indirect perceptual report and showed that the mouse PR matches the human PR to perceived luminance changes. Our findings resolve debates over whether V1 neurons are involved in processing illusions and highlight the involvement of feedback from HVAs.

The role of pattern coherence in interocular grouping during binocular rivalry: Insights from individual differences

Interocular grouping during binocular rivalry occurs when two images presented to each eye combine into a coherent pattern. The experience of interocular grouping is thought to be influenced by both eye-of-origin, which involves excitatory lateral connections among monocular neurons, and pattern coherence, which results from top-down intervention from higher visual areas. However, it remains unclear which factor plays a more significant role in the interocularly-grouped percepts during binocular rivalry. The current study employed an individual difference approach to investigate whether grouping dynamics are mainly determined by eye-of-origin or pattern coherence. We found that participants who perceived interocularly-driven coherent percepts for a longer duration also tended to experience longer periods of monocularly-driven coherent percepts. In contrast, participants who experienced non-coherent piecemeal percepts for an extended duration in conventional rivalry also had longer duration of non-coherent percepts in the interocular coherence setting. This individual differences in experiencing interocular grouping suggest that pattern coherence exerts a stronger influence on grouping dynamics during binocular rivalry compared to eye-of-origin factors.

Time-varying generalized linear models: characterizing and decoding neuronal dynamics in higher visual areas.

To create a behaviorally relevant representation of the visual world, neurons in higher visual areas exhibit dynamic response changes to account for the time-varying interactions between external (e.g., visual input) and internal (e.g., reward value) factors. The resulting high-dimensional representational space poses challenges for precisely quantifying individual factors' contributions to the representation and readout of sensory information during a behavior. The widely used point process generalized linear model (GLM) approach provides a powerful framework for a quantitative description of neuronal processing as a function of various sensory and non-sensory inputs (encoding) as well as linking particular response components to particular behaviors (decoding), at the level of single trials and individual neurons. However, most existing variations of GLMs assume the neural systems to be time-invariant, making them inadequate for modeling nonstationary characteristics of neuronal sensitivity in higher visual areas. In this review, we summarize some of the existing GLM variations, with a focus on time-varying extensions. We highlight their applications to understanding neural representations in higher visual areas and decoding transient neuronal sensitivity as well as linking physiology to behavior through manipulation of model components. This time-varying class of statistical models provide valuable insights into the neural basis of various visual behaviors in higher visual areas and hold significant potential for uncovering the fundamental computational principles that govern neuronal processing underlying various behaviors in different regions of the brain.

Orientation representation in human visual cortices: contributions of non-visual information and action-related process.

Orientation processing in the human brain plays a crucial role in guiding grasping actions toward an object. Remarkably, despite the absence of visual input, the human visual cortex can still process orientation information. Instead of visual input, non-visual information, including tactile and proprioceptive sensory input from the hand and arm, as well as feedback from action-related processes, may contribute to orientation processing. However, the precise mechanisms by which the visual cortices process orientation information in the context of non-visual sensory input and action-related processes remain to be elucidated. Thus, our study examined the orientation representation within the visual cortices by analyzing the blood-oxygenation-level-dependent (BOLD) signals under four action conditions: direct grasp (DG), air grasp (AG), non-grasp (NG), and uninformed grasp (UG). The images of the cylindrical object were shown at +45° or - 45° orientations, corresponding to those of the real object to be grasped with the whole-hand gesture. Participants judged their orientation under all conditions. Grasping was performed without online visual feedback of the hand and object. The purpose of this design was to investigate the visual areas under conditions involving tactile feedback, proprioception, and action-related processes. To address this, a multivariate pattern analysis was used to examine the differences among the cortical patterns of the four action conditions in orientation representation by classification. Overall, significant decoding accuracy over chance level was discovered for the DG; however, during AG, only the early visual areas showed significant accuracy, suggesting that the object's tactile feedback influences the orientation process in higher visual areas. The NG showed no statistical significance in any area, indicating that without the grasping action, visual input does not contribute to cortical pattern representation. Interestingly, only the dorsal and ventral divisions of the third visual area (V3d and V3v) showed significant decoding accuracy during the UG despite the absence of visual instructions, suggesting that the orientation representation was derived from action-related processes in V3d and visual recognition of object visualization in V3v. The processing of orientation information during non-visually guided grasping of objects relies on other non-visual sources and is specifically divided by the purpose of action or recognition.

Experience dependent plasticity of higher visual cortical areas in the mouse.

Experience dependent plasticity in the visual cortex is a key paradigm for the study of mechanisms underpinning learning and memory. Despite this, studies involving manipulating visual experience have largely been limited to the primary visual cortex, V1, across various species. Here we investigated the effects of monocular deprivation (MD) on the ocular dominance (OD) and orientation selectivity of neurons in four visual cortical areas in the mouse: the binocular zone of V1 (V1b), the putative "ventral stream" area LM and the putative "dorsal stream" areas AL and PM. We employed two-photon calcium imaging to record neuronal responses in young adult mice before MD, immediately after MD, and following binocular recovery. OD shifts following MD were greatest in LM and smallest in AL and PM; in LM and AL, these shifts were mediated primarily through a reduction of deprived-eye responses, in V1b and LM through an increase in response through the non-deprived eye. The OD index recovered to pre-MD levels within 2weeks in V1 only. MD caused a reduction in orientation selectivity of deprived-eye responses in V1b and LM only. Our results suggest that changes in OD in higher visual areas are not uniformly inherited from V1.

Visuomotor interactions in the mouse forebrain mediated by extrastriate cortico-cortical pathways.

The mammalian visual system can be broadly divided into two functional processing pathways: a dorsal stream supporting visually and spatially guided actions, and a ventral stream enabling object recognition. In rodents, the majority of visual signaling in the dorsal stream is transmitted to frontal motor cortices via extrastriate visual areas surrounding V1, but exactly where and to what extent V1 feeds into motor-projecting visual regions is not well known. We employed a dual labeling strategy in male and female mice in which efferent projections from V1 were labeled anterogradely, and motor-projecting neurons in higher visual areas were labeled with retrogradely traveling adeno-associated virus (rAAV-retro) injected in M2. We characterized the labeling in both flattened and coronal sections of dorsal cortex and made high-resolution 3D reconstructions to count putative synaptic contacts in different extrastriate areas. The most pronounced colocalization V1 output and M2 input occurred in extrastriate areas AM, PM, RL and AL. Neurons in both superficial and deep layers in each project to M2, but high resolution volumetric reconstructions indicated that the majority of putative synaptic contacts from V1 onto M2-projecting neurons occurred in layer 2/3. These findings support the existence of a dorsal processing stream in the mouse visual system, where visual signals reach motor cortex largely via feedforward projections in anteriorly and medially located extrastriate areas.

Narrowband gamma oscillations propagate and synchronize throughout the mouse thalamocortical visual system

Oscillations of neural activity permeate sensory systems. In the visual system, broadband gamma oscillations (30-80Hz) are thought to act as a communication mechanism underlying perception. However, these oscillations show widely varying frequency and phase, providing constraints for coordinating spike timing across areas. Here, we examined Allen Brain Observatory data and performed causal experiments to show that narrowband gamma (NBG) oscillations (50-70Hz) propagate and synchronize throughout the awake mouse visual system. Lateral geniculate nucleus (LGN) neurons fired precisely relative to NBG phase in primary visual cortex (V1) and multiple higher visual areas (HVAs). NBG neurons across areas showed a higher likelihood of functional connectivity and stronger visual responses; remarkably, NBG neurons in LGN, preferring bright (ON) versus dark (OFF), fired at distinct NBG phases aligned across the cortical hierarchy. NBG oscillations may thus serve to coordinate spike timing across brain areas and facilitate communication of distinct visual features during perception.

Independent spatiotemporal effects of spatial attention and background clutter on human object location representations

Spatial attention helps us to efficiently localize objects in cluttered environments. However, the processing stage at which spatial attention modulates object location representations remains unclear. Here we investigated this question identifying processing stages in time and space in an EEG and fMRI experiment respectively. As both object location representations and attentional effects have been shown to depend on the background on which objects appear, we included object background as an experimental factor. During the experiments, human participants viewed images of objects appearing in different locations on blank or cluttered backgrounds while either performing a task on fixation or on the periphery to direct their covert spatial attention away or towards the objects. We used multivariate classification to assess object location information. Consistent across the EEG and fMRI experiment, we show that spatial attention modulated location representations during late processing stages (>150 ms, in middle and high ventral visual stream areas) independent of background condition. Our results clarify the processing stage at which attention modulates object location representations in the ventral visual stream and show that attentional modulation is a cognitive process separate from recurrent processes related to the processing of objects on cluttered backgrounds.

A whole-brain monosynaptic input connectome to neuron classes in mouse visual cortex.

Identification of structural connections between neurons is a prerequisite to understanding brain function. Here we developed a pipeline to systematically map brain-wide monosynaptic input connections to genetically defined neuronal populations using an optimized rabies tracing system. We used mouse visual cortex as the exemplar system and revealed quantitative target-specific, layer-specific and cell-class-specific differences in its presynaptic connectomes. The retrograde connectivity indicates the presence of ventral and dorsal visual streams and further reveals topographically organized and continuously varying subnetworks mediated by different higher visual areas. The visual cortex hierarchy can be derived from intracortical feedforward and feedback pathways mediated by upper-layer and lower-layer input neurons. We also identify a new role for layer 6 neurons in mediating reciprocal interhemispheric connections. This study expands our knowledge of the visual system connectomes and demonstrates that the pipeline can be scaled up to dissect connectivity of different cell populations across the mouse brain.

Rapid development of motion-streak coding in the mouse visual cortex

Despite its importance, the development of higher visual areas (HVAs) at the cellular resolution remains largely unknown. Here, we conducted 2-photon calcium imaging of mouse HVAs lateromedial (LM) and anterolateral (AL) and V1 to observe developmental changes in visual response properties. HVA neurons showed selectivity for orientations and directions similar to V1 neurons at eye opening, which became sharper in the following weeks. Neurons in all areas over all developmental stages tended to respond selectively to dots moving along an axis perpendicular to their preferred orientation at slow speeds, suggesting a certain level of conventional motion coding already at eye opening. In contrast, at high speeds, many neurons responded to dots moving along the axis parallel to the preferred orientation in older animals but rarely after eye opening, indicating a lack of motion-streak coding in the earlier stage. Together, our results uncover the development of visual properties in HVAs.

The spatial properties of adaptation-induced distance compression.

Exposure to a dynamic texture reduces the perceived separation between objects, altering the mapping between physical relations in the environment and their neural representations. Here we investigated the spatial tuning and spatial frame of reference of this aftereffect to understand the stage(s) of processing where adaptation-induced changes occur. In Experiment 1, we measured apparent separation at different positions relative to the adapted area, revealing a strong but tightly tuned compression effect. We next tested the spatial frame of reference of the effect, either by introducing a gaze shift between adaptation and test phase (Experiment 2) or by decoupling the spatial selectivity of adaptation in retinotopic and world-centered coordinates (Experiment 3). Results across the two experiments indicated that both retinotopic and world-centered adaptation effects can occur independently. Spatial attention to the location of the adaptor alone could not account for the world-centered transfer we observed, and retinotopic adaptation did not transfer to world-centered coordinates after a saccade (Experiment 4). Finally, we found that aftereffects in different reference frames have a similar, narrow spatial tuning profile (Experiment 5). Together, our results suggest that the neural representation of local separation resides early in the visual cortex, but it can also be modulated by activity in higher visual areas.

Understanding transformation tolerant visual object representations in the human brain and convolutional neural networks

Forming transformation-tolerant object representations is critical to high-level primate vision. Despite its significance, many details of tolerance in the human brain remain unknown. Likewise, despite the ability of convolutional neural networks (CNNs) to exhibit human-like object categorization performance, whether CNNs form tolerance similar to that of the human brain is unknown. Here we provide the first comprehensive documentation and comparison of three tolerance measures in the human brain and CNNs. We measured fMRI responses from human ventral visual areas to real-world objects across both Euclidean and non-Euclidean feature changes. In single fMRI voxels in higher visual areas, we observed robust object response rank-order preservation across feature changes. This is indicative of functional smoothness in tolerance at the fMRI meso-scale level that has never been reported before. At the voxel population level, we found highly consistent object representational structure across feature changes towards the end of ventral processing. Rank-order preservation, consistency, and a third tolerance measure, cross-decoding success (i.e., a linear classifier's ability to generalize performance across feature changes) showed an overall tight coupling. These tolerance measures were in general lower for Euclidean than non-Euclidean feature changes in lower visual areas, but increased over the course of ventral processing for all feature changes. These characteristics of tolerance, however, were absent in eight CNNs pretrained with ImageNet images with varying network architecture, depth, the presence/absence of recurrent processing, or whether a network was pretrained with the original or stylized ImageNet images that encouraged shape processing. CNNs do not appear to develop the same kind of tolerance as the human brain over the course of visual processing.

Diversity in spatial frequency, temporal frequency, and speed tuning across mouse visual cortical areas and layers.

The mouse visual system consists of several visual cortical areas thought to be specialized for different visual features and/or tasks. Previous studies have revealed differences between primary visual cortex (V1) and other higher visual areas, namely, anterolateral (AL) and posteromedial (PM), and their tuning preferences for spatial and temporal frequency. However, these differences have primarily been characterized using methods that are biased toward superficial layers of cortex, such as two-photon calcium imaging. Fewer studies have investigated cell types in deeper layers of these areas and their tuning preferences. Because superficial versus deep-layer neurons and different types of deep-layer neurons are known to have different feedforward and feedback inputs and outputs, comparing the tuning preferences of these groups is important for understanding cortical visual information processing. In this study, we used extracellular electrophysiology and two-photon calcium imaging targeted toward two different layer 5 cell classes to characterize their tuning properties in V1, AL, and PM. We find that deep-layer neurons, similar to superficial layer neurons, are also specialized for different spatial and temporal frequencies, with the strongest differences between AL and V1, and AL and PM, but not V1 and PM. However, we note that the deep-layer neuron populations preferred a larger range of SFs and TFs compared to previous studies. We also find that extratelencephalically projecting layer 5 neurons are more direction selective than intratelencephalically projecting layer 5 neurons.

Modular strategy for development of the hierarchical visual network in mice.

Hierarchical and parallel networks are fundamental structures of the mammalian brain1-8. During development, lower- and higher-order thalamic nuclei and many cortical areas in the visual system form interareal connections and build hierarchical dorsal and ventral streams9-13. One hypothesis for the development of visual network wiring involves a sequential strategy wherein neural connections are sequentially formed alongside hierarchical structures from lower to higher areas14-17. However, this sequential strategy would be inefficient for building the entire visual network comprising numerous interareal connections. We show that neural pathways from the mouse retina to primary visual cortex (V1) or dorsal/ventral higher visual areas (HVAs) through lower- or higher-order thalamic nuclei form as parallel modules before corticocortical connections. Subsequently, corticocortical connections among V1 and HVAs emerge to combine these modules. Retina-derived activity propagating the initial parallel modules is necessary to establish retinotopic inter-module connections. Thus, the visual network develops in a modular manner involving initial establishment of parallel modules and their subsequent concatenation. Findings in this study raise the possibility that parallel modules from higher-order thalamic nuclei to HVAs act as templates for cortical ventral and dorsal streams and suggest that the brain has an efficient strategy for the development of a hierarchical network comprising numerous areas.

V1-Origin Bidirectional Plasticity in Visual Thalamo-Ventral Pathway and Its Contribution to Saliency Detection of Dynamic Visual Inputs.

Visual neural plasticity and V1 saliency detection are vital for efficient coding of dynamically changing visual inputs. However, how does neural plasticity contribute to saliency detection of temporal statistically distributed visual stream remains unclear. Therefore, we adopted randomly presented but unevenly distributed stimuli with multiple orientations and examined the single-unit responses evoked by this biased orientation-adaptation protocol by single-unit recordings in the visual thalamo-ventral pathway of cats (of either sex). We found neuronal responses potentiated when the probability of biased orientation was slightly higher than other nonbiased ones and suppressed when the probability became much higher. This single neuronal short-term bidirectional plasticity is selectively induced by optimal stimuli but is interocularly transferable. It is inducible in LGN, Area 17, and Area 21a with distinct and hierarchically progressive patterns. With the results of latency analysis, receptive field structural test, cortical lesion, and simulations, we suggest this bidirectional plasticity may principally originate from the adaptation competition between excitatory and inhibitory components of V1 neuronal receptive field. In our simulation, above bidirectional plasticity could achieve saliency detection of dynamic visual inputs. These findings demonstrate a rapid probability dependent plasticity on the neural coding of visual stream and suggest its functional role in the efficient coding and saliency detection of dynamic environment.SIGNIFICANCE STATEMENT Novel elements within a dynamic visual stream can pop up from the context, which is vital for rapid response to a dynamically changing world. Saliency detection is a promising bottom-up mechanism contributing to efficient selection of visual inputs, wherein visual adaptation also plays a significant role. However, the saliency detection of dynamic visual stream is poorly understood. Here, we found a novel form of visual short-term bidirectional plasticity in multistages of the visual system that contributes to saliency detection of dynamic visual inputs. This bidirectional plasticity may principally originate from the local balance of excitation inhibition in primary visual cortex and propagates to lower and higher visual areas with progressive pattern change. Our findings suggest the excitation-inhibition balance within the visual system contributes to visual efficient coding.

Visual illusion susceptibility in autism: A neural model.

While atypical sensory perception is reported among individuals with autism spectrum disorder (ASD), the underlying neural mechanisms of autism that give rise to disruptions in sensory perception remain unclear. We developed a neural model with key physiological, functional and neuroanatomical parameters to investigate mechanisms underlying the range of representations of visual illusions related to orientation perception in typically developed subjects compared to individuals with ASD. Our results showed that two theorized autistic traits, excitation/inhibition imbalance and weakening of top‐down modulation, could be potential candidates for reduced susceptibility to some visual illusions. Parametric correlation between cortical suppression, balance of excitation/inhibition, feedback from higher visual areas on one hand and susceptibility to a class of visual illusions related to orientation perception on the other hand provide the opportunity to investigate the contribution and complex interactions of distinct sensory processing mechanisms in ASD. The novel approach used in this study can be used to link behavioural, functional and neuropathological studies; estimate and predict perceptual and cognitive heterogeneity in ASD; and form a basis for the development of novel diagnostics and therapeutics.

The emergence of view-symmetric neural responses to familiar and unfamiliar faces

Successful recognition of familiar faces is thought to depend on the ability to integrate view-dependent representations of a face into a view-invariant representation. It has been proposed that a key intermediate step in achieving view invariance is the representation of symmetrical views. However, key unresolved questions remain, such as whether these representations are specific for naturally occurring changes in viewpoint and whether view-symmetric representations exist for familiar faces. To address these issues, we compared behavioural and neural responses to natural (canonical) and unnatural (noncanonical) rotations of the face. Similarity judgements revealed that symmetrical viewpoints were perceived to be more similar than non-symmetrical viewpoints for both canonical and non-canonical rotations. Next, we measured patterns of neural response from early to higher level regions of visual cortex. Early visual areas showed a view-dependent representation for natural or canonical rotations of the face, such that the similarity between patterns of response were related to the difference in rotation. View symmetric patterns of neural response to canonically rotated faces emerged in higher visual areas, particularly in face-selective regions. The emergence of a view-symmetric representation from a view-dependent representation for canonical rotations of the face was also evident for familiar faces, suggesting that view-symmetry is an important intermediate step in generating view-invariant representations. Finally, we measured neural responses to unnatural or non-canonical rotations of the face. View-symmetric patterns of response were also found in face-selective regions. However, in contrast to natural or canonical rotations of the face, these view-symmetric responses did not arise from an initial view-dependent representation in early visual areas. This suggests differences in the way that view-symmetrical representations emerge with canonical or non-canonical rotations. The similarity in the neural response to canonical views of familiar and unfamiliar faces in the core face network suggests that the neural correlates of familiarity emerge at later stages of processing.

Electrophysiological dataset from macaque visual cortical area MST in response to a novel motion stimulus

Establishing the cortical neural representation of visual stimuli is a central challenge of systems neuroscience. Publicly available data would allow a broad range of scientific analyses and hypothesis testing, but are rare and largely focused on the early visual system. To address the shortage of open data from higher visual areas, we provide a comprehensive dataset from a neurophysiology study in macaque monkey visual cortex that includes a complete record of extracellular action potential recordings from the extrastriate medial superior temporal (MST) area, behavioral data, and detailed stimulus records. It includes spiking activity of 172 single neurons recorded in 139 sessions from 4 hemispheres of 3 rhesus macaque monkeys. The data was collected across 3 experiments, designed to characterize the response properties of MST neurons to complex motion stimuli. This data can be used to elucidate visual information processing at the level of single neurons in a high-level area of primate visual cortex. Providing open access to this dataset also promotes the 3R-principle of responsible animal research.

Dynamics of absolute and relative disparity processing in human visual cortex

Cortical processing of binocular disparity is believed to begin in V1 where cells are sensitive to absolute disparity, followed by the extraction of relative disparity in higher visual areas. While much is known about the cortical distribution and spatial tuning of disparity-selective neurons, the relationship between their spatial and temporal properties is less well understood. Here, we use steady-state Visual Evoked Potentials and dynamic random dot stereograms to characterize the temporal dynamics of spatial mechanisms in human visual cortex that are primarily sensitive to either absolute or relative disparity. Stereograms alternated between disparate and non-disparate states at 2 Hz. By varying the disparity-defined spatial frequency content of the stereograms from a planar surface to corrugated ones, we biased responses towards absolute vs. relative disparities. Reliable Components Analysis was used to derive two dominant sources from the 128 channel EEG records. The first component (RC1) was maximal over the occipital pole. In RC1, first harmonic responses were sustained, tuned for corrugation frequency, and sensitive to the presence of disparity references, consistent with prior psychophysical sensitivity measurements. By contrast, the second harmonic, associated with transient processing, was not spatially tuned and was indifferent to references, consistent with it being generated by an absolute disparity mechanism. Thus, our results reveal a duplex coding strategy in the disparity domain, where relative disparities are computed via sustained mechanisms and absolute disparities are computed via transient mechanisms.