Ventral Visual Stream Research Articles

Event-related phase synchronization propagates rapidly across human ventral visual cortex

Visual inputs to early visual cortex integrate with semantic, linguistic and memory inputs in higher visual cortex, in a manner that is rapid and accurate, and enables complex computations such as face recognition and word reading. This implies the existence of fundamental organizational principles that enable such efficiency. To elaborate on this, we performed intracranial recordings in 82 individuals while they performed tasks of varying visual and cognitive complexity. We discovered that visual inputs induce highly organized posterior-to-anterior propagating patterns of phase modulation across the ventral occipitotemporal cortex. At individual electrodes there was a stereotyped temporal pattern of phase progression following both stimulus onset and offset, consistent across trials and tasks. The phase of low frequency activity in anterior regions was predicted by the prior phase in posterior cortical regions. This spatiotemporal propagation of phase likely serves as a feed-forward organizational influence enabling the integration of information across the ventral visual stream. This phase modulation manifests as the early components of the event related potential; one of the most commonly used measures in human electrophysiology. These findings illuminate fundamental organizational principles of the higher order visual system that enable the rapid recognition and characterization of a variety of inputs.

Processing of visual and non-visual naturalistic spatial information in the "parahippocampal place area"

The “parahippocampal place area” (PPA) in the human ventral visual stream exhibits increased hemodynamic activity correlated with the perception of landscape photos compared to faces or objects. Here, we investigate the perception of scene-related, spatial information embedded in two naturalistic stimuli. The same 14 participants were watching a Hollywood movie and listening to its audio-description as part of the open-data resource studyforrest.org. We model hemodynamic activity based on annotations of selected stimulus features, and compare results to a block-design visual localizer. On a group level, increased activation correlating with visual spatial information occurring in the movie is overlapping with a traditionally localized PPA. Activation correlating with semantic spatial information occurring in the audio-description is more restricted to the anterior PPA. On an individual level, we find significant bilateral activity in the PPA of nine individuals and unilateral activity in one individual. Results suggest that activation in the PPA generalizes to spatial information embedded in a movie and an auditory narrative, and may call for considering a functional subdivision of the PPA.

Activity in perirhinal and entorhinal cortex predicts perceived visual similarities among category exemplars with highest precision.

Vision neuroscience has made great strides in understanding the hierarchical organization of object representations along the ventral visual stream (VVS). How VVS representations capture fine-grained visual similarities between objects that observers subjectively perceive has received limited examination so far. In the current study, we addressed this question by focussing on perceived visual similarities among subordinate exemplars of real-world categories. We hypothesized that these perceived similarities are reflected with highest fidelity in neural activity patterns downstream from inferotemporal regions, namely in perirhinal (PrC) and anterolateral entorhinal cortex (alErC) in the medial temporal lobe. To address this issue with functional magnetic resonance imaging (fMRI), we administered a modified 1-back task that required discrimination between category exemplars as well as categorization. Further, we obtained observer-specific ratings of perceived visual similarities, which predicted behavioural discrimination performance during scanning. As anticipated, we found that activity patterns in PrC and alErC predicted the structure of perceived visual similarity relationships among category exemplars, including its observer-specific component, with higher precision than any other VVS region. Our findings provide new evidence that subjective aspects of object perception that rely on fine-grained visual differentiation are reflected with highest fidelity in the medial temporal lobe.

Development of white matter tracts between and within the dorsal and ventral streams.

The degree of interaction between the ventral and dorsal visual streams has been discussed in multiple scientific domains for decades. Recently, several white matter tracts that directly connect cortical regions associated with the dorsal and ventral streams have become possible to study due to advancements in automated and reproducible methods. The developmental trajectory of this set of tracts, here referred to as the posterior vertical pathway (PVP), has yet to be described. We propose an input-driven model of white matter development and provide evidence for the model by focusing on the development of the PVP. We used reproducible, cloud-computing methods and diffusion imaging from adults and children (ages 5-8years) to compare PVP development to that of tracts within the ventral and dorsal pathways. PVP microstructure was more adult-like than dorsal stream microstructure, but less adult-like than ventral stream microstructure. Additionally, PVP microstructure was more similar to the microstructure of the ventral than the dorsal stream and was predicted by performance on a perceptual task in children. Overall, results suggest a potential role for the PVP in the development of the dorsal visual stream that may be related to its ability to facilitate interactions between ventral and dorsal streams during learning. Our results are consistent with the proposed model, suggesting that the microstructural development of major white matter pathways is related, at least in part, to the propagation of sensory information within the visual system.

A neural network account of memory replay and knowledge consolidation.

Replay can consolidate memories through offline neural reactivation related to past experiences. Category knowledge is learned across multiple experiences, and its subsequent generalization is promoted by consolidation and replay during rest and sleep. However, aspects of replay are difficult to determine from neuroimaging studies. We provided insights into category knowledge replay by simulating these processes in a neural network which approximated the roles of the human ventral visual stream and hippocampus. Generative replay, akin to imagining new category instances, facilitated generalization to new experiences. Consolidation-related replay may therefore help to prepare us for the future as much as remember the past. Generative replay was more effective in later network layers functionally similar to the lateral occipital cortex than layers corresponding to early visual cortex, drawing a distinction between neural replay and its relevance to consolidation. Category replay was most beneficial for newly acquired knowledge, suggesting replay helps us adapt to changes in our environment. Finally, we present a novel mechanism for the observation that the brain selectively consolidates weaker information, namely a reinforcement learning process in which categories were replayed according to their contribution to network performance. This reinforces the idea of consolidation-related replay as an active rather than passive process.

The spatiotemporal neural dynamics of object location representations in the human brain

To interact with objects in complex environments, we must know what they are and where they are in spite of challenging viewing conditions. Here, we investigated where, how and when representations of object location and category emerge in the human brain when objects appear on cluttered natural scene images using a combination of functional magnetic resonance imaging, electroencephalography and computational models. We found location representations to emerge along the ventral visual stream towards lateral occipital complex, mirrored by gradual emergence in deep neural networks. Time-resolved analysis suggested that computing object location representations involves recurrent processing in high-level visual cortex. Object category representations also emerged gradually along the ventral visual stream, with evidence for recurrent computations. These results resolve the spatiotemporal dynamics of the ventral visual stream that give rise to representations of where and what objects are present in a scene under challenging viewing conditions.

The role of ventral stream areas for viewpoint-invariant object recognition

Object constancy is one of the most crucial mechanisms of the human visual system enabling viewpoint invariant object recognition. However, the neuronal foundations of object constancy are widely unknown. Research has shown that the ventral visual stream is involved in processing of various kinds of object stimuli and that several regions along the ventral stream are possibly sensitive to the orientation of an object in space. To systematically address the question of viewpoint sensitive object perception, we conducted a study with stroke patients as well as an fMRI experiment with healthy participants applying object stimuli in several spatial orientations, for example in typical and atypical viewing conditions. In the fMRI experiment, we found stronger BOLD signals and above-chance classification accuracies for objects presented in atypical viewing conditions in fusiform face sensitive and lateral occipito-temporal object preferring areas. In the behavioral patient study, we observed that lesions of the right fusiform gyrus were associated with lower performance in object recognition for atypical views. The complementary results from both experiments emphasize the contributions of fusiform and lateral-occipital areas to visual object constancy and indicate that visual object constancy is particularly enabled through increased neuronal activity and specific activation patterns for objects in demanding viewing conditions.

A test of the unified model of vision and attention: Effects of parietal-occipital damage on visual orienting.

Visual orienting was studied in a patient (FM) with parietal-occipital damage due to oligodendroglioma and associated surgery, and in eighteen control participants. The ability of FM and control participants to shift attention in response to spatial landmark cues, and in response to cues that recruit endogenous orienting via encoding of cue identity, were assessed. According to the unified model of vision and attention (Lambert, A. et al., Journal of Experimental Psychology: Human Perception & Performance, 44, 412–432) FM should find it difficult to orient attention in response to spatial landmarks due to impaired functioning of the dorsal visual stream; but shifting attention in response to cue identity, encoded via the ventral visual stream, should be spared. Consistent with these predictions, FM was unable to shift attention in the landmark cueing task, but shifted attention effectively in response to identity cues; and her visual orienting performance differed reliably from controls. These findings complement our earlier observation of preserved orienting towards landmark cues in a patient with bilateral damage to the ventral visual stream, and add to a growing body of evidence in support of the unified model of vision and attention.

The forgotten tract of vision in multiple sclerosis: vertical occipital fasciculus, its fiber properties, and visuospatial memory.

Visual disturbances are a common disease manifestation in multiple sclerosis (MS) due to lesions damaging white matter tracts involved in vision. Vertical occipital fasciculus (VOF), a tract located vertically in the occipital lobe, was neglected for more than a century. We hypothesize that VOF is involved in integrating information between dorsal and ventral visual streams. Thus, its damage in MS, as well as its probable role in visual processing (by using MS as a VOF damage model) needs to be clarified. To study fiber characteristics of VOF in MS, and their clinical and visual learning associations, 57 relapsing-remitting MS (RRMS) and 25 healthy controls (HC) were recruited. We acquired MS Functional Composite, Expanded Disability Status Scale (EDSS), and Brief Visuospatial Memory Test-Revised (BVMT-R), and diffusion MRI scans. Tractography of VOF and optic radiation (OR) was done. VOF's metrics were statistically tested for between-group differences and clinical and visual tests associations. Along-tract analysis and laterality were also tested. RRMS patients had higher mean, axial, and radial diffusivity (nearly in all fiber points), and lower fractional anisotropy in bilateral VOFs compared to HC. No laterality was noted. These were associated with poor clinical outcomes, poor visual scores in EDSS, and lower total immediate and delayed recall in BVMT-R in RRMS, after adjusting for age, gender, and fiber metrics of OR. VOF damage is present in RRMS and is associated with visual symptoms and visuospatial learning impairments. It seems VOF is involved in integrating information between visual streams.

Age-related differences in encoding-retrieval similarity and their relationship to false memory

Typical aging is associated with increases in false memory rates among older adults. Such errors are frequently associated with differential neural activity during encoding and retrieval in older compared to younger adults within visual cortices and the hippocampus. It remains unknown how pattern similarity reductions relate to false memories in healthy aging. Using encoding-retrieval similarity (ERS) analyses in a sample of younger and older adults, we examined how the similarity of neural patterns between memory phases associated with target and lure objects was impacted by age and contributed to false memory rates. Single-item ERS for targets and lures was reduced by age throughout much of the ventral visual stream and the posterior hippocampus. Furthermore, ERS associated with perceptual lures within the visual stream maintained differential relationships with false memory. Finally, a global ERS metric accounted for age deficits in single-item ERS, but did not contribute to false memory rates. These findings highlight the contribution of age-related reductions in ERS across multiple representational levels to false memories in healthy aging.

Spatial context and the functional role of the postrhinal cortex

The postrhinal cortex (POR) serves as a key input area to the hippocampal system. It receives highly processed information from the ventral visual stream and other limbic areas including the retrosplenial cortex, parahippocampal areas, and portions of the limbic thalamus. The POR was studied early on by David Bucci and colleagues who first postulated that the POR plays a major role in contextual learning. Here we review a number of approaches and experimental studies that have explored POR’s role in contextual processing. We discuss POR lesion studies that monitored deficits in fear conditioning tasks and the effects that these lesions had on processing visual landmark information. We then review the types of spatial correlates encoded by POR cells. A large number of head direction (HD) cells are present, although recent findings suggest that many of them are more accurately characterized as landmark modulated-HD cells as opposed to classic HD cells. A significant number of POR cells are also tuned to egocentric properties of the environment, such as the spatial relationship of the animal to the center of its environment, or the distance between the animal and either the environment’s center or its boundaries. We suggest potential frameworks through which these functional cell types might support contextual processing. We then discuss deficits seen in humans who have damage to the homologous parahippocampal cortex, and we finish by reviewing functional imaging studies that found activation of this area while human subjects performed various tasks. A preponderance of evidence suggests that the POR, along with its interactions with retrosplenial cortex, plays a key role in contextual information processing.

Informative neural representations of unseen contents during higher-order processing in human brains and deep artificial networks.

A framework to pinpoint the scope of unconscious processing is critical to improve models of visual consciousness. Previous research observed brain signatures of unconscious processing in visual cortex, but these were not reliably identified. Further, whether unconscious contents are represented in high-level stages of the ventral visual stream and linked parieto-frontal areas remains unknown. Using a within-subject, high-precision functional magnetic resonance imaging approach, we show that unconscious contents can be decoded from multi-voxel patterns that are highly distributed alongside the ventral visual pathway and also involving parieto-frontal substrates. Classifiers trained with multi-voxel patterns of conscious items generalized to predict the unconscious counterparts, indicating that their neural representations overlap. These findings suggest revisions to models of consciousness such as the neuronal global workspace. We then provide a computational simulation of visual processing/representation without perceptual sensitivity by using deep neural networks performing a similar visual task. The work provides a framework for pinpointing the representation of unconscious knowledge across different task domains.

The vertical superior longitudinal fascicle and the vertical occipital fascicle.

Association fibers of the human brain have long been considered to exclusively follow an anterior-posterior direction. Using magnetic resonance imaging techniques that allow in-vivo fiber dissection, vertically oriented association fibers have been rediscovered or newly described. Aside from the frontal aslant tract (FAT) in the frontal lobe, the vertical occipital fascicle (VOF) and the vertical portion of the superior longitudinal fascicle system (vSLF) have been studied in recent years. The aim of this review was to give an overview on the current knowledge regarding these two fiber tracts. A review of the available literature in the Medline database was conducted to gather all available publications dealing with either the VOF or the vSLF. One thousand two hundred seventy-three articles were obtained from the literature search of which a total of 71 articles met the final inclusion criteria of this review. We describe the history of the discovery of the respective fiber tract, its anatomical course and its boundaries integrating blunt fiber dissection studies and functional MRI/tractography studies. We discuss the functional properties of the respective fiber tract and its relevance in neurosurgery. The VOF is a fiber tract that has been discovered in the late XIX century and long been forgotten before being rediscovered in the 1970's. It lies lateral to the fibers of the sagittal stratum and mainly connects the superior and inferior occipital lobe. It plays a major role in reading and visual word and language comprehension and is said to be the main link between dorsal and ventral visual streams. The vSLF has many synonyms and is part of the superior longitudinal fascicle system. Recent studies were able to provide more insight into this set of fiber tracts showing distinct connections running from the superior and inferior parietal lobule to the posterior part of the temporal lobe. Its functional role is still not completely cleared. It is said to play a role in visual and auditory semantic language comprehension. It lies directly lateral to the arcuate fascicle. The VOF and the vSLF are vertically oriented fiber tracts connecting the temporo-parieto-occipital region and play a major role in the communication of dorsal and ventral visual streams (VOF), reading (VOF, vSLF) and visual and auditory semantic language comprehension (vSLF). They can consistently be identified using ex vivo blunt dissection techniques and in-vivo fiber tractography. Because of their localization and orientation these two fiber tracts can be combined to a fiber bundle system called posterior transverse system (PTS).

Two 'what' pathways for action and object recognition.

The ventral visual stream is conceived as a pathway for object recognition. However, we also recognize the actions an object can be involved in. Here, we show that action recognition critically depends on a pathway in lateral occipitotemporal cortex, partially overlapping and topographically aligned with object representations that are precursors for action recognition. By contrast, object features that are more relevant for object recognition, such as color and texture, are typically found in ventral occipitotemporal cortex. We argue that occipitotemporal cortex contains similarly organized lateral and ventral 'what' pathways for action and object recognition, respectively. This account explains a number of observed phenomena, such as the duplication of object domains and the specific representational profiles in lateral and ventral cortex.

Visual hallucinations in Lewy body disease: pathophysiological insights from phenomenology

Visual hallucinations (VH) in Lewy body disease (LBD) have a heterogenous phenomenology classified into minor phenomena (MVH) and complex hallucinations (CVH). Mechanisms underpinning VH and their temporal aspects are largely unknown. According to the hodotopic model, we investigated whether changes in distinct cognitive domains and neural networks in the hallucination trait underpin temporal aspects of MVH and CVH in the hallucination state. 35 LBD patients with VH underwent a complete neuropsychological evaluation and resting-state fMRI. North-East-Visual-Hallucinations-Interview was used to assess their typical VH content, duration, and frequency. We found that MVH was not associated with cognitive impairment, while CVH was associated with impairments in visuoperceptual processes, attention and visual abstract reasoning. In seed-to-seed functional connectivity (FC) analysis we identified functional couplings associated with MVH and CVH temporal severity (duration x frequency), duration and frequency. MVH severity was negatively associated with FC between early visual areas (EVA) and ventral-visual-stream regions, and negatively associated with FC between brainstem and EVA, which may be linked to LBD brainstem neuropathology. CVH duration was positively associated with FC between ventral-visual stream and salience network (SN). CVH frequency was negatively associated with FC between DMN and SN. Functional alterations in distinct visual and attentional networks and their dynamic interaction in trait LBD hallucinators are linked to both the phenomenology of state content and its temporal characteristics. Within a network, VH frequency and duration may be linked to different types of functional alterations: increased connectivity leading to sustained activity prolonging VH (duration) and decreased connectivity increasing dysregulated, spontaneous activity (frequency). These findings support the hodotopic hypothesis of VH and may reflect a link between VH phenomenology, LBD neuropathological progression and the involvement of specific neurotransmitter systems.

A self-supervised domain-general learning framework for human ventral stream representation

Anterior regions of the ventral visual stream encode substantial information about object categories. Are top-down category-level forces critical for arriving at this representation, or can this representation be formed purely through domain-general learning of natural image structure? Here we present a fully self-supervised model which learns to represent individual images, rather than categories, such that views of the same image are embedded nearby in a low-dimensional feature space, distinctly from other recently encountered views. We find that category information implicitly emerges in the local similarity structure of this feature space. Further, these models learn hierarchical features which capture the structure of brain responses across the human ventral visual stream, on par with category-supervised models. These results provide computational support for a domain-general framework guiding the formation of visual representation, where the proximate goal is not explicitly about category information, but is instead to learn unique, compressed descriptions of the visual world.

Increasing neural network robustness improves match to macaque V1 eigenspectrum, spatial frequency preference and predictivity.

Task-optimized convolutional neural networks (CNNs) show striking similarities to the ventral visual stream. However, human-imperceptible image perturbations can cause a CNN to make incorrect predictions. Here we provide insight into this brittleness by investigating the representations of models that are either robust or not robust to image perturbations. Theory suggests that the robustness of a system to these perturbations could be related to the power law exponent of the eigenspectrum of its set of neural responses, where power law exponents closer to and larger than one would indicate a system that is less susceptible to input perturbations. We show that neural responses in mouse and macaque primary visual cortex (V1) obey the predictions of this theory, where their eigenspectra have power law exponents of at least one. We also find that the eigenspectra of model representations decay slowly relative to those observed in neurophysiology and that robust models have eigenspectra that decay slightly faster and have higher power law exponents than those of non-robust models. The slow decay of the eigenspectra suggests that substantial variance in the model responses is related to the encoding of fine stimulus features. We therefore investigated the spatial frequency tuning of artificial neurons and found that a large proportion of them preferred high spatial frequencies and that robust models had preferred spatial frequency distributions more aligned with the measured spatial frequency distribution of macaque V1 cells. Furthermore, robust models were quantitatively better models of V1 than non-robust models. Our results are consistent with other findings that there is a misalignment between human and machine perception. They also suggest that it may be useful to penalize slow-decaying eigenspectra or to bias models to extract features of lower spatial frequencies during task-optimization in order to improve robustness and V1 neural response predictivity.

Spatial and Temporal Context Jointly Modulate the Sensory Response within the Ventral Visual Stream.

Both spatial and temporal context play an important role in visual perception and behavior. Humans can extract statistical regularities from both forms of context to help process the present and to construct expectations about the future. Numerous studies have found reduced neural responses to expected stimuli compared with unexpected stimuli, for both spatial and temporal regularities. However, it is largely unclear whether and how these forms of context interact. In the current fMRI study, 33 human volunteers were exposed to pairs of object stimuli that could be expected or surprising in terms of their spatial and temporal context. We found reliable independent contributions of both spatial and temporal context in modulating the neural response. Specifically, neural responses to stimuli in expected compared with unexpected contexts were suppressed throughout the ventral visual stream. These results suggest that both spatial and temporal context may aid sensory processing in a similar fashion, providing evidence on how different types of context jointly modulate perceptual processing.

Neural Representations of Conceptual Fixation during Creative Imagination

ABSTRACT A large body of research has revealed that viewing example image stimuli tends to constrain creative idea generation. However, the neurocognitive mechanisms underlying such visual fixation in creative cognition are unclear. In the present experiment, we explored whether example images impacted creative imagination and patterns of neural activity within brain regions associated with visual object recognition. Participants first viewed example images (ambiguous line drawings) accompanied by high-constraint and low-constraint labels. High-constraint labels resembled the line drawings, whereas low constraint labels did not. Next, participants imagined new labels for the same line drawings, with the initial labels removed. Consistent with our predictions, semantic distance analysis comparing cue labels to newly generated labels showed lower average semantic distance (i.e., less creative ideas) on high-constraint trials compared to low-constraint trials. Using representational similarity analysis, we also demonstrated that neural pattern similarity was anticorrelated (less similar) from object recognition to high-constraint imagination trials within the right inferior temporal gyrus, right middle temporal gyrus, and right superior occipital gyrus. Broadly, these findings suggest that salient visual examples may guide the formation of strong mental representations that constrain creative imagination. This research also offers a first step toward identifying neurocognitive signatures associated with the effortful process of producing new, creative ideas following exposure to fixating examples – particularly at the early level of object recognition/representation in the ventral visual stream.

Electrophysiological response to visual symmetry: Effects of the number of symmetry axes

The symmetry axes of a stimulus are a critical determinant of visual perception. Although much is known about the effects of a single symmetry axis on perception, the effects of multiple symmetry axes are still poorly understood. Here, we investigated the influence of the number of symmetry axes on brain activity using event-related potentials (ERPs). Our results showed that altering the number of symmetry axes affects both the amplitude and the latency of ERPs. Specifically, the amplitude of ERP components increased as the number of symmetry axes increased, starting at the N1 (165–175 ms) component and lasting until the P2 (230–250 ms) component in the bilateral posterior areas and until the N2 (340 ms) component in the frontal-central areas. Importantly, the latency of ERP components was reduced when the number of symmetry axes increased, starting at the N1 in the right posterior area and lasting until the P2 component in the bilateral posterior areas. The temporal and spatial differences in these effects imply that activity related to symmetry axes gradually changes throughout the ventral visual streams in the human brain.