Anterior Ventral Temporal Cortex Research Articles

Perceptual and Semantic Representations at Encoding Contribute to True and False Recognition of Objects.

When encoding new episodic memories, visual and semantic processing is proposed to make distinct contributions to accurate memory and memory distortions. Here, we used fMRI and preregistered representational similarity analysis to uncover the representations that predict true and false recognition of unfamiliar objects. Two semantic models captured coarse-grained taxonomic categories and specific object features, respectively, while two perceptual models embodied low-level visual properties. Twenty-eight female and male participants encoded images of objects during fMRI scanning, and later had to discriminate studied objects from similar lures and novel objects in a recognition memory test. Both perceptual and semantic models predicted true memory. When studied objects were later identified correctly, neural patterns corresponded to low-level visual representations of these object images in the early visual cortex, lingual, and fusiform gyri. In a similar fashion, alignment of neural patterns with fine-grained semantic feature representations in the fusiform gyrus also predicted true recognition. However, emphasis on coarser taxonomic representations predicted forgetting more anteriorly in the anterior ventral temporal cortex, left inferior frontal gyrus and, in an exploratory analysis, left perirhinal cortex. In contrast, false recognition of similar lure objects was associated with weaker visual analysis posteriorly in early visual and left occipitotemporal cortex. The results implicate multiple perceptual and semantic representations in successful memory encoding and suggest that fine-grained semantic as well as visual analysis contributes to accurate later recognition, while processing visual image detail is critical for avoiding false recognition errors.SIGNIFICANCE STATEMENT People are able to store detailed memories of many similar objects. We offer new insights into the encoding of these specific memories by combining fMRI with explicit models of how image properties and object knowledge are represented in the brain. When people processed fine-grained visual properties in occipital and posterior temporal cortex, they were more likely to recognize the objects later and less likely to falsely recognize similar objects. In contrast, while object-specific feature representations in fusiform gyrus predicted accurate memory, coarse-grained categorical representations in frontal and temporal regions predicted forgetting. The data provide the first direct tests of theoretical assumptions about encoding true and false memories, suggesting that semantic representations contribute to specific memories as well as errors.

Reading music and words: The anatomical connectivity of musicians’ visual cortex

Musical score reading and word reading have much in common, from their historical origins to their cognitive foundations and neural correlates. In the ventral occipitotemporal cortex (VOT), the specialization of the so-called Visual Word Form Area for word reading has been linked to its privileged structural connectivity to distant language regions. Here we investigated how anatomical connectivity relates to the segregation of regions specialized for musical notation or words in the VOT. In a cohort of professional musicians and non-musicians, we used probabilistic tractography combined with task-related functional MRI to identify the connections of individually defined word- and music-selective left VOT regions. Despite their close proximity, these regions differed significantly in their structural connectivity, irrespective of musical expertise. The music-selective region was significantly more connected to posterior lateral temporal regions than the word-selective region, which, conversely, was significantly more connected to anterior ventral temporal cortex. Furthermore, musical expertise had a double impact on the connectivity of the music region. First, music tracts were significantly larger in musicians than in non-musicians, associated with marginally higher connectivity to perisylvian music-related areas. Second, the spatial similarity between music and word tracts was significantly increased in musicians, consistently with the increased overlap of language and music functional activations in musicians, as compared to non-musicians. These results support the view that, for music as for words, very specific anatomical connections influence the specialization of distinct VOT areas, and that reciprocally those connections are selectively enhanced by the expertise for word or music reading.

Orthogonal Representations of Object Shape and Category in Deep Convolutional Neural Networks and Human Visual Cortex

Deep Convolutional Neural Networks (CNNs) are gaining traction as the benchmark model of visual object recognition, with performance now surpassing humans. While CNNs can accurately assign one image to potentially thousands of categories, network performance could be the result of layers that are tuned to represent the visual shape of objects, rather than object category, since both are often confounded in natural images. Using two stimulus sets that explicitly dissociate shape from category, we correlate these two types of information with each layer of multiple CNNs. We also compare CNN output with fMRI activation along the human visual ventral stream by correlating artificial with neural representations. We find that CNNs encode category information independently from shape, peaking at the final fully connected layer in all tested CNN architectures. Comparing CNNs with fMRI brain data, early visual cortex (V1) and early layers of CNNs encode shape information. Anterior ventral temporal cortex encodes category information, which correlates best with the final layer of CNNs. The interaction between shape and category that is found along the human visual ventral pathway is echoed in multiple deep networks. Our results suggest CNNs represent category information independently from shape, much like the human visual system.

Finding decodable information that can be read out in behaviour

Multivariate decoding methods applied to neuroimaging data have become the standard in cognitive neuroscience for unravelling statistical dependencies between brain activation patterns and experimental conditions. The current challenge is to demonstrate that decodable information is in fact used by the brain itself to guide behaviour. Here we demonstrate a promising approach to do so in the context of neural activation during object perception and categorisation behaviour. We first localised decodable information about visual objects in the human brain using a multivariate decoding analysis and a spatially-unbiased searchlight approach. We then related brain activation patterns to behaviour by testing whether the classifier used for decoding can be used to predict behaviour. We show that while there is decodable information about visual category throughout the visual brain, only a subset of those representations predicted categorisation behaviour, which were strongest in anterior ventral temporal cortex. Our results have important implications for the interpretation of neuroimaging studies, highlight the importance of relating decoding results to behaviour, and suggest a suitable methodology towards this aim.

Beyond the core face-processing network: intracerebral stimulation of a face-selective area in the right anterior fusiform gyrus elicits transient prosopagnosia

According to neuropsychological evidence, a distributed network of face-selective regions of the ventral visual pathway supports face recognition. However, fMRI studies have generally confined to the posterior face-selective areas, i.e., the occipital face area (OFA) and the fusiform face area (FFA). There is recent evidence that intracranial electrical stimulation of the OFA and FFA elicits face recognition impairments. Here we were able to test for face recognition in a patient implanted with depth electrodes in the anterior temporal cortex. Electrically stimulating the right anterior fusiform gyrus, in a region located anteriorly to the FFA, induced a transient inability to recognize familiar faces (see Figure 1 in additional information; movies available). This region was shown face-selective as revealed by intracerebral face-selective event-related potentials and gamma band activity recorded at these critical electrodes. However, no fMRI face-selective responses were found in this region due to severe BOLD signal drop-out caused by the ear canal artifacts (see Figure 2 in additional information). These results point to a causal role in face recognition of the right anterior fusiform gyrus and more generally of face-selective areas located beyond the "core" face processing network in the anterior ventral temporal cortex. It also illustrates the diagnostic value of intracerebral recordings and stimulation in understanding the neural basis of face recognition and visual recognition in general. Meeting abstract presented at VSS 2016

Conceptual Object Representations in Human Anterior Temporal Cortex

Interaction with everyday objects requires the representation of conceptual object properties, such as where and how an object is used. What are the neural mechanisms that support this knowledge? While research on semantic dementia has provided evidence for a critical role of the anterior temporal lobes (ATLs) in object knowledge, fMRI studies using univariate analysis have primarily implicated regions outside the ATL. In the present human fMRI study we used multivoxel pattern analysis to test whether activity patterns in ATLs carry information about conceptual object properties. Participants viewed objects that differed on two dimensions: where the object is typically found (in the kitchen or the garage) and how the object is commonly used (with a rotate or a squeeze movement). Anatomical region-of-interest analyses covering the ventral visual stream revealed that information about the location and action dimensions increased from posterior to anterior ventral temporal cortex, peaking in the temporal pole. Whole-brain multivoxel searchlight analysis confirmed these results, revealing highly significant and regionally specific information about the location and action dimensions in the anterior temporal lobes bilaterally. In contrast to conceptual object properties, perceptual and low-level visual properties of the objects were reflected in activity patterns in posterior lateral occipitotemporal cortex and occipital cortex, respectively. These results provide fMRI evidence that object representations in the anterior temporal lobes are abstracted away from perceptual properties, categorizing objects in semantically meaningful groups to support conceptual object knowledge.

Action-Related Properties Shape Object Representations in the Ventral Stream

The principles driving the organization of the ventral object-processing stream remain unknown. Here, we show that stimulus-specific repetition suppression (RS) in one region of the ventral stream is biased according to motor-relevant properties of objects. Quantitative analysis confirmed that this result was not confounded with similarity in visual shape. A similar pattern of biases in RS according to motor-relevant properties of objects was observed in dorsal stream regions in the left hemisphere. These findings suggest that neural specificity for "tools" in the ventral stream is driven by similarity metrics computed over motor-relevant information represented in dorsal structures. Support for this view is provided by converging results from functional connectivity analyses of the fMRI data and a separate neuropsychological study. More generally, these data suggest that a basic organizing principle giving rise to "category specificity" in the ventral stream may involve similarity metrics computed over information represented elsewhere in the brain.

Retrieval Properties of Hopfield and Correlated Attractors in an Associative Memory Model

We examine a previouly introduced attractor neural network model that explains the persistent activities of neurons in the anterior ventral temporal cortex of the brain. In this model, the coexistence of several attractors including correlated attractors was reported in the cases of finite and infinite loading. In this paper, by means of a statistical mechanical method, we study the statics and dynamics of the model in both finite and extensive loading, mainly focusing on the retrieval properties of the Hopfield and correlated attractors. In the extensive loading case, we derive the evolution equations by the dynamical replica theory. We found several characteristic temporal behaviours, both in the finite and extensive loading cases. The theoretical results were confirmed by numerical simulations.

Electrophysiological studies of human face perception. I: Potentials generated in occipitotemporal cortex by face and non-face stimuli.

This and the following two papers describe event-related potentials (ERPs) evoked by visual stimuli in 98 patients in whom electrodes were placed directly upon the cortical surface to monitor medically intractable seizures. Patients viewed pictures of faces, scrambled faces, letter-strings, number-strings, and animate and inanimate objects. This paper describes ERPs generated in striate and peristriate cortex, evoked by faces, and evoked by sinusoidal gratings, objects and letter-strings. Short-latency ERPs generated in striate and peristriate cortex were sensitive to elementary stimulus features such as luminance. Three types of face-specific ERPs were found: (i) a surface-negative potential with a peak latency of approximately 200 ms (N200) recorded from ventral occipitotemporal cortex, (ii) a lateral surface N200 recorded primarily from the middle temporal gyrus, and (iii) a late positive potential (P350) recorded from posterior ventral occipitotemporal, posterior lateral temporal and anterior ventral temporal cortex. Face-specific N200s were preceded by P150 and followed by P290 and N700 ERPs. N200 reflects initial face-specific processing, while P290, N700 and P350 reflect later face processing at or near N200 sites and in anterior ventral temporal cortex. Face-specific N200 amplitude was not significantly different in males and females, in the normal and abnormal hemisphere, or in the right and left hemisphere. However, cortical patches generating ventral face-specific N200s were larger in the right hemisphere. Other cortical patches in the same region of extrastriate cortex generated grating-sensitive N180s and object-specific or letter-string-specific N200s, suggesting that the human ventral object recognition system is segregated into functionally discrete regions.

Perception, learning and identification studied with reversible suppression of cortical visual areas in monkeys

We use cold to reversibly suppress cortical areas involved in visual perception, learning and retrieval and we found a localization of functions essential for performance of delayed match-to-sample (DMS) in anterior ventral temporal cortex, we call ventral TE (TEv). We also found a visual input for this area that is separate from the one going to the heart of inferotemporal cortex and suppressing this input also impairs performance of DMS. Suppressing the dorsal half of TE (TEd) disrupts retrieval of some, but not all complex images, and different images are disrupted in different animals. This variability within and between animals is extreme, with perfect performance on some complex images and below chance on others. We suggested that TEd represents some, but not all elements of the images. In attempting to discover what those elements might be, we found that TEd suppression disrupts the perception of small figures, but not the larger figures that they compose. We also found that it impaired the discrimination and matching of colors, without impairing the ability to detect and differentiate hues. We proposed that TEd represents the details and colors of things, but not global figures. Also, complex objects do not have a representation in one area, rather its representation involves the entire visual system, including TE with different elements of the image represented in different parts.

Neuronal correlate of visual associative long-term memory in the primate temporal cortex.

In human long-term memory, ideas and concepts become associated in the learning process. No neuronal correlate for this cognitive function has so far been described, except that memory traces are thought to be localized in the cerebral cortex; the temporal lobe has been assigned as the site for visual experience because electric stimulation of this area results in imagery recall and lesions produce deficits in visual recognition of objects. We previously reported that in the anterior ventral temporal cortex of monkeys, individual neurons have a sustained activity that is highly selective for a few of the 100 coloured fractal patterns used in a visual working-memory task. Here I report the development of this selectivity through repeated trials involving the working memory. The few patterns for which a neuron was conjointly selective were frequently related to each other through stimulus-stimulus association imposed during training. The results indicate that the selectivity acquired by these cells represents a neuronal correlate of the associative long-term memory of pictures.