Neurons In Inferotemporal Cortex Research Articles

Object representation in a gravitational reference frame.

When your head tilts laterally, as in sports, reaching, and resting, your eyes counterrotate less than 20%, and thus eye images rotate, over a total range of about 180°. Yet, the world appears stable and vision remains normal. We discovered a neural strategy for rotational stability in anterior inferotemporal cortex (IT), the final stage of object vision in primates. We measured object orientation tuning of IT neurons in macaque monkeys tilted +25 and -25° laterally, producing ~40° difference in retinal image orientation. Among IT neurons with consistent object orientation tuning, 63% remained stable with respect to gravity across tilts. Gravitational tuning depended on vestibular/somatosensory but also visual cues, consistent with previous evidence that IT processes scene cues for gravity's orientation. In addition to stability across image rotations, an internal gravitational reference frame is important for physical understanding of a world where object position, posture, structure, shape, movement, and behavior interact critically with gravity.

Contribution of individual features to repetition suppression in macaque inferotemporal cortex.

When an image is presented twice in succession, neurons in area TE of macaque inferotemporal cortex exhibit repetition suppression, responding less strongly to the second presentation than to the first. Suppression is known to occur if the adapter and the test image are subtly different from each other. However, it is not known whether cross suppression occurs between images that are radically different from each other but that share a subset of features. To explore this issue, we measured repetition suppression using colored shapes. On interleaved trials, the test image might be identical to the adapter, might share its shape or color alone, or might differ from it totally. At the level of the neuronal population as a whole, suppression was especially deep when adapter and test were identical, intermediate when they shared only one attribute, and minimal when they shared neither attribute. At the level of the individual neuron, the degree of suppression depended not only on the properties of the two images but also on the preferences of the neuron. Suppression was deeper when the repeated color or shape was preferred by the neuron than when it was not. This effect might arise from feature-specific adaptation or alternatively from adapter-induced fatigue. Both mechanisms conform to the principle that the degree of suppression is determined by the preferences of the neuron.NEW & NOTEWORTHY When an image is presented twice in rapid succession, neurons of inferotemporal cortex exhibit repetition suppression, responding less strongly to the second than to the first presentation. It has been unclear whether this phenomenon depends on the selectivity of the neuron under study. Here, we show that, for a given neuron, suppression is deepest when features preferred by that neuron are repeated. The results argue for a mechanism based either on feature-specific suppression or fatigue.

Contribution of receptive field center and surround to repetition suppression in macaque visual area V2

Monkey inferotemporal cortex (ITC) neurons respond with declining strength to repeated presentations of relatively large and complex natural images. This phenomenon – repetition suppression – has often been assumed to arise at the level of ITC because ITC neurons possess the large receptive fields and sophisticated stimulus selectivity necessary for recognizing the image as a repetition. It has been recently discovered, however, that neurons in V2 exhibit repetition suppression under conditions identical to those in studies of ITC. This raises the question: How do V2 neurons, with classical receptive fields encompassing only a small fraction of the image, recognize it as a repetition? One possibility is that they are sensitive to repetition of image content not only in the classical receptive field but also in the receptive field surround. To assess this possibility, we monitored neuronal responses to sequential displays in which we controlled independently the repetition of elements in the classical receptive field and in the surround. Each stimulus consisted of a disk scaled to and centered on the classical receptive field and an adjoining annulus (8° outer diameter). The disks and annuli were taken from different natural scenes. The display on each trial consisted of a prime, a delay, and a probe, each 300 ms in duration. We found that repetition of the central disk was sufficient to produce repetition suppression but that suppression was enhanced by simultaneous repetition of the annulus. Suppression arising from repetition of the annulus occurred in a relatively late phase of the response, in accordance with the idea that it might have been mediated by horizontal connections in V2 and/or feedback from downstream areas. Overall, this work demonstrates that sensitivity to content outside the classical receptive field contributes to the image specificity of repetition suppression observed in V2 neurons to large natural images.

Mixed selectivity for visual features and economic value in the primate orbitofrontal cortex

Primates use their acute sense of vision not only to identify objects, but also to assess their value, that is, their potential for benefit or harm. How the brain transforms visual information into value information is still poorly understood, but recent findings suggest a key role for the orbitofrontal cortex (OFC). The OFC includes several cytoarchitectonic areas within the ventral frontal lobe, and has a long-recognized role in representing object value and organizing value-driven behavior. One of the OFC’s most striking anatomical features is the massive, direct input it receives from the inferotemporal cortex, a ventral temporal region implicated in object identification. A natural hypothesis, therefore, is that in addition to well-documented value coding properties, OFC neurons may also represent visual features in a manner similar to neurons in the ventral visual stream. To test this hypothesis, we recorded OFC neurons in macaque monkeys performing behavioral tasks in which the value of visible objects was manipulated independently from their visual features. Preliminary findings include a subset of OFC cells that were modulated by object value, but only in response to objects that shared a particular visual feature (e.g. the color red). This form of ‘mixed’ selectivity suggests that the OFC may be an intermediate computational stage between visual identification and value retrieval. Moreover, recent work showing similar mixed value-feature selectivity in inferotemporal cortex neurons suggests that neural mechanisms of object valuation may be distributed over a continuum of cortical regions, rather than compartmentalized in a strict hierarchy.

Single neurons may encode simultaneous stimuli by switching between activity patterns

How the brain preserves information about multiple simultaneous items is poorly understood. We report that single neurons can represent multiple stimuli by interleaving signals across time. We record single units in an auditory region, the inferior colliculus, while monkeys localize 1 or 2 simultaneous sounds. During dual-sound trials, we find that some neurons fluctuate between firing rates observed for each single sound, either on a whole-trial or on a sub-trial timescale. These fluctuations are correlated in pairs of neurons, can be predicted by the state of local field potentials prior to sound onset, and, in one monkey, can predict which sound will be reported first. We find corroborating evidence of fluctuating activity patterns in a separate dataset involving responses of inferotemporal cortex neurons to multiple visual stimuli. Alternation between activity patterns corresponding to each of multiple items may therefore be a general strategy to enhance the brain processing capacity, potentially linking such disparate phenomena as variable neural firing, neural oscillations, and limits in attentional/memory capacity.

連合学習を経験した三次元物体の観察角度像に対する側頭葉下部皮質神経細胞集団の応答

Object recognition requires discrimination across different objects in spite of the view change of the same object. In our previous experiment, single neurons of inferotemporal cortex responding to the training object sets that the monkeys had experienced in association between views of the objects across up to 90 deg in viewing angle responded with a viewing angle tuning curve width of about 60 deg. To reveal the neuronal mechanism underlying the association training, we analyzed the response patterns of cell population to the object images experienced in the object discrimination across up to 90 deg. We found that the similarities for response patterns of views of the same objects were significantly larger than that across different objects for viewing angle difference of 30 deg, 60 deg, and 90 deg. The results suggest that the object discrimination across large viewing angle could be explained by the neuronal activity of cell population in inferotemporal cortex.

Coding of relative size in monkey inferotemporal cortex.

We seldom mistake a closer object as being larger, even though its retinal image is bigger. One underlying mechanism could be to calculate the size of the retinal image relative to that of another nearby object. Here we set out to investigate whether single neurons in the monkey inferotemporal cortex (IT) are sensitive to the relative size of parts in a display. Each neuron was tested on shapes containing two parts that could be conjoined or spatially separated. Each shape was presented in four versions created by combining the two parts at each of two possible sizes. In this design, neurons sensitive to the absolute size of parts would show the greatest response modulation when both parts are scaled up, whereas neurons encoding relative size would show similar responses. Our main findings are that 1) IT neurons responded similarly to all four versions of a shape, but tuning tended to be more consistent between versions with proportionately scaled parts; 2) in a subpopulation of cells, we observed interactions that resulted in similar responses to proportionately scaled parts; 3) these interactions developed together with sensitivity to absolute size for objects with conjoined parts but developed slightly later for objects with spatially separate parts. Taken together, our results demonstrate for the first time that there is a subpopulation of neurons in IT that encodes the relative size of parts in a display, forming a potential neural substrate for size constancy.

A Channel for 3D Environmental Shape in Anterior Inferotemporal Cortex

Inferotemporal cortex (IT) has long been studied as a single pathway dedicated to object vision, but connectivity analysis reveals anatomically distinct channels, through ventral superior temporal sulcus (STSv) and dorsal/ventral inferotemporal gyrus (TEd, TEv). Here, we report a major functional distinction between channels. We studied individual IT neurons in monkeys viewing stereoscopic 3D images projected on a large screen. We used adaptive stimuli to explore neural tuning for 3D abstract shapes ranging in scale and topology from small, closed, bounded objects to large, open, unbounded environments (landscape-like surfaces and cave-like interiors). In STSv, most neurons were more responsive to objects, as expected. In TEd, surprisingly, most neurons were more responsive to 3D environmental shape. Previous studies have localized environmental information to posterior cortical modules. Our results show it is also channeled through anterior IT, where extensive cross-connections between STSv and TEd could integrate object and environmental shape information.

Image familiarization sharpens response dynamics of neurons in inferotemporal cortex.

Repeated viewing of an image over days and weeks induces a marked reduction in the strength with which neurons in monkey inferotemporal cortex respond to it. The processing advantage that attaches to this reduction is unknown. One possibility is that truncation of the response to a familiar image leaves neurons in a state of readiness to respond to ensuing images and thereby enhances their ability to track rapidly changing displays. We explored this possibility by assessing neuronal responses to familiar and novel images in rapid serial visual displays. Inferotemporal neurons responded more strongly to familiar than to novel images in such displays. The effect was stronger among putative inhibitory neurons than among putative excitatory neurons. A comparable effect occurred at the level of the scalp potential in humans. We conclude that long-term familiarization sharpens the response dynamics of neurons in both monkey and human extrastriate visual cortex.

Dynamic target match signals in perirhinal cortex can be explained by instantaneous computations that act on dynamic input from inferotemporal cortex.

Finding sought objects requires the brain to combine visual and target signals to determine when a target is in view. To investigate how the brain implements these computations, we recorded neural responses in inferotemporal cortex (IT) and perirhinal cortex (PRH) as macaque monkeys performed a delayed-match-to-sample target search task. Our data suggest that visual and target signals were combined within or before IT in the ventral visual pathway and then passed onto PRH, where they were reformatted into a more explicit target match signal over ∼10-15 ms. Accounting for these dynamics in PRH did not require proposing dynamic computations within PRH itself but, rather, could be attributed to instantaneous PRH computations performed upon an input representation from IT that changed with time. We found that the dynamics of the IT representation arose from two commonly observed features: individual IT neurons whose response preferences were not simply rescaled with time and variable response latencies across the population. Our results demonstrate that these types of time-varying responses have important consequences for downstream computation and suggest that dynamic representations can arise within a feedforward framework as a consequence of instantaneous computations performed upon time-varying inputs.

Ventral-stream-like shape representation: from pixel intensity values to trainable object-selective COSFIRE models.

The remarkable abilities of the primate visual system have inspired the construction of computational models of some visual neurons. We propose a trainable hierarchical object recognition model, which we call S-COSFIRE (S stands for Shape and COSFIRE stands for Combination Of Shifted FIlter REsponses) and use it to localize and recognize objects of interests embedded in complex scenes. It is inspired by the visual processing in the ventral stream (V1/V2 → V4 → TEO). Recognition and localization of objects embedded in complex scenes is important for many computer vision applications. Most existing methods require prior segmentation of the objects from the background which on its turn requires recognition. An S-COSFIRE filter is automatically configured to be selective for an arrangement of contour-based features that belong to a prototype shape specified by an example. The configuration comprises selecting relevant vertex detectors and determining certain blur and shift parameters. The response is computed as the weighted geometric mean of the blurred and shifted responses of the selected vertex detectors. S-COSFIRE filters share similar properties with some neurons in inferotemporal cortex, which provided inspiration for this work. We demonstrate the effectiveness of S-COSFIRE filters in two applications: letter and keyword spotting in handwritten manuscripts and object spotting in complex scenes for the computer vision system of a domestic robot. S-COSFIRE filters are effective to recognize and localize (deformable) objects in images of complex scenes without requiring prior segmentation. They are versatile trainable shape detectors, conceptually simple and easy to implement. The presented hierarchical shape representation contributes to a better understanding of the brain and to more robust computer vision algorithms.

Statistical learning of serial visual transitions by neurons in monkey inferotemporal cortex.

If monkeys repeatedly, over the course of weeks, view displays in which two images appear in fixed sequence, then neurons of inferotemporal cortex (ITC) come to exhibit prediction suppression. The response to the trailing image is weaker if it follows the leading image with which it was paired during training than if it follows some other leading image. Prediction suppression is a plausible neural mechanism for statistical learning of visual transitions such as has been demonstrated in behavioral studies of human infants and adults. However, in the human studies, subjects are exposed to continuous sequences in which the same image can be both predicted and predicting and statistical dependency can exist between nonadjacent items. The aim of the present study was to investigate whether prediction suppression in ITC develops under such circumstances. To resolve this issue, we exposed monkeys repeatedly to triplets of images presented in fixed order. The results indicate that prediction suppression can be induced by training not only with pairs of images but also with longer sequences.

The Effects of Prefrontal Cortex Inactivation on Object Responses of Single Neurons in the Inferotemporal Cortex during Visual Search

Inferotemporal cortex (IT) is believed to be directly involved in object processing and necessary for accurate and efficient object recognition. The frontal eye field (FEF) is an area in the primate prefrontal cortex that is involved in visual spatial selection and is thought to guide spatial attention and eye movements. We show that object-selective responses of IT neurons and behavioral performance are affected by changes in frontal eye field activity. This was found in monkeys performing a search classification task by temporarily inactivating subregions of FEF while simultaneously recording the activity from single neurons in IT. The effect on object selectivity and performance was specific, occurring in a predictable spatially dependent manner and was strongest when the IT neuron's preferred target was presented in the presence of distractors. FEF inactivation did not affect IT responses on trials in which the nonpreferred target was presented in the search array.

Global Image Dissimilarity in Macaque Inferotemporal Cortex Predicts Human Visual Search Efficiency

Finding a target in a visual scene can be easy or difficult depending on the nature of the distractors. Research in humans has suggested that search is more difficult the more similar the target and distractors are to each other. However, it has not yielded an objective definition of similarity. We hypothesized that visual search performance depends on similarity as determined by the degree to which two images elicit overlapping patterns of neuronal activity in visual cortex. To test this idea, we recorded from neurons in monkey inferotemporal cortex (IT) and assessed visual search performance in humans using pairs of images formed from the same local features in different global arrangements. The ability of IT neurons to discriminate between two images was strongly predictive of the ability of humans to discriminate between them during visual search, accounting overall for 90% of the variance in human performance. A simple physical measure of global similarity--the degree of overlap between the coarse footprints of a pair of images--largely explains both the neuronal and the behavioral results. To explain the relation between population activity and search behavior, we propose a model in which the efficiency of global oddball search depends on contrast-enhancing lateral interactions in high-order visual cortex.

Learning alters theta-nested gamma oscillations in inferotemporal cortex

AbstractHow coupled brain rhythms influence cortical information processing to support learning is unresolved. Local field potential and neuronal activity recordings from 64- electrode arrays in sheep inferotemporal cortex showed that visual discrimination learning increased the amplitude of theta oscillations during stimulus presentation. Coupling between theta and gamma oscillations, the theta/gamma ratio and the regularity of theta phase were also increased, but not neuronal firing rates. A neural network model with fast and slow inhibitory interneurons was developed which generated theta nested gamma. By increasing N-methyl-D-aspartate receptor sensitivity similar learning-evoked changes could be produced. The model revealed that altered theta nested gamma could potentiate downstream neuron responses by temporal desynchronization of excitatory neuron output independent of changes in overall firing frequency. This learning-associated desynchronization was also exhibited by inferotemporal cortex neurons. Changes in theta nested gamma may therefore facilitate learning-associated potentiation by temporal modulation of neuronal firing.

Learning alters theta-nested gamma oscillations in inferotemporal cortex

AbstractHow coupled brain rhythms influence cortical information processing to support learning is unresolved. Local field potential and neuronal activity recordings from 64- electrode arrays in sheep inferotemporal cortex showed that visual discrimination learning increased the amplitude of theta oscillations during stimulus presentation. Coupling between theta and gamma oscillations, the theta/gamma ratio and the regularity of theta phase were also increased, but not neuronal firing rates. A neural network model with fast and slow inhibitory interneurons was developed which generated theta nested gamma. By increasing N-methyl-D-aspartate receptor sensitivity similar learning-evoked changes could be produced. The model revealed that altered theta nested gamma could potentiate downstream neuron responses by temporal desynchronization of excitatory neuron output independent of changes in overall firing frequency. This learning-associated desynchronization was also exhibited by inferotemporal cortex neurons. Changes in theta nested gamma may therefore facilitate learning-associated potentiation by temporal modulation of neuronal firing.

Linearly additive shape and color signals in monkey inferotemporal cortex.

How does the brain represent a red circle? One possibility is that there is a specialized and possibly time-consuming process whereby the attributes of shape and color, carried by separate populations of neurons in low-order visual cortex, are bound together into a unitary neural representation. Another possibility is that neurons in high-order visual cortex are selective, by virtue of their bottom-up input from low-order visual areas, for particular conjunctions of shape and color. A third possibility is that they simply sum shape and color signals linearly. We tested these ideas by measuring the responses of inferotemporal cortex neurons to sets of stimuli in which two attributes-shape and color-varied independently. We find that a few neurons exhibit conjunction selectivity but that in most neurons the influences of shape and color sum linearly. Contrary to the idea of conjunction coding, few neurons respond selectively to a particular combination of shape and color. Contrary to the idea that binding requires time, conjunction signals, when present, occur as early as feature signals. We argue that neither conjunction selectivity nor a specialized feature binding process is necessary for the effective representation of shape-color combinations.

Robust learning of position invariant visual representations with OFF responses

Event Abstract Back to Event Robust learning of position invariant visual representations with OFF responses Our ability to recognize objects in spite of variations in size or position is important for a successful interaction with the environment. One possible neural basis for this ability is the visual representations that have been found in the temporal lobe of primates. The firing rate of neurons in these regions have been shown to be object specific but largely invariant to the retinal position and scale of the object [1]. Recent experiments by DiCarlo and colleagues suggest that the position invariance of visual representations in inferotemporal cortex (IT) of monkeys are learned by exploiting the temporal contiguity of our environment [2,3]. The rational behind the learning paradigm DiCarlo tested is that the visual stimuli received before and after a saccade are likely to contain the same objects, but at different retinal positions. Therefore, position invariant representations can be learned by changing the sensory processing such that the firing rate of neurons in IT in response to the visual stimuli before and after saccade should be the same. Any learning rule that implements this idea has to compare the pre- and postsaccadic stimulus and therefore requires a memory trace that stores either the identity of the presaccadic stimulus or the firing rate of the neuron in response to that stimulus. Here, we propose that this memory trace could be represented by neuronal OFF responses. OFF responses are short periods of increased activity after the offset of a stimulus, which are displayed by many neurons in the visual system. Because these responses are stimulus specific while overlapping temporally with the initial response to the next stimulus, they introduce correlations between responses to stimuli that occur in temporal succession. Here, we show that a simple rate-based model that combines OFF responses with Hebbian learning and homeostatic plasticity allows (a) the unsupervised learning of position invariant representations, (b) the reproduction of the experimental results of Li and DiCarlo [2] and (c) the development of neuronal selectivity with a qualitative match to experimental results on neuronal selectivity in IT [4]. Our simulations show that learning with OFF responses implements a time scale invariance in that it is very robust to variations in the intersaccade interval. A comparison shows that the trace rule [5], the classical online rule for learning invariances, is more sensitive to fixation time variations. Our results suggest a new computational role for OFF responses in sensory learning and show that we may lose interesting learning mechanisms by concentrating on mean firing rates and neglecting the temporal response pattern of sensory neurons.

Modulation of neural responses in inferotemporal cortex during the interpretation of ambiguous photographs

Ambiguous images are interpreted in the context of biases about what they might be; these biases and the behavioral consequences induced by them may influence the processing of images. In this report, we examine neural responses in inferotemporal cortex (IT) during the interpretation of ambiguous photographs created by morphing between two photographs. Monkeys classified different images as being one of two choices and learned to classify most of the samples correctly. For one image (the ambiguous sample) reward was administered randomly for either possible choice, and the monkeys were free to classify that image based on their own interpretation, with no learning possible. The ambiguous samples were not classified randomly: the monkey interpreted the samples differently during different sessions. The interpretation of the ambiguous sample was, in turn, highly correlated with the normalized response of individual neurons in IT to the ambiguous sample. If an ambiguous sample was interpreted as a particular choice during a session, the response to that ambiguous sample more closely resembled the response to that choice. Identical ambiguous images were interpreted differently during different sessions, and neural responses reflected the differing interpretations of the image during that session. The relationship between the interpretation of the image and neural responses strengthened over the course of a session because neural responses shifted to more closely resemble the response to the initial interpretation of the image. The data support a flexible representation of visual stimuli in higher visual areas.

Trade-Off between Object Selectivity and Tolerance in Monkey Inferotemporal Cortex

Object recognition requires both selectivity among different objects and tolerance to vastly different retinal images of the same object, resulting from natural variation in (e.g.) position, size, illumination, and clutter. Thus, discovering neuronal responses that have object selectivity and tolerance to identity-preserving transformations is fundamental to understanding object recognition. Although selectivity and tolerance are found at the highest level of the primate ventral visual stream [the inferotemporal cortex (IT)], both properties are highly varied and poorly understood. If an IT neuron has very sharp selectivity for a unique combination of object features ("diagnostic features"), this might automatically endow it with high tolerance. However, this relationship cannot be taken as given; although some IT neurons are highly object selective and some are highly tolerant, the empirical connection of these key properties is unknown. In this study, we systematically measured both object selectivity and tolerance to different identity-preserving image transformations in the spiking responses of a population of monkey IT neurons. We found that IT neurons with high object selectivity typically have low tolerance (and vice versa), regardless of how object selectivity was quantified and the type of tolerance examined. The discovery of this trade-off illuminates object selectivity and tolerance in IT and unifies a range of previous, seemingly disparate results. This finding also argues against the idea that diagnostic conjunctions of features guarantee tolerance. Instead, it is naturally explained by object recognition models in which object selectivity is built through AND-like tuning mechanisms.