Neural Representations Of Objects Research Articles

The roles of edge-based and surface-based information in the dynamic neural representation of objects

We combined multivariate pattern analysis (MVPA) and electroencephalogram (EEG) to investigate the role of edge, color, and other surface information in the neural representation of visual objects. Participants completed a one-back task in which they were presented with color photographs, grayscale images, and line drawings of animals, tools, and fruits. Our results provide the first neural evidence that line drawings elicit similar neural activities as color photographs and grayscale images during the 175–305 ms window after the stimulus onset. Furthermore, we found that other surface information, rather than color information, facilitates decoding accuracy in the early stages of object representations and affects the speed of this. These results provide new insights into the role of edge-based and surface-based information in the dynamic process of neural representations of visual objects.

Neuronal Population Activity in Macaque Visual Cortices Dynamically Changes through Repeated Fixations in Active Free Viewing.

During free viewing, we move our eyes and fixate on objects to recognize the visual scene of our surroundings. To investigate the neural representation of objects in this process, we studied individual and population neuronal activity in three different visual regions of the brains of macaque monkeys (Macaca fuscata): the primary and secondary visual cortices (V1, V2) and the inferotemporal cortex (IT). We designed a task where the animal freely selected objects in a stimulus image to fixate on while we examined the relationship between spiking activity, the order of fixations, and the fixated objects. We found that activity changed across repeated fixations on the same object in all three recorded areas, with observed reductions in firing rates. Furthermore, the responses of individual neurons became sparser and more selective with individual objects. The population activity for individual objects also became distinct. These results suggest that visual neurons respond dynamically to repeated input stimuli through a smaller number of spikes, thereby allowing for discrimination between individual objects with smaller energy.

Representations of Temporal Community Structure in Hippocampus and Precuneus Predict Inductive Reasoning Decisions.

Our understanding of the world is shaped by inferences about underlying structure. For example, at the gym, you might notice that the same people tend to arrive around the same time and infer that they are friends that work out together. Consistent with this idea, after participants are presented with a temporal sequence of objects that follows an underlying community structure, they are biased to infer that objects from the same community share the same properties. Here, we used fMRI to measure neural representations of objects after temporal community structure learning and examine how these representations support inference about object relationships. We found that community structure learning affected inferred object similarity: When asked to spatially group items based on their experience, participants tended to group together objects from the same community. Neural representations in perirhinal cortex predicted individual differences in object grouping, suggesting that high-level object representations are affected by temporal community learning. Furthermore, participants were biased to infer that objects from the same community would share the same properties. Using computational modeling of temporal learning and inference decisions, we found that inductive reasoning is influenced by both detailed knowledge of temporal statistics and abstract knowledge of the temporal communities. The fidelity of temporal community representations in hippocampus and precuneus predicted the degree to which temporal community membership biased reasoning decisions. Our results suggest that temporal knowledge is represented at multiple levels of abstraction, and that perirhinal cortex, hippocampus, and precuneus may support inference based on this knowledge.

Real-world size of objects serves as an axis of object space

Our mind can represent various objects from physical world in an abstract and complex high-dimensional object space, with axes encoding critical features to quickly and accurately recognize objects. Among object features identified in previous neurophysiological and fMRI studies that may serve as the axes, objects’ real-world size is of particular interest because it provides not only visual information for broad conceptual distinctions between objects but also ecological information for objects’ affordance. Here we use deep convolutional neural networks (DCNNs), which enable direct manipulation of visual experience and units’ activation, to explore how objects’ real-world size is extracted to construct the axis of object space. Like the human brain, the DCNNs pre-trained for object recognition also encode objects’ size as an independent axis of the object space. Further, we find that the shape of objects, rather than retinal size, context, task demands or texture features, is critical to inferring objects’ size for both DCNNs and humans. In short, with DCNNs as a brain-like model, our study devises a paradigm supplemental to conventional approaches to explore the structure of object space, which provides computational support for empirical observations on human perceptual and neural representations of objects.

The nature of neural object representations during dynamic occlusion

Objects disappearing briefly from sight due to occlusion is an inevitable occurrence in everyday life. Yet we generally have a strong experience that occluded objects continue to exist, despite the fact that they objectively disappear. This indicates that neural object representations must be maintained during dynamic occlusion. However, it is unclear what the nature of such representation is and in particular whether it is perception-like or more abstract, for example, reflecting limited features such as position or movement direction only. In this study, we address this question by examining how different object features such as object shape, luminance, and position are represented in the brain when a moving object is dynamically occluded. We apply multivariate decoding methods to Magnetoencephalography (MEG) data to track how object representations unfold over time. Our methods allow us to contrast the representations of multiple object features during occlusion and enable us to compare the neural responses evoked by visible and occluded objects. The results show that object position information is represented during occlusion to a limited extent while object identity features are not maintained through the period of occlusion. Together, this suggests that the nature of object representations during dynamic occlusion is different from visual representations during perception.

Where there is no object formation, there is no perceptual organization: Evidence from the configural superiority effect

Object formation is considered the aim of perceptual organization, but such a proposition has been neglected in empirical studies. In the current study, we investigated the role of object formation in configural superiority. Essentially, discrimination on bar orientations was enhanced by adding a right angle to each of the bars. Such facilitation is due to the emergent feature (EF) of closure formed by combining the bars with right angles. To study object formation, visual stimuli were generated by random dot stereograms to form objects or holes in 3D. Behaviorally, we found that the EF of closure facilitated oddball discrimination on objects, as demonstrated by previous studies, but did not facilitate oddball discrimination on holes with the same shape as objects. Multivariate pattern analysis of functional magnetic resonance imaging (fMRI) data showed that the EF of closure increased the object classification accuracy compared to the holes in the lateral occipital cortex (LOC), where object information is encoded, but not in the early visual cortex (EVC). The neural representations of objects and holes with and without EFs were further investigated using representational similarity analysis. The results demonstrate that in the LOC, the neural representations of objects with EFs showed a greater difference than those of the other three, that is, objects without EFs and holes with or without EFs. However, the uniqueness of objects with EFs was not observed in the EVC. Thus, our results suggest that the EF of closure, which leads to the configural superiority effect, only emerges for objects but not for holes, and only in the LOC but not the EVC. Our study provides the first empirical evidence suggesting that object formation plays an indispensable role in perceptual organization.

Using the force: STEM knowledge and experience construct shared neural representations of engineering concepts

How does STEM knowledge learned in school change students’ brains? Using fMRI, we presented photographs of real-world structures to engineering students with classroom-based knowledge and hands-on lab experience, examining how their brain activity differentiated them from their “novice” peers not pursuing engineering degrees. A data-driven MVPA and machine-learning approach revealed that neural response patterns of engineering students were convergent with each other and distinct from novices’ when considering physical forces acting on the structures. Furthermore, informational network analysis demonstrated that the distinct neural response patterns of engineering students reflected relevant concept knowledge: learned categories of mechanical structures. Information about mechanical categories was predominantly represented in bilateral anterior ventral occipitotemporal regions. Importantly, mechanical categories were not explicitly referenced in the experiment, nor does visual similarity between stimuli account for mechanical category distinctions. The results demonstrate how learning abstract STEM concepts in the classroom influences neural representations of objects in the world.

Emergence of an Invariant Representation of Texture in Primate Somatosensory Cortex.

A major function of sensory processing is to achieve neural representations of objects that are stable across changes in context and perspective. Small changes in exploratory behavior can lead to large changes in signals at the sensory periphery, thus resulting in ambiguous neural representations of objects. Overcoming this ambiguity is a hallmark of human object recognition across sensory modalities. Here, we investigate how the perception of tactile texture remains stable across exploratory movements of the hand, including changes in scanning speed, despite the concomitant changes in afferent responses. To this end, we scanned a wide range of everyday textures across the fingertips of rhesus macaques at multiple speeds and recorded the responses evoked in tactile nerve fibers and somatosensory cortical neurons (from Brodmann areas 3b, 1, and 2). We found that individual cortical neurons exhibit a wider range of speed-sensitivities than do nerve fibers. The resulting representations of speed and texture in cortex are more independent than are their counterparts in the nerve and account for speed-invariant perception of texture. We demonstrate that this separation of speed and texture information is a natural consequence of previously described cortical computations.

A data-driven approach to stimulus selection reveals an image-based representation of objects in high-level visual areas.

The ventral visual pathway is directly involved in the perception and recognition of objects. However, the extent to which the neural representation of objects in this region reflects low-level or high-level properties remains unresolved. A problem in resolving this issue is that only a small proportion of the objects experienced during natural viewing can be shown during a typical experiment. This can lead to an uneven sampling of objects that biases our understanding of how they are represented. To address this issue, we developed a data-driven approach to stimulus selection that involved describing a large number objects in terms of their image properties. In the first experiment, clusters of objects were evenly selected from this multi-dimensional image space. Although the clusters did not have any consistent semantic features, each elicited a distinct pattern of neural response. In the second experiment, we asked whether high-level, category-selective patterns of response could be elicited by objects from other categories, but with similar image properties. Object clusters were selected based on the similarity of their image properties to objects from five different categories (bottle, chair, face, house, and shoe). The pattern of response to each metameric object cluster was similar to the pattern elicited by objects from the corresponding category. For example, the pattern for bottles was similar to the pattern for objects with similar image properties to bottles. In both experiments, the patterns of response were consistent across participants providing evidence for common organising principles. This study provides a more ecological approach to understanding the perceptual representations of objects and reveals the importance of image properties.

Ultrafast Object Detection in Naturalistic Vision Relies on Ultrafast Distractor Suppression.

People are quicker to detect examples of real-world object categories in natural scenes than is predicted by classic attention theories. One explanation for this puzzle suggests that experience renders the visual system sensitive to midlevel features diagnosing target presence. These are detected without the need for spatial attention, much as occurs for targets defined by low-level features like color or orientation. The alternative is that naturalistic search relies on spatial attention but is highly efficient because global scene information can be used to quickly reject nontarget objects and locations. Here, we use ERPs to differentiate between these possibilities. Results show that hallmark evidence of ultrafast target detection in frontal brain activity is preceded by an index of spatially specific distractor suppression in visual cortex. Naturalistic search for heterogenous targets therefore appears to rely on spatial operations that act on neural object representations, as predicted by classic attention theory. People appear able to rapidly reject nontarget objects and locations, consistent with the idea that global scene information is used to constrain naturalistic search and increase search efficiency.

Ultrafast object detection in naturalistic vision relies on ultrafast distractor suppression

People are quicker to detect examples of real-world object categories in natural scenes than is predicted by classic attention theories. One explanation for this puzzle suggests that experience renders the visual system sensitive to midlevel features diagnosing target presence. These are detected without the need for spatial attention, much as occurs for targets defined by low-level features like color or orientation. The alternative is that naturalistic search relies on spatial attention but is highly efficient because global scene information can be used to quickly reject nontarget objects and locations. Here, we use ERPs to differentiate between these possibilities. Results show that hallmark evidence of ultrafast target detection in frontal brain activity is preceded by an index of spatially specific distractor suppression in visual cortex. Naturalistic search for heterogenous targets therefore appears to rely on spatial operations that act on neural object representations, as predicted by classic attention theory. People appear able to rapidly reject nontarget objects and locations, consistent with the idea that global scene information is used to constrain naturalistic search and increase search efficiency.

Illusory edges comingle with real edges in the neural representation of objects

The visual system must transform a point-by-point biological representation from the photoreceptors into neural representations of separate objects. Even a uniform circular patch of light that slowly modulates in luminance can be segmented into separate central and surrounding areas merely by introducing black lines to outline a central square. The black lines cause brightness induction in the center even though the light inside and outside the square is always identical, as predicted by spatial antagonism between the square central area and its surround. Importantly, illusory Kanizsa lines forming the square are as effective for this brightness induction as real black lines, suggesting a 'form-cue invariant' cortical neural representation that does not distinguish between a central region set off by real or illusory edges. An open question is whether separate subsystems generate objects defined by real versus illusory edges, each providing the same form-cue invariant neural representation of an object, or whether form-cue invariance extends to integrating component pieces that together define an object. Experiments here show object segmentation when subparts of a square are defined by a mixture of real and illusory edges. Subjects adjusted the Michelson contrast of a separate patch to match the perceived modulation depth within the central region of a circular field that slowly oscillated in luminance. A closed, four-sided figure, no matter how constructed, reduced the perceived modulation depth within the central region. This shows that both real and illusory subparts can be integrated to segment center from surround. It supports a strong version of form-cue invariance in which neural mechanisms responsible for object segmentation are impartial to the piecemeal cues that are integrated to define an object.

A data-driven approach to stimulus selection reveals the importance of visual properties in the neural representation of objects.

A data-driven approach to stimulus selection reveals the importance of visual properties in the neural representation of objects.

Mid-level perceptual features contain early cues to animacy.

While substantial work has focused on how the visual system achieves basic-level recognition, less work has asked about how it supports large-scale distinctions between objects, such as animacy and real-world size. Previous work has shown that these dimensions are reflected in our neural object representations (Konkle & Caramazza, 2013), and that objects of different real-world sizes have different mid-level perceptual features (Long, Konkle, Cohen, & Alvarez, 2016). Here, we test the hypothesis that animates and manmade objects also differ in mid-level perceptual features. To do so, we generated synthetic images of animals and objects that preserve some texture and form information ("texforms"), but are not identifiable at the basic level. We used visual search efficiency as an index of perceptual similarity, as search is slower when targets are perceptually similar to distractors. Across three experiments, we find that observers can find animals faster among objects than among other animals, and vice versa, and that these results hold when stimuli are reduced to unrecognizable texforms. Electrophysiological evidence revealed that this mixed-animacy search advantage emerges during early stages of target individuation, and not during later stages associated with semantic processing. Lastly, we find that perceived curvature explains part of the mixed-animacy search advantage and that observers use perceived curvature to classify texforms as animate/inanimate. Taken together, these findings suggest that mid-level perceptual features, including curvature, contain cues to whether an object may be animate versus manmade. We propose that the visual system capitalizes on these early cues to facilitate object detection, recognition, and classification.

Neural representation of geometry and surface properties in object and scene perception

Multiple cortical regions are crucial for perceiving the visual world, yet the processes shaping representations in these regions are unclear. To address this issue, we must elucidate how perceptual features shape representations of the environment. Here, we explore how the weighting of different visual features affects neural representations of objects and scenes, focusing on the scene-selective parahippocampal place area (PPA), but additionally including the retrosplenial complex (RSC), occipital place area (OPA), lateral occipital (LO) area, fusiform face area (FFA) and occipital face area (OFA). Across three experiments, we examined functional magnetic resonance imaging (fMRI) activity while human observers viewed scenes and objects that varied in geometry (shape/layout) and surface properties (texture/material). Interestingly, we found equal sensitivity in the PPA for these properties within a scene, revealing that spatial-selectivity alone does not drive activation within this cortical region. We also observed sensitivity to object texture in PPA, but not to the same degree as scene texture, and representations in PPA varied when objects were placed within scenes. We conclude that PPA may process surface properties in a domain-specific manner, and that the processing of scene texture and geometry is equally-weighted in PPA and may be mediated by similar underlying neuronal mechanisms.

Neural Representations of Natural and Scrambled Movies Progressively Change from Rat Striate to Temporal Cortex.

In recent years, the rodent has come forward as a candidate model for investigating higher level visual abilities such as object vision. This view has been backed up substantially by evidence from behavioral studies that show rats can be trained to express visual object recognition and categorization capabilities. However, almost no studies have investigated the functional properties of rodent extrastriate visual cortex using stimuli that target object vision, leaving a gap compared with the primate literature. Therefore, we recorded single-neuron responses along a proposed ventral pathway in rat visual cortex to investigate hallmarks of primate neural object representations such as preference for intact versus scrambled stimuli and category-selectivity. We presented natural movies containing a rat or no rat as well as their phase-scrambled versions. Population analyses showed increased dissociation in representations of natural versus scrambled stimuli along the targeted stream, but without a clear preference for natural stimuli. Along the measured cortical hierarchy the neural response seemed to be driven increasingly by features that are not V1-like and destroyed by phase-scrambling. However, there was no evidence for category selectivity for the rat versus nonrat distinction. Together, these findings provide insights about differences and commonalities between rodent and primate visual cortex.

Editorial: Integrating Computational and Neural Findings in Visual Object Perception.

Recognizing objects despite infinite variations in their appearance is a highly challenging computational task the visual system performs in a remarkably fast, accurate, and robust fashion. The complexity of the underlying mechanisms is reflected in the large proportion of cortical real-estate dedicated to visual processing, as well as in the difficulties encountered when trying to build models whose performance matches human proficiency. The articles in this Research Topic provide an overview of recent advances in our understanding of the neural mechanisms underlying visual object perception, focusing on integrative approaches which encompass both computational and empirical work. Given the vast expanse of topics covered in the discipline of computational visual neuroscience, it is impossible to provide a comprehensive overview of the field's status-quo. Instead, the presented papers highlight interesting extensions to existing models and novel insights into computational principles and their neural underpinnings. Contributions could be coarsely subdivided into three different sections: Two papers focused on implementing biologically-valid learning rules and heuristics in well-established neural models of the visual pathway (i.e., “VisNet” and “HMAX”) to improve flexible object recognition. Three other studies investigated the role of sparseness, selectivity, and correlation in optimizing neural coding of object features. Finally, another set of contributions focused on integrating computational vision models and human brain responses to gain more insights in the computational mechanisms underlying neural object representations.

The neural representation of objects formed through the spatiotemporal integration of visual transients

Oftentimes, objects are only partially and transiently visible as parts of them become occluded during observer or object motion. The visual system can integrate such object fragments across space and time into perceptual wholes or spatiotemporal objects. This integrative and dynamic process may involve both ventral and dorsal visual processing pathways, along which shape and spatial representations are thought to arise. We measured fMRI BOLD response to spatiotemporal objects and used multi-voxel pattern analysis (MVPA) to decode shape information across 20 topographic regions of visual cortex. Object identity could be decoded throughout visual cortex, including intermediate (V3A, V3B, hV4, LO1–2,) and dorsal (TO1–2, and IPS0–1) visual areas. Shape-specific information, therefore, may not be limited to early and ventral visual areas, particularly when it is dynamic and must be integrated. Contrary to the classic view that the representation of objects is the purview of the ventral stream, intermediate and dorsal areas may play a distinct and critical role in the construction of object representations across space and time.

The origins of word learning: Brain responses of 3‐month‐olds indicate their rapid association of objects and words

The present study explored the origins of word learning in early infancy. Using event-related potentials (ERP) we monitored the brain activity of 3-month-old infants when they were repeatedly exposed to several initially novel words paired consistently with each the same initially novel objects or inconsistently with different objects. Our results provide strong evidence that these young infants extract statistic regularities in the distribution of the co-occurrences of objects and words extremely quickly. The data suggest that this ability is based on the rapid formation of associations between the neural representations of objects and words, but that the new associations are not retained in long-term memory until the next day. The type of brain response moreover indicates that, unlike in older infants, in 3-month-olds a semantic processing stage is not involved. Their ability to combine words with meaningful information is caused by a primary learning mechanism that enables the formation of proto-words and acts as a precursor for the acquisition of genuine words.

Size precedes view: developmental emergence of invariant object representations in lateral occipital complex.

Although object perception involves encoding a wide variety of object properties (e.g., size, color, viewpoint), some properties are irrelevant for identifying the object. The key to successful object recognition is having an internal representation of the object identity that is insensitive to these properties while accurately representing important diagnostic features. Behavioral evidence indicates that the formation of these kinds of invariant object representations takes many years to develop. However, little research has investigated the developmental emergence of invariant object representations in the ventral visual processing stream, particularly in the lateral occipital complex (LOC) that is implicated in object processing in adults. Here, we used an fMR adaptation paradigm to evaluate age-related changes in the neural representation of objects within LOC across variations in size and viewpoint from childhood through early adulthood. We found a dissociation between the neural encoding of object size and object viewpoint within LOC: by age of 5-10 years, area LOC demonstrates adaptation across changes in size, but not viewpoint, suggesting that LOC responses are invariant to size variations, but that adaptation across changes in view is observed in LOC much later in development. Furthermore, activation in LOC was correlated with behavioral indicators of view invariance across the entire sample, such that greater adaptation was correlated with better recognition of objects across changes in viewpoint. We did not observe similar developmental differences within early visual cortex. These results indicate that LOC acquires the capacity to compute invariance specific to different sources of information at different time points over the course of development.