Human Visual Object Recognition Research Articles

A large and rich EEG dataset for modeling human visual object recognition

A large and rich EEG dataset for modeling human visual object recognition

A large and rich EEG dataset for modeling human visual object recognition

The human brain achieves visual object recognition through multiple stages of linear and nonlinear transformations operating at a millisecond scale. To predict and explain these rapid transformations, computational neuroscientists employ machine learning modeling techniques. However, state-of-the-art models require massive amounts of data to properly train, and to the present day there is a lack of vast brain datasets which extensively sample the temporal dynamics of visual object recognition. Here we collected a large and rich dataset of high temporal resolution EEG responses to images of objects on a natural background. This dataset includes 10 participants, each with 82,160 trials spanning 16,740 image conditions. Through computational modeling we established the quality of this dataset in five ways. First, we trained linearizing encoding models that successfully synthesized the EEG responses to arbitrary images. Second, we correctly identified the recorded EEG data image conditions in a zero-shot fashion, using EEG synthesized responses to hundreds of thousands of candidate image conditions. Third, we show that both the high number of conditions as well as the trial repetitions of the EEG dataset contribute to the trained models’ prediction accuracy. Fourth, we built encoding models whose predictions well generalize to novel participants. Fifth, we demonstrate full end-to-end training of randomly initialized DNNs that output EEG responses for arbitrary input images. We release this dataset as a tool to foster research in visual neuroscience and computer vision.

A Developmental Approach to Machine Learning?

Visual learning depends on both the algorithms and the training material. This essay considers the natural statistics of infant- and toddler-egocentric vision. These natural training sets for human visual object recognition are very different from the training data fed into machine vision systems. Rather than equal experiences with all kinds of things, toddlers experience extremely skewed distributions with many repeated occurrences of a very few things. And though highly variable when considered as a whole, individual views of things are experienced in a specific order – with slow, smooth visual changes moment-to-moment, and developmentally ordered transitions in scene content. We propose that the skewed, ordered, biased visual experiences of infants and toddlers are the training data that allow human learners to develop a way to recognize everything, both the pervasively present entities and the rarely encountered ones. The joint consideration of real-world statistics for learning by researchers of human and machine learning seems likely to bring advances in both disciplines.

Multivariate pattern analysis of MEG and EEG reveals the dynamics of human object processing

Despite the increasing popularity of multivariate pattern classification methods for electrophysiological data, little is known about the decoding performance of MEG vs. EEG data. The two modalities measure electromagnetic signals from the same underlying neural sources, yet they have systematic differences in sampling neural activity. Here, we investigated the extent to which such measurement differences consistently bias the information coded in MEG and EEG signals in human visual object recognition. We conducted a concurrent MEG/EEG study while participants (N=15) viewed images of 92 everyday objects and compared MEG/EEG multivariate results in both time and space. Comparison in time relied on evaluating classification time courses directly, and via representational similarity analysis (RSA). Comparison in space relied on fusion of MEG/EEG data with fMRI data based on RSA. This enabled direct localization of MEG/EEG signals with independent fMRI data, bypassing the inherent ambiguities of inverse solutions. Single image classification revealed increased MEG sensitivity to early components (peak at 112ms, 95% CI: 109-124ms), versus increased EEG sensitivity to late components (peak at 181ms; 131-195ms). Despite such bias, categorical information (animate vs. inanimate; faces vs. bodies; and others) was mostly equivalent between the two modalities. Fusion with fMRI also revealed comparable spatiotemporal dynamics for MEG and EEG. However, investigation of V1 and IT revealed unexpected results: while the two modalities equivalently matched fMRI data in V1, MEG was more similar to fMRI in IT than EEG, despite the increased sensitivity of EEG to late components. Overall, we found EEG and MEG were sensitive to partly common and partly unique aspects of visual representations. Together, our results offer a novel comparison of MEG and EEG signals in representational space, and motivate the wider adoption of multivariate analysis methods in both MEG and EEG. Meeting abstract presented at VSS 2017

Performance-optimized hierarchical models predict neural responses in higher visual cortex.

The ventral visual stream underlies key human visual object recognition abilities. However, neural encoding in the higher areas of the ventral stream remains poorly understood. Here, we describe a modeling approach that yields a quantitatively accurate model of inferior temporal (IT) cortex, the highest ventral cortical area. Using high-throughput computational techniques, we discovered that, within a class of biologically plausible hierarchical neural network models, there is a strong correlation between a model's categorization performance and its ability to predict individual IT neural unit response data. To pursue this idea, we then identified a high-performing neural network that matches human performance on a range of recognition tasks. Critically, even though we did not constrain this model to match neural data, its top output layer turns out to be highly predictive of IT spiking responses to complex naturalistic images at both the single site and population levels. Moreover, the model's intermediate layers are highly predictive of neural responses in the V4 cortex, a midlevel visual area that provides the dominant cortical input to IT. These results show that performance optimization--applied in a biologically appropriate model class--can be used to build quantitative predictive models of neural processing.

Relations Among Early Object Recognition Skills: Objects and Letters

Human visual object recognition is multifaceted and comprised of several domains of expertise. Developmental relations between young children's letter recognition and their 3-dimensional object recognition abilities are implicated on several grounds but have received little research attention. Here, we ask how preschoolers' success in recognizing letters relates to their ability to recognize 3-dimensional objects from sparse shape information alone. Seventy-three 2 1/2 to 5-year-old children completed a “Letter Recognition” task, measuring the ability to identify a named letter among 3 letters, and a “Shape Caricature Recognition” task, measuring recognition of familiar objects from sparse, abstract information about their part shapes and the spatial relations among those parts. Children also completed a control “Shape Bias” task, in which success depends on matching exact shapes but not on building an internal representation of the configuration of features characteristic of an object category or letter. Children's success in letter recognition was positively related to their shape caricature recognition scores, but not to their shape bias scores. The results suggest that letter recognition builds upon developing skills in attending to and representing the relational structure of object shape, and that these skills are common to both 2-dimensional and 3-dimensional object perception.

Tactile guidance of prey capture in Etruscan shrews

Whereas visuomotor behaviors and visual object recognition have been studied in detail, we know relatively little about tactile object representations. We investigate a new model system for the tactile guidance of behavior, namely prey (cricket) capture by one of the smallest mammals, the Etruscan shrew, Suncus etruscus. Because of their high metabolic rate and nocturnal lifestyle, Etruscan shrews are forced to detect, overwhelm, and kill prey in large numbers in darkness. Crickets are exquisitely mechanosensitive, fast-moving prey, almost as big as the shrew itself. Shrews succeed in hunting by lateralized, precise, and fast attacks. Removal experiments demonstrate that both macrovibrissae and microvibrissae are required for prey capture, with the macrovibrissae being involved in attack targeting. Experiments with artificial prey replica show that tactile shape cues are both necessary and sufficient for evoking attacks. Prey representations are motion- and size-invariant. Shrews distinguish and memorize prey features. Corrective maneuvers and cricket shape manipulation experiments indicate that shrew behavior is guided by Gestalt-like prey descriptions. Thus, tactile object recognition in Etruscan shrews shares characteristics of human visual object recognition, but it proceeds faster and occurs in a 20,000-times-smaller brain.

Object recognition: view-specificity and motion-specificity

This paper describes an experiment to distinguish between two theories of human visual object recognition. According to the view-specificity hypothesis, object recognition is based on particular learned views, whereas the motion-specificity hypothesis states that object recognition depends on particular directed view-sequences. Both hypotheses imply a degree of view-bias (i.e. recognition of a given object is associated with a small number of views). Whereas the view-specificity hypothesis attributes this view-bias to a preference for particular views, the motion-specificity hypothesis attributes view-bias to a preference for particular directed view-sequences. Results presented here suggest that recognition of 3D rotating objects involves significant view-bias. This view-bias appears to be associated with an underlying bias for particular directed view-sequences, and not for particular views.

Human visual object recognition: What have we learned from neuroimaging?

We review the neuroimaging literature for studies whose designs permit the identification of regions specialized for visual recognition. The results of these 17 studies are combined and analyzed with the goal of answering the following questions: (1) How well localized are areas involved in visual recognition across subjects? (2) Are there cortical areas that are specialized for the perception of different categories of stimuli (e.g., faces, words, and general objects)? The concept of specialization is defined and examined, as are the inferential limitations of neuroimaging methodology. Local maxima across studies were poorly colocalized within posterior inferior cortex, and there was no consistent segregation of activation sites dependent on the category of stimulus used. We discuss several possible reasons why the results of this review do not agree with the predictions of lesion and neurophysiology studies.

A sequence of object-processing stages revealed by fMRI in the human occipital lobe

Functional magnetic resonance imaging was used in combined functional selectivity and retinotopic mapping tests to reveal object-related visual areas in the human occipital lobe. Subjects were tested with right, left, up, or down hemivisual field stimuli which were composed of images of natural objects (faces, animals, man-made objects) or highly scrambled (1,024 elements) versions of the same images. In a similar fashion, the horizontal and vertical meridians were mapped to define the borders of these areas. Concurrently, the same cortical sites were tested for their sensitivity to image-scrambling by varying the number of scrambled picture fragments (from 16-1,024) while controlling for the Fourier power spectrum of the pictures and their order of presentation. Our results reveal a stagewise decrease in retinotopy and an increase in sensitivity to image-scrambling. Three main distinct foci were found in the human visual object recognition pathway (Ungerleider and Haxby [1994]: Curr Opin Neurobiol 4:157-165): 1) Retinotopic primary areas V1-3 did not exhibit significant reduction in activation to scrambled images. 2) Areas V4v (Sereno et al., [1995]: Science 268:889-893) and V3A (De Yoe et al., [1996]: Proc Natl Acad Sci USA 93:2382-2386; Tootell et al., [1997]: J Neurosci 71:7060-7078) manifested both retinotopy and decreased activation to highly scrambled images. 3) The essentially nonretinotopic lateral occipital complex (LO) (Malach et al., [1995]: Proc Natl Acad Sci USA 92:8135-8139; Tootell et al., [1996]: Trends Neurosci 19:481-489) exhibited the highest sensitivity to image scrambling, and appears to be homologous to macaque the infero-temporal (IT) cortex (Tanaka [1996]: Curr Opin Neurobiol 523-529). Breaking the images into 64, 256, or 1,024 randomly scrambled blocks reduced activation in LO voxels. However, many LO voxels remained significantly activated by mildly scrambled images (16 blocks). These results suggest the existence of object-fragment representation in LO.

A sequence of object‐processing stages revealed by fMRI in the human occipital lobe

Functional magnetic resonance imaging was used in combined functional selectivity and retinotopic mapping tests to reveal object-related visual areas in the human occpital lobe. Subjects were tested with right, left, up, or down hemivisual field stimuli which were composed of images of natural objects (faces, animals, man-made objects) or highly scrambled (1,024 elements) versions of the same images. In a similar fashion, the horizontal and vertical meridians were mapped to define the borders of these areas. Concurrently, the same cortical sites were tested for their sensitivity to image-scrambling by varying the number of scrambled picture fragments (from 16–1,024) while controlling for the Fourier power spectrum of the pictures and their order of presentation. Our results reveal a stagewise decrease in retinotopy and an increase in sensitivity to image-scrambling. Three main distinct foci were found in the human visual object recognition pathway (Ungerleider and Haxby [1994]: Curr Opin Neurobiol 4:157–165): 1) Retinotopic primary areas V1–3 did not exhibit significant reduction in activation to scrambled images. 2) Areas V4v (Sereno et al., [1995]: Science 268:889–893) and V3A (DeYoe et al., [1996]: Proc Natl Acad Sci USA 93:2382–2386; Tootell et al., [1997]: J Neurosci 71:7060–7078) manifested both retinotopy and decreased activation to highly scrambled images. 3) The essentially nonretinotopic lateral occipital complex (LO) (Malach et al., [1995]: Proc Natl Acad Sci USA 92:8135–8139; Tootell et al., [1996]: Trends Neurosci 19:481–489) exhibited the highest sensitivity to image scrambling, and appears to be homologous to macaque the infero-temporal (IT) cortex (Tanaka [1996]: Curr Opin Neurobiol 523–529). Breaking the images into 64, 256, or 1,024 randomly scrambled blocks reduced activation in LO voxels. However, many LO voxels remained significantly activated by mildly scrambled images (16 blocks). These results suggest the existence of object-fragment representation in LO. Hum. Brain Mapping 6:316–328, 1998. © 1998 Wiley-Liss, Inc.