Object Recognition In Humans Research Articles

Neural response time analysis: Explainable artificial intelligence using only a stopwatch

AbstractHow would you describe the features that a deep learning model composes if you were restricted to measuring observable behaviours? Explainable artificial intelligence (XAI) methods rely on privileged access to model architecture and parameters that is not always feasible for most users, practitioners and regulators. Inspired by cognitive psychology research on humans, we present a case for measuring response times (RTs) of a forward pass using only the system clock as a technique for XAI. Our method applies to the growing class of models that use input‐adaptive dynamic inference and we also extend our approach to standard models that are converted to dynamic inference post hoc. The experimental logic is simple: If the researcher can contrive a stimulus set where variability among input features is tightly controlled, differences in RT for those inputs can be attributed to the way the model composes those features. First, we show that RT is sensitive to difficult, complex features by comparing RTs from ObjectNet and ImageNet. Next, we make specific a priori predictions about RT for abstract features present in the SCEGRAM data set, where object recognition in humans depends on complex intrascene object‐object relationships. Finally, we show that RT profiles bear specificity for class identity and therefore the features that define classes. These results cast light on the model's feature space without opening the black box.

Topography of Visual Features in the Human Ventral Visual Pathway.

Visual object recognition in humans and nonhuman primates is achieved by the ventral visual pathway (ventral occipital-temporal cortex, VOTC), which shows a well-documented object domain structure. An on-going question is what type of information is processed in the higher-order VOTC that underlies such observations, with recent evidence suggesting effects of certain visual features. Combining computational vision models, fMRI experiment using a parametric-modulation approach, and natural image statistics of common objects, we depicted the neural distribution of a comprehensive set of visual features in the VOTC, identifying voxel sensitivities with specific feature sets across geometry/shape, Fourier power, and color. The visual feature combination pattern in the VOTC is significantly explained by their relationships to different types of response-action computation (fight-or-flight, navigation, and manipulation), as derived from behavioral ratings and natural image statistics. These results offer a comprehensive visual feature map in the VOTC and a plausible theoretical explanation as a mapping onto different types of downstream response-action systems.

Combining convolutional neural networks and cognitive models to predict novel object recognition in humans.

Object representations from convolutional neural network (CNN) models of computer vision (LeCun, Bengio, & Hinton, 2015) were used to drive a cognitive model of decision making, the linear ballistic accumulator (LBA) model (Brown & Heathcote, 2008), to predict errors and response times (RTs) in a novel object recognition task in humans. CNNs have become very successful at visual tasks like classifying objects in real-world images (e.g., He, Zhang, Ren, & Sun, 2015; Krizhevsky, Sutskever, & Hinton, 2012). We asked whether object representations learned by CNNs previously trained on a large corpus of natural images could be used to predict performance recognizing novel objects the network has never been trained on; we used novel Greebles, Ziggerins, and Sheinbugs that have been used in a number of previous object recognition studies. We specifically investigated whether a model combining high-level CNN representations of these novel objects could be used to drive an LBA model of decision making to account for errors and RTs in a same-different matching task (from Richler et al., 2019). Combining linearly transformed CNN object representations with the LBA provided reasonable accounts of performance not only on average, but at the individual-participant level and the item level as well. We frame the findings in the context of growing interest in using CNN models to understand visual object representations and the promise of using CNN representations to extend cognitive models to explain more complex aspects of human behavior. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Machine vision benefits from human contextual expectations

Scene context is known to facilitate object recognition in both machines and humans, suggesting that the underlying representations may be similar. Alternatively, they may be qualitatively different since the training experience of machines and humans are strikingly different. Machines are typically trained on images containing objects and their context, whereas humans frequently experience scenes without objects (such as highways without cars). If these context representations are indeed different, machine vision algorithms will be improved on augmenting them with human context representations, provided these expectations can be measured and are systematic. Here, we developed a paradigm to measure human contextual expectations. We asked human subjects to indicate the scale, location and likelihood at which cars or people might occur in scenes without these objects. This yielded highly systematic expectations that we could then accurately predict using scene features. This allowed us to predict human expectations on novel scenes without requiring explicit measurements. Next we augmented decisions made by deep neural networks with these predicted human expectations and obtained substantial gains in accuracy for detecting cars and people (1–3%) as well as on detecting associated objects (3–20%). In contrast, augmenting deep network decisions with other conventional computer vision features yielded far smaller gains. Taken together, our results show that augmenting deep neural networks with human-derived contextual expectations improves their performance, suggesting that contextual representations are qualitatively different in humans and deep neural networks.

Invariant behavioural templates for object recognition in humans and rats

Invariant behavioural templates for object recognition in humans and rats

Tactile shape discrimination recruits human lateral occipital complex during early perceptual processing

Neuroimaging studies investigating somatosensory-based object recognition in humans have revealed activity in the lateral occipital complex, a cluster of regions primarily associated with visual object recognition. To date, determining whether this activity occurs during or subsequent to recognition per se, has been difficult to assess due to the low temporal resolution of the hemodynamic response. To more finely measure the timing of somatosensory object recognition processes we employed high density EEG using a modified version of a paradigm previously applied to neuroimaging experiments. Simple geometric shapes were presented to the right index finger of 10 participants while the ongoing EEG was measured time locked to the stimulus. In the condition of primary interest participants discriminated the shape of the stimulus. In the alternate condition they judged stimulus duration. Using traditional event-related potential analysis techniques we found significantly greater amplitudes in the evoked potentials of the shape discrimination condition between 140 and 160 ms, a timeframe in which LOC mediated perceptual processes are believed to occur during visual object recognition. Scalp voltage topography and source analysis procedures indicated the lateral occipital complex as the likely source behind this effect. This finding supports a multisensory role for the lateral occipital complex during object recognition.

Stimulus timing dependent plasticity in high- and low-level vision

Event Abstract Back to Event Stimulus timing dependent plasticity in high- and low-level vision David B. McMahon1* and David A. Leopold1 1 National Institute of Mental Health, United States Object recognition in humans is extremely plastic, as evidenced by our ability to recognize and remember new faces after a few seconds of exposure. This form of visual learning is presumably mediated by experience dependent changes in synaptic efficacy among neurons in ventral visual cortical areas that are involved in face recognition. In contrast, perceptual acuity for low-level visual features tends to be more stable, and the small changes that can occur do so only after extensive training over long periods of time. At the cellular level, synaptic strength is sensitive to the precise relative timing of pre- and post-synaptic events. This phenomenon, known as spike timing dependent plasticity, has been linked to changes in the stimulus selectivity of V1 neurons measured physiologically, and to shifts in perceived line orientation in human subjects measured psychophysically (Yao and Dan 2001). In the current study, we tested the idea that visual responses in high-level visual cortex, and thus object recognition, can be modified by stimulus timing dependent plasticity. We hypothesized that the plasticity induced by stimulus timing manipulations would be greater in a perceptual task that depends on high-level visual cortex than for a task that can be solved based on early visual processing. To test this idea, we measured psychometric functions of subjects performing two perceptual tasks that required either a high-level (face identity) or a low-level (line orientation) judgement. We first determined the stimulus value that corresponded to the point of perceptual equivalence for both stimulus types. We then induced a shift in this perceptual midpoint using a conditioning protocol designed to induce spike timing dependent plasticity in the connections between neurons in visual cortex. In experiments measuring face identity perception, the subjects were exposed to a block of conditioning stimulation that consisted of 100 pairings of two brief (10 ms) presentations of faces A and B. In experiments measuring perceived line orientation, an identical pairing protocol was employed using gratings with slightly different orientations. In both cases, pairings presented with a brief (20 ms) stimulus onset asynchrony shifted the perceptual midpoint to the left or right depending on the order of the stimulus pairing (AB or BA). The magnitude of this perceptual shift decreased substantially or was abolished altogether when the stimulus onset asynchrony was increased beyond 60 ms. When expressed as a proportion of the slope of the psychometric function for each stimulus type, a significantly greater shift was induced for perception of face identity than for line orientation. This result supports the idea that high-level visual areas are particularly susceptible to stimulus timing dependent plasticity. Conference: Computational and Systems Neuroscience 2010, Salt Lake City, UT, United States, 25 Feb - 2 Mar, 2010. Presentation Type: Poster Presentation Topic: Poster session II Citation: McMahon DB and Leopold DA (2010). Stimulus timing dependent plasticity in high- and low-level vision. Front. Neurosci. Conference Abstract: Computational and Systems Neuroscience 2010. doi: 10.3389/conf.fnins.2010.03.00327 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 08 Mar 2010; Published Online: 08 Mar 2010. * Correspondence: David B McMahon, National Institute of Mental Health, Bethesda, United States, mcmahond@mail.nih.gov Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers David B McMahon David A Leopold Google David B McMahon David A Leopold Google Scholar David B McMahon David A Leopold PubMed David B McMahon David A Leopold Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.

Development of object recognition in humans

Although the ability to perceive simple shapes emerges in infancy, the ability to recognize individual objects as well as adults do continues to develop through childhood into adolescence. Despite this slow development, recent neuroimaging studies have revealed that an area of the ventral visual cortex that responds selectively to the category of common objects is adult-like by 5-8 years of age. The challenge for future research will be to identify the specific visual skills involved in object recognition that continue to develop through childhood and adolescence, and the neural mechanisms underlying this protracted development.

The effect of senescence on orientation discrimination and mechanism tuning

Accurately processing orientation information is a fundamental component of visual performance. Single-unit recordings have shown that the orientation tuning of individual neurons in macaque cortical areas V1 and V2 is reduced dramatically with age (M. T. Schmolesky, Y. Wang, M. Pu, & A. G. Leventhal, 2000; S. Yu, Y. Wang, X. Li, Y. Zhou, & A. G. Leventhal, 2006). These researchers suggest that losses in single-unit orientation selectivity result in declines in orientation discrimination and object recognition in older humans. Three experiments were conducted to determine whether human performance is affected by putative age-related changes in tuning of cortical neurons. Ten younger and ten older observers participated in this study. Experiment 1 demonstrated significant differences in the contrast sensitivity of the two age groups. Experiment 2 showed significant differences in orientation discrimination thresholds. However, when thresholds were plotted in terms of multiples of detection threshold, age-related differences were not observed. In Experiment 3, perceptual orientation tuning curves did not significantly differ in shape for younger and older subjects. As in Experiment 2, at any given contrast, there is a large difference in sensitivity between younger and older adults. This implies a model of orientation processing that allows the adult visual system to maintain consistent and reliable orientation information at the network and ultimately the perceptual level.

Effects of sex on object recognition and spatial navigation in humans

Human tests designed to mirror rodent tests of object recognition and spatial navigation were administered to adult cognitively healthy humans. Facial recognition was also assessed. There was no sex difference in facial recognition, consistent with earlier studies. In the object recognition test, the test–retest NINL total scores during the same visit were highly correlated, comparable to the test–retest correlations obtained in the established facial recognition test. There were no effects of sex on object recognition. However, in the spatial navigation test, there were effects of sex on spatial learning and memory during the session with the hidden, but not visible, target. These tests might be useful to compare assessments of object recognition and spatial learning and memory in humans and animal models.

Dynamic object recognition in pigeons and humans

We investigated the role of dynamic information in human and pigeon object recognition. Both species were trained to discriminate between two objects that each had a characteristic motion, so that either cue could be used to perform the task successfully. The objects were either easy or difficult to decompose into parts. At test, the learned objects could appear in their learned motions, the reverse of the learned motions, or an entirely new motion, or a new object could appear in one of the learned motions. For humans, any change in the learned motion produced a decrement in performance for both the decomposable and the nondecomposable objects, but participants did not respond differentially to new objects that appeared in the learned motions. Pigeons showed the same pattern of responding as did humans for the decomposable objects, except that pigeons responded differentially to new objects in the learned motions. For the nondecomposable objects, pigeons used motion cues exclusively. We suggest that for some types of objects, dynamic information may be weighted differently by pigeons and humans.

Comparative Cognition of Object Recognition

Comparative Cognition of Object Recognition by Marcia L. Spetch, University of Alberta Alinda Friedman, University of Alberta Reading Options: Download/Read PDF | Add to Endnote Abstract Object recognition is fundamental in the lives of most animals. The authors review research comparing object recognition in pigeons and humans.

Functional imaging of human crossmodal identification and object recognition

The perception of objects is a cognitive function of prime importance. In everyday life, object perception benefits from the coordinated interplay of vision, audition, and touch. The different sensory modalities provide both complementary and redundant information about objects, which may improve recognition speed and accuracy in many circumstances. We review crossmodal studies of object recognition in humans that mainly employed functional magnetic resonance imaging (fMRI). These studies show that visual, tactile, and auditory information about objects can activate cortical association areas that were once believed to be modality-specific. Processing converges either in multisensory zones or via direct crossmodal interaction of modality-specific cortices without relay through multisensory regions. We integrate these findings with existing theories about semantic processing and propose a general mechanism for crossmodal object recognition: The recruitment and location of multisensory convergence zones varies depending on the information content and the dominant modality.

SHORT COMMUNICATIONS: Semantic memory retrieval: cortical couplings in object recognition in the N400 window

To characterize the regional changes in neuronal couplings and information transfer related to semantic aspects of object recognition in humans we used partial-directed EEG-coherence analysis (PDC). We examined the differences of processing recognizable and unrecognizable pictures as reflected by changes in cortical networks within the time-window of a determined event-related potential (ERP) component, namely the N400. Fourteen participants performed an image recognition task, while sequentially confronted with pictures of recognizable and unrecognizable objects. The time-window of N400 as indicative of object semantics was defined from the ERP. Differences of PDC in the beta-band between these tasks were represented topographically as patterns of electrical couplings, possibly indicating changing degrees of functional cooperation between brain areas. Successful memory retrieval of picture meaning appears to be supported by networks comprising left temporal and parietal regions and bilateral frontal brain areas.

Multisensory visual-auditory object recognition in humans: a high-density electrical mapping study.

Multisensory object-recognition processes were investigated by examining the combined influence of visual and auditory inputs upon object identification--in this case, pictures and vocalizations of animals. Behaviorally, subjects were significantly faster and more accurate at identifying targets when the picture and vocalization were matched (i.e. from the same animal), than when the target was represented in only one sensory modality. This behavioral enhancement was accompanied by a modulation of the evoked potential in the latency range and general topographic region of the visual evoked N1 component, which is associated with early feature processing in the ventral visual stream. High-density topographic mapping and dipole modeling of this multisensory effect were consistent with generators in lateral occipito-temporal cortices, suggesting that auditory inputs were modulating processing in regions of the lateral occipital cortices. Both the timing and scalp topography of this modulation suggests that there are multisensory effects during what is considered to be a relatively early stage of visual object-recognition processes, and that this modulation occurs in regions of the visual system that have traditionally been held to be unisensory processing areas. Multisensory inputs also modulated the visual 'selection-negativity', an attention dependent component of the evoked potential this is usually evoked when subjects selectively attend to a particular feature of a visual stimulus.

Distance and shape: perception of the 3-dimensional world by weakly electric fish

Weakly electric fish orient at night in complete darkness by employing their active electrolocation system. They emit short electric signals and perceive the consequences of these emissions with epidermal electroreceptors. Objects are detected by analyzing the electric images which they project onto the animal's electroreceptive skin surface. This process corresponds to similar processes during vision, where visual images are cast onto the retinas of eyes. Behavioral experiments have shown that electric fish can measure the distance of objects during active electrolocation, thus possessing three-dimensional depth perception of their surroundings. The fundamental mechanism for distance determination differs from stereopsis used during vision by two-eyed animals, but resembles some supplementary mechanisms for distance deduction in humans. Weakly electric fish can also perceive the three-dimensional shape of objects. The fish can learn to identify certain objects and discriminate them from all other objects. In addition, they spontaneously categorize objects according to their shapes and not according to object size or material properties. There is good evidence that some fundamental types of perceptional invariances during visual object recognition in humans are also found in electric fish during active electrolocation. These include size invariance (maybe including size constancy), rotational invariance, and translational invariance. The mechanisms of shape detection during electrolocation are still unknown, and their discoveries require additional experiments.

Auditory-visual integration during multimodal object recognition in humans: a behavioral and electrophysiological study.

The aim of this study was (1) to provide behavioral evidence for multimodal feature integration in an object recognition task in humans and (2) to characterize the processing stages and the neural structures where multisensory interactions take place. Event-related potentials (ERPs) were recorded from 30 scalp electrodes while subjects performed a forced-choice reaction-time categorization task: At each trial, the subjects had to indicate which of two objects was presented by pressing one of two keys. The two objects were defined by auditory features alone, visual features alone, or the combination of auditory and visual features. Subjects were more accurate and rapid at identifying multimodal than unimodal objects. Spatiotemporal analysis of ERPs and scalp current densities revealed several auditory-visual interaction components temporally, spatially, and functionally distinct before 200 msec poststimulus. The effects observed were (1) in visual areas, new neural activities (as early as 40 msec poststimulus) and modulation (amplitude decrease) of the N185 wave to unimodal visual stimulus, (2) in the auditory cortex, modulation (amplitude increase) of subcomponents of the unimodal auditory N1 wave around 90 to 110 msec, and (3) new neural activity over the right fronto-temporal area (140 to 165 msec). Furthermore, when the subjects were separated into two groups according to their dominant modality to perform the task in unimodal conditions (shortest reaction time criteria), the integration effects were found to be similar for the two groups over the nonspecific fronto-temporal areas, but they clearly differed in the sensory-specific cortices, affecting predominantly the sensory areas of the nondominant modality. Taken together, the results indicate that multisensory integration is mediated by flexible, highly adaptive physiological processes that can take place very early in the sensory processing chain and operate in both sensory-specific and nonspecific cortical structures in different ways.

Imaging visual recognition: PET and fMRI studies of the functional anatomy of human visual recognition.

Until recently, the neural bases of visual object recognition in humans could be studied only by the use of brain-damaged subjects with naturally occurring lesions. Functional neuroimaging has given us the capability of studying visual recognition in the normal human brain. In the past ten years a number of PET and fMRI studies have attempted to isolate the neural substrates of human visual recognition. We have reviewed these studies and compared their conclusions regarding the anatomical locations of visual recognition processing in the human brain. The outcome was disappointing, revealing a wide range of locations. Our attempts to reduce the scatter by subgrouping the studies according to different task and stimulus properties were not successful. We discuss possible reasons for the lack of agreement among studies, including differences in the kinds of information yielded by lesion and imaging studies, and issues in the design and analysis of functional neuroimaging experiments. We conclude with a review of a more recent approach to the neuroimaging of human visual recognition, in which the effects of recognizing different types of visual stimuli are compared directly. With these experimental designs neuroimaging yields more replicable results, which also accord better with the known effects of lesions.

Functional anatomy of object recognition in humans: evidence from positron emission tomography and functional magnetic resonance imaging.

The ability to recognize elements of the environment as familiar objects, events, or classes underlies virtually all human cognition. Positron emission tomography and functional magnetic resonance imaging studies of the recognition of visually presented objects suggest that several brain areas crucially support this function. However, a review of these findings reveals great variability in the cortical localization of object recognition in humans. Although there is converging evidence for the importance of the temporal and occipitotemporal cortex in object recognition, a consistent picture of the functional organization within these areas and their relation to function in other cortical regions has not yet emerged from brain imaging studies.

Visual pathways for object-oriented action and object recognition: functional anatomy with PET.

The purpose of this study was to identify the functional anatomy of the mechanisms involved in visually guided prehension and in object recognition in humans. The cerebral blood flow of seven subjects was investigated by positron emission tomography. Three conditions were performed using the same set of stimuli. In the 'grasping' condition, subjects were instructed to accurately grasp the objects. In the 'matching' condition, subjects were requested to compare the shape of the presented object with that of the previous one. In the 'pointing' condition (control), subjects pointed towards the objects. The comparison between grasping and pointing showed a regional cerebral blood flow (rCBF) increase in the anterior part of the inferior parietal cortex and part of the posterior parietal cortex. The comparison between grasping and matching showed an rCBF increase in the cerebellum, the left frontal cortex around the central sulcus, the mesial frontal cortex and the left inferior parietal cortex. Finally, the comparison between matching and pointing showed an rCBF increase in the right temporal cortex and the right posterior parietal cortex. Thus object-oriented action and object recognition activate a common posterior parietal area, suggesting that some kind of within-object spatial analysis was processed by this area whatever the goal of the task.