View-invariant Object Recognition Research Articles

The Development of Object Recognition Requires Experience with the Surface Features of Objects.

What role does visual experience play in the development of object recognition? Prior controlled-rearing studies suggest that newborn animals require slow and smooth visual experiences to develop object recognition. Here, we examined whether the development of object recognition also requires experience with the surface features of objects. We raised newborn chicks in automated controlled-rearing chambers that contained a single virtual object, then tested their ability to recognize that object from familiar and novel viewpoints. When chicks were reared with an object that had surface features, the chicks developed view-invariant object recognition. In contrast, when chicks were reared with a line drawing of an object, the chicks failed to develop object recognition. The chicks reared with line drawings performed at chance level, despite acquiring over 100 h of visual experience with the object. These results indicate that the development of object recognition requires experience with the surface features of objects.

Object discrimination performance and dynamics evaluated by inferotemporal cell population activity.

We have previously reported an increase in response tolerance of inferotemporal cells around trained views. However, an inferotemporal cell usually displays different response patterns in an initial response phase immediately after the stimulus onset and in a late phase from approximately 260 ms after stimulus onset. This study aimed to understand the difference between the two time periods and their involvement in the view-invariant object recognition. Responses to object images with and without prior experience of object discrimination across views, recorded by microelectrodes, were pooled together from our previous experiments. With a machine learning algorithm, we trained to build classifiers for object discrimination. In the early phase, the performance of classifiers created based on data of responses to the object images with prior training of object discrimination across views did not significantly differ from that based on data of responses to the object images without prior experience of object discrimination across views. However, the performance was significantly better in the late phase. Furthermore, compared to the preferred stimulus image in the early phase, we found 2/3 of cells changed their preference in the late phase. For object images with prior experience of training with object discrimination across views, a significant higher percentage of cells responded in the late phase to the same objects as in the early phase, but under different views. The results demonstrate the dynamics of selectivity changes and suggest the involvement of the late phase in the view-invariant object recognition rather than that of the early phase.

An intrinsic algorithm for viewing angle tolerance of object discrimination in human subjects.

Prevailing theories suggest that view-invariant object recognition is accomplished via spatiotemporal correlations of multiple views that bind different views to the same object. However, it is unknown how the exposure to multiple views without association affects view-invariant recognition generating. Behavioural studies have shown that monkeys acquired view-invariant object recognition capability in a range of the viewing angles of 30° to 60° after experiencing discrimination of similar objects at each of several viewing angles without associating different views, but the monkeys could not discriminate novel objects from similar distractors when the viewing angle changed. In the present study the development of the view invariance was tested in the human subjects and compared with the results of the monkeys previously reported. The view-invariant object recognition capability of human subjects was tested using either familiar objects that the subjects experienced in a preparatory object discrimination task at the same viewpoints or novel objects that the subjects had never experienced. In the viewing angle range of 30°, human subjects showed significant object discrimination capability across views, with no need for prior experience of the objects. Prior discrimination experience within the same viewpoints endowed the human subjects with broadening of the viewing angle tuning, because an object discrimination test immediately after the prior discrimination experience at a single view showed wider viewing angle tuning than a test without prior experience. (PsycINFO Database Record (c) 2020 APA, all rights reserved).

Long-term Object Discrimination at Several Viewpoints Develops Neural Substrates of View-invariant Object Recognition in Inferotemporal Cortex

Object recognition requires differentiation across different objects and generalization across views of the same object. We previously demonstrated that discrimination of object images at several views without any possibility of association was enough to achieve object recognition within a certain range of viewing angles and confirmed the response tolerance of monkey inferotemporal cells within a similar range of viewing angles. However, neither behavioral object recognition nor electrophysiological response tolerance was complete across views. In the present study, we extended such learning past performance saturation and recorded neuronal activity during the further learning period. When monkeys were trained to discriminate objects at several views, we found that they could discriminate the trained objects regardless of the eventual change in viewing angle, and confirmed a response tolerance at the population level over a large viewing angle range covering all the viewpoints experienced. At the cell population level, such overtraining leads to significantly higher neural response similarity for views of the same objects than for views of different objects regardless of the extent of viewing angle separation. These results suggest a possible method of view-invariant object recognition development.

Non-accidental properties, metric invariance, and encoding by neurons in a model of ventral stream visual object recognition, VisNet.

When objects transform into different views, some properties are maintained, such as whether the edges are convex or concave, and these non-accidental properties are likely to be important in view-invariant object recognition. The metric properties, such as the degree of curvature, may change with different views, and are less likely to be useful in object recognition. It is shown that in a model of invariant visual object recognition in the ventral visual stream, VisNet, non-accidental properties are encoded much more than metric properties by neurons. Moreover, it is shown how with the temporal trace rule training in VisNet, non-accidental properties of objects become encoded by neurons, and how metric properties are treated invariantly. We also show how VisNet can generalize between different objects if they have the same non-accidental property, because the metric properties are likely to overlap. VisNet is a 4-layer unsupervised model of visual object recognition trained by competitive learning that utilizes a temporal trace learning rule to implement the learning of invariance using views that occur close together in time. A second crucial property of this model of object recognition is, when neurons in the level corresponding to the inferior temporal visual cortex respond selectively to objects, whether neurons in the intermediate layers can respond to combinations of features that may be parts of two or more objects. In an investigation using the four sides of a square presented in every possible combination, it was shown that even though different layer 4 neurons are tuned to encode each feature or feature combination orthogonally, neurons in the intermediate layers can respond to features or feature combinations present is several objects. This property is an important part of the way in which high capacity can be achieved in the four-layer ventral visual cortical pathway. These findings concerning non-accidental properties and the use of neurons in intermediate layers of the hierarchy help to emphasise fundamental underlying principles of the computations that may be implemented in the ventral cortical visual stream used in object recognition.

The Development of Invariant Object Recognition Requires Visual Experience With Temporally Smooth Objects.

How do newborns learn to recognize objects? According to temporal learning models in computational neuroscience, the brain constructs object representations by extracting smoothly changing features from the environment. To date, however, it is unknown whether newborns depend on smoothly changing features to build invariant object representations. Here, we used an automated controlled-rearing method to examine whether visual experience with smoothly changing features facilitates the development of view-invariant object recognition in a newborn animal model-the domestic chick (Gallus gallus). When newborn chicks were reared with a virtual object that moved smoothly over time, the chicks created view-invariant representations that were selective for object identity and tolerant to viewpoint changes. Conversely, when newborn chicks were reared with a temporally non-smooth object, the chicks developed less selectivity for identity features and less tolerance to viewpoint changes. These results provide evidence for a "smoothness constraint" on the development of invariant object recognition and indicate that newborns leverage the temporal smoothness of natural visual environments to build abstract mental models of objects.

Development of novel tasks for studying view-invariant object recognition in rodents: Sensitivity to scopolamine

The capacity to recognize objects from different view-points or angles, referred to as view-invariance, is an essential process that humans engage in daily. Currently, the ability to investigate the neurobiological underpinnings of this phenomenon is limited, as few ethologically valid view-invariant object recognition tasks exist for rodents. Here, we report two complementary, novel view-invariant object recognition tasks in which rodents physically interact with three-dimensional objects. Prior to experimentation, rats and mice were given extensive experience with a set of ‘pre-exposure’ objects. In a variant of the spontaneous object recognition task, novelty preference for pre-exposed or new objects was assessed at various angles of rotation (45°, 90° or 180°); unlike control rodents, for whom the objects were novel, rats and mice tested with pre-exposed objects did not discriminate between rotated and un-rotated objects in the choice phase, indicating substantial view-invariant object recognition. Secondly, using automated operant touchscreen chambers, rats were tested on pre-exposed or novel objects in a pairwise discrimination task, where the rewarded stimulus (S+) was rotated (180°) once rats had reached acquisition criterion; rats tested with pre-exposed objects re-acquired the pairwise discrimination following S+ rotation more effectively than those tested with new objects. Systemic scopolamine impaired performance on both tasks, suggesting involvement of acetylcholine at muscarinic receptors in view-invariant object processing. These tasks present novel means of studying the behavioral and neural bases of view-invariant object recognition in rodents.

Deep Networks Can Resemble Human Feed-forward Vision in Invariant Object Recognition.

Deep convolutional neural networks (DCNNs) have attracted much attention recently, and have shown to be able to recognize thousands of object categories in natural image databases. Their architecture is somewhat similar to that of the human visual system: both use restricted receptive fields, and a hierarchy of layers which progressively extract more and more abstracted features. Yet it is unknown whether DCNNs match human performance at the task of view-invariant object recognition, whether they make similar errors and use similar representations for this task, and whether the answers depend on the magnitude of the viewpoint variations. To investigate these issues, we benchmarked eight state-of-the-art DCNNs, the HMAX model, and a baseline shallow model and compared their results to those of humans with backward masking. Unlike in all previous DCNN studies, we carefully controlled the magnitude of the viewpoint variations to demonstrate that shallow nets can outperform deep nets and humans when variations are weak. When facing larger variations, however, more layers were needed to match human performance and error distributions, and to have representations that are consistent with human behavior. A very deep net with 18 layers even outperformed humans at the highest variation level, using the most human-like representations.

Humans and Deep Networks Largely Agree on Which Kinds of Variation Make Object Recognition Harder.

View-invariant object recognition is a challenging problem that has attracted much attention among the psychology, neuroscience, and computer vision communities. Humans are notoriously good at it, even if some variations are presumably more difficult to handle than others (e.g., 3D rotations). Humans are thought to solve the problem through hierarchical processing along the ventral stream, which progressively extracts more and more invariant visual features. This feed-forward architecture has inspired a new generation of bio-inspired computer vision systems called deep convolutional neural networks (DCNN), which are currently the best models for object recognition in natural images. Here, for the first time, we systematically compared human feed-forward vision and DCNNs at view-invariant object recognition task using the same set of images and controlling the kinds of transformation (position, scale, rotation in plane, and rotation in depth) as well as their magnitude, which we call “variation level.” We used four object categories: car, ship, motorcycle, and animal. In total, 89 human subjects participated in 10 experiments in which they had to discriminate between two or four categories after rapid presentation with backward masking. We also tested two recent DCNNs (proposed respectively by Hinton's group and Zisserman's group) on the same tasks. We found that humans and DCNNs largely agreed on the relative difficulties of each kind of variation: rotation in depth is by far the hardest transformation to handle, followed by scale, then rotation in plane, and finally position (much easier). This suggests that DCNNs would be reasonable models of human feed-forward vision. In addition, our results show that the variation levels in rotation in depth and scale strongly modulate both humans' and DCNNs' recognition performances. We thus argue that these variations should be controlled in the image datasets used in vision research.

Neural substrates of view-invariant object recognition developed without experiencing rotations of the objects.

One fails to recognize an unfamiliar object across changes in viewing angle when it must be discriminated from similar distractor objects. View-invariant recognition gradually develops as the viewer repeatedly sees the objects in rotation. It is assumed that different views of each object are associated with one another while their successive appearance is experienced in rotation. However, natural experience of objects also contains ample opportunities to discriminate among objects at each of the multiple viewing angles. Our previous behavioral experiments showed that after experiencing a new set of object stimuli during a task that required only discrimination at each of four viewing angles at 30° intervals, monkeys could recognize the objects across changes in viewing angle up to 60°. By recording activities of neurons from the inferotemporal cortex after various types of preparatory experience, we here found a possible neural substrate for the monkeys' performance. For object sets that the monkeys had experienced during the task that required only discrimination at each of four viewing angles, many inferotemporal neurons showed object selectivity covering multiple views. The degree of view generalization found for these object sets was similar to that found for stimulus sets with which the monkeys had been trained to conduct view-invariant recognition. These results suggest that the experience of discriminating new objects in each of several viewing angles develops the partially view-generalized object selectivity distributed over many neurons in the inferotemporal cortex, which in turn bases the monkeys' emergent capability to discriminate the objects across changes in viewing angle.

Development of view-invariant object recognition capability without view-invariance learning of the objects: Neuronal substrates

Development of view-invariant object recognition capability without view-invariance learning of the objects: Neuronal substrates

How do children learn to follow gaze, share joint attention, imitate their teachers, and use tools during social interactions?

How does an infant learn through visual experience to imitate actions of adult teachers, despite the fact that the infant and adult view one another and the world from different perspectives? To accomplish this, an infant needs to learn how to share joint attention with adult teachers and to follow their gaze towards valued goal objects. The infant also needs to be capable of view-invariant object learning and recognition whereby it can carry out goal-directed behaviors, such as the use of tools, using different object views than the ones that its teachers use. Such capabilities are often attributed to "mirror neurons". This attribution does not, however, explain the brain processes whereby these competences arise. This article describes the CRIB (Circular Reactions for Imitative Behavior) neural model of how the brain achieves these goals through inter-personal circular reactions. Inter-personal circular reactions generalize the intra-personal circular reactions of Piaget, which clarify how infants learn from their own babbled arm movements and reactive eye movements how to carry out volitional reaches, with or without tools, towards valued goal objects. The article proposes how intra-personal circular reactions create a foundation for inter-personal circular reactions when infants and other learners interact with external teachers in space. Both types of circular reactions involve learned coordinate transformations between body-centered arm movement commands and retinotopic visual feedback, and coordination of processes within and between the What and Where cortical processing streams. Specific breakdowns of model processes generate formal symptoms similar to clinical symptoms of autism.

View‐invariant object recognition ability develops after discrimination, not mere exposure, at several viewing angles

One usually fails to recognize an unfamiliar object across changes in viewing angle when it has to be discriminated from similar distractor objects. Previous work has demonstrated that after long-term experience in discriminating among a set of objects seen from the same viewing angle, immediate recognition of the objects across 30-60 degrees changes in viewing angle becomes possible. The capability for view-invariant object recognition should develop during the within-viewing-angle discrimination, which includes two kinds of experience: seeing individual views and discriminating among the objects. The aim of the present study was to determine the relative contribution of each factor to the development of view-invariant object recognition capability. Monkeys were first extensively trained in a task that required view-invariant object recognition (Object task) with several sets of objects. The animals were then exposed to a new set of objects over 26 days in one of two preparatory tasks: one in which each object view was seen individually, and a second that required discrimination among the objects at each of four viewing angles. After the preparatory period, we measured the monkeys' ability to recognize the objects across changes in viewing angle, by introducing the object set to the Object task. Results indicated significant view-invariant recognition after the second but not first preparatory task. These results suggest that discrimination of objects from distractors at each of several viewing angles is required for the development of view-invariant recognition of the objects when the distractors are similar to the objects.

A structured model of video reproduces primary visual cortical organisation.

The visual system must learn to infer the presence of objects and features in the world from the images it encounters, and as such it must, either implicitly or explicitly, model the way these elements interact to create the image. Do the response properties of cells in the mammalian visual system reflect this constraint? To address this question, we constructed a probabilistic model in which the identity and attributes of simple visual elements were represented explicitly and learnt the parameters of this model from unparsed, natural video sequences. After learning, the behaviour and grouping of variables in the probabilistic model corresponded closely to functional and anatomical properties of simple and complex cells in the primary visual cortex (V1). In particular, feature identity variables were activated in a way that resembled the activity of complex cells, while feature attribute variables responded much like simple cells. Furthermore, the grouping of the attributes within the model closely parallelled the reported anatomical grouping of simple cells in cat V1. Thus, this generative model makes explicit an interpretation of complex and simple cells as elements in the segmentation of a visual scene into basic independent features, along with a parametrisation of their moment-by-moment appearances. We speculate that such a segmentation may form the initial stage of a hierarchical system that progressively separates the identity and appearance of more articulated visual elements, culminating in view-invariant object recognition.

View-invariant object category learning, recognition, and search: How spatial and object attention are coordinated using surface-based attentional shrouds

How does the brain learn to recognize an object from multiple viewpoints while scanning a scene with eye movements? How does the brain avoid the problem of erroneously classifying parts of different objects together? How are attention and eye movements intelligently coordinated to facilitate object learning? A neural model provides a unified mechanistic explanation of how spatial and object attention work together to search a scene and learn what is in it. The ARTSCAN model predicts how an object’s surface representation generates a form-fitting distribution of spatial attention, or “attentional shroud”. All surface representations dynamically compete for spatial attention to form a shroud. The winning shroud persists during active scanning of the object. The shroud maintains sustained activity of an emerging view-invariant category representation while multiple view-specific category representations are learned and are linked through associative learning to the view-invariant object category. The shroud also helps to restrict scanning eye movements to salient features on the attended object. Object attention plays a role in controlling and stabilizing the learning of view-specific object categories. Spatial attention hereby coordinates the deployment of object attention during object category learning. Shroud collapse releases a reset signal that inhibits the active view-invariant category in the What cortical processing stream. Then a new shroud, corresponding to a different object, forms in the Where cortical processing stream, and search using attention shifts and eye movements continues to learn new objects throughout a scene. The model mechanistically clarifies basic properties of attention shifts (engage, move, disengage) and inhibition of return. It simulates human reaction time data about object-based spatial attention shifts, and learns with 98.1% accuracy and a compression of 430 on a letter database whose letters vary in size, position, and orientation. The model provides a powerful framework for unifying many data about spatial and object attention, and their interactions during perception, cognition, and action.

A developmental dissociation of view-dependent and view-invariant object recognition in adolescence

Spatial generalization skills in school children aged 8–16 were studied with regard to unfamiliar objects that had been previously learned in a cross-modal priming and learning paradigm. We observed a developmental dissociation with younger children recognizing objects only from previously learnt perspectives whereas older children generalized acquired object knowledge to new viewpoints as well. Haptic and – to a lesser extent – visual priming improved spatial generalization in all but the youngest children. The data supports the idea of dissociable, view-dependent and view-invariant object representations with different developmental trajectories that are subject to modulatory effects of priming. Late-developing areas in the parietal or the prefrontal cortex may account for the retarded onset of view-invariant object recognition.

Spatial vs temporal continuity in view invariant visual object recognition learning

We show in a 4-layer competitive neuronal network that continuous transformation learning, which uses spatial correlations and a purely associative (Hebbian) synaptic modification rule, can build view invariant representations of complex 3D objects. This occurs even when views of the different objects are interleaved, a condition where temporal trace learning fails. Human psychophysical experiments showed that view invariant object learning can occur when spatial but not temporal continuity applies because of interleaving of stimuli, although sequential presentation, which produces temporal continuity, can facilitate learning. Thus continuous transformation learning is an important principle that may contribute to view invariant object recognition.

Event-related potential component associated with the recognition of three-dimensional objects

To understand the physiological process underlying view-invariant object recognition, we designed a delayed matching-to-sample task under two different conditions: in the first condition, the sample and test stimuli were presented in the same view, and in the second, the two stimuli were presented in different views. The event-related potential (ERP) component, posterior N1, exhibited a significantly larger amplitude when the test image was in the same view as the sample image than when the test and the sample images were in different views. The result indicates that the posterior N1 is significantly sensitive to view association, and clarifies the difference between the neuronal activity that occurs in 3D object recognition and 2D image identification at the level of N1.

3D target recognition based on projective invariant relationships

In this paper, we propose a new 3D target recognition algorithm using a single image. Our approach is based on geometrically invariant relationships. By employing a practical CCD camera model for projective image formation, and analyzing the constraints on the projection parameters, we derive two invariant relationships using six pairs of 3D space features and corresponding image features, such as points and lines. Compared to the conventional approach in which only a single invariant relationship is derived from six point pairs based on the general finite projective camera model, the two relationships obtained from the practical camera model can increase the effectiveness of matching. Based on the derived invariant relationships, a new view-invariant object recognition algorithm using a single image is proposed. The performance of the proposed recognition algorithm is demonstrated by various computer simulations on synthetic and real images of objects. The experimental results show that 3D objects in the image can be robustly recognized, using the linear features, by the proposed algorithm.

Are cortical models really bound by the "binding problem"?

Are cortical models really bound by the "binding problem"?