Stages Of Object Recognition Research Articles

Traffic Object Detection and Recognition Based on the Attentional Visual Field of Drivers

Traffic object detection and recognition systems play an essential role in Advanced Driver Assistance Systems (ADAS) and Autonomous Vehicles (AV). In this research, we focus on four important classes of traffic objects: traffic signs, road vehicles, pedestrians, and traffic lights. We first review the major traditional machine learning and deep learning methods that have been used in the literature to detect and recognize these objects. We provide a vision-based framework that detects and recognizes traffic objects inside and outside the attentional visual area of drivers. This approach uses the driver 3D absolute coordinates of the gaze point obtained by the combined, cross-calibrated use of a front-view stereo imaging system and a non-contact 3D gaze tracker. A combination of multi-scale HOG-SVM and Faster R-CNN-based models are utilized in the detection stage. The recognition stage is performed with a ResNet-101 network to verify sets of generated hypotheses. We applied our approach on real data collected during drives in an urban environment with the RoadLAB instrumented vehicle. Our framework achieved 91% of correct object detections and provided promising results in the object recognition stage.

Context effects on object recognition in real-world environments: A study protocol.

Background: The environments that we live in impact on our ability to recognise objects, with recognition being facilitated when objects appear in expected locations (congruent) compared to unexpected locations (incongruent). However, these findings are based on experiments where the object is isolated from its environment. Moreover, it is not clear which components of the recognition process are impacted by the environment. In this experiment, we seek to examine the impact real world environments have on object recognition. Specifically, we will use mobile electroencephalography (mEEG) and augmented reality (AR) to investigate how the visual and semantic processing aspects of object recognition are changed by the environment. Methods: We will use AR to place congruent and incongruent virtual objects around indoor and outdoor environments. During the experiment a total of 34 participants will walk around the environments and find these objects while we record their eye movements and neural signals. We will perform two primary analyses. First, we will analyse the event-related potential (ERP) data using paired samples t-tests in the N300/400 time windows in an attempt to replicate congruency effects on the N300/400. Second, we will use representational similarity analysis (RSA) and computational models of vision and semantics to determine how visual and semantic processes are changed by congruency. Conclusions: Based on previous literature, we hypothesise that scene-object congruence would facilitate object recognition. For ERPs, we predict a congruency effect in the N300/N400, and for RSA we predict that higher level visual and semantic information will be represented earlier for congruent scenes than incongruent scenes. By collecting mEEG data while participants are exploring a real-world environment, we will be able to determine the impact of a natural context on object recognition, and the different processing stages of object recognition.

Context effects on object recognition in real-world environments: A study protocol

Background: The environments that we live in impact on our ability to recognise objects, with recognition being facilitated when objects appear in expected locations (congruent) compared to unexpected locations (incongruent). However, these findings are based on experiments where the object is isolated from its environment. Moreover, it is not clear which components of the recognition process are impacted by the environment. In this experiment, we seek to examine the impact real world environments have on object recognition. Specifically, we will use mobile electroencephalography (mEEG) and augmented reality (AR) to investigate how the visual and semantic processing aspects of object recognition are changed by the environment. Methods: We will use AR to place congruent and incongruent virtual objects around indoor and outdoor environments. During the experiment a total of 34 participants will walk around the environments and find these objects while we record their eye movements and neural signals. We will perform two primary analyses. First, we will analyse the event-related potential (ERP) data using paired samples t-tests in the N300/400 time windows in an attempt to replicate congruency effects on the N300/400. Second, we will use representational similarity analysis (RSA) and computational models of vision and semantics to determine how visual and semantic processes are changed by congruency. Conclusions: Based on previous literature, we hypothesise that scene-object congruence would facilitate object recognition. For ERPs, we predict a congruency effect in the N300/N400, and for RSA we predict that higher level visual and semantic information will be represented earlier for congruent scenes than incongruent scenes. By collecting mEEG data while participants are exploring a real-world environment, we will be able to determine the impact of a natural context on object recognition, and the different processing stages of object recognition.

Performance Evaluation of Convolutional Networks on Heterogeneous Architectures for Applications in Autonomous Robotics

Humanoid robots find application in human-robot interaction tasks. However, despite their capabilities, their sequential computing system limits the execution of computationally expensive algorithms such as convolutional neural networks, which have demonstrated good performance in recognition tasks. As an alternative to sequential computing units, Field-Programmable Gate Arrays and Graphics Processing Units have a high degree of parallelism and low power consumption. This study aims to improve the visual perception of a humanoid robot called NAO using these embedded systems running a convolutional neural network. The methodology adopted here is based on image acquisition and transmission using simulation software: Webots and Choreographe. In each embedded system, an object recognition stage is performed using commercial convolutional neural network acceleration frameworks. Xilinx® Ultra96™, Intel® Cyclone® V-SoC and NVIDIA® Jetson™ TX2 cards were used, and Tinier-YOLO, AlexNet, Inception-V1 and Inception V3 transfer-learning networks were executed. Real-time metrics were obtained when Inception V1, Inception V3 transfer-learning and AlexNet were run on the Ultra96 and Jetson TX2 cards, with frame rates between 28 and 30 frames per second. The results demonstrated that the use of these embedded systems and convolutional neural networks can provide humanoid robots such as NAO with greater visual recognition in tasks that require high accuracy and autonomy.

Intermediate, Wholistic Shape Representation in Object Recognition: A Pre-Attentive Stage of Processing?

Evidence is presented for intermediate, wholistic visual representations of objects and non-objects that are computed online and independent of visual attention. Short-term visual priming was examined between visually similar shapes, with targets either falling at the (valid) location cued by primes or at another (invalid) location. Object decision latencies were facilitated when the overall shapes of the stimuli were similar irrespective of whether the location of the prime was valid or invalid, with the effects being equally large for object and non-object targets. In addition, the effects were based on the overall outlines of the stimuli and low spatial frequency components, not on local parts. In conclusion, wholistic shape representations based on outline form, are rapidly computed online during object recognition. Moreover, activation of common wholistic shape representations prime the processing of subsequent objects and non-objects irrespective of whether they appear at attended or unattended locations. Rapid derivation of wholistic form provides a key intermediate stage of object recognition.

Bird expertise does not increase motion sensitivity to bird flight motion.

While motion information is important for the early stages of vision, it also contributes to later stages of object recognition. For example, human observers can detect the presence of a human, judge its actions, and judge its gender and identity simply based on motion cues conveyed in a point-light display. Here we examined whether object expertise enhances the observer's sensitivity to its characteristic movement. Bird experts and novices were shown point-light displays of upright and inverted birds in flight, or upright and inverted human walkers, and asked to discriminate them from spatially scrambled point-light displays of the same stimuli. While the spatially scrambled stimuli retained the local motion of each dot of the moving objects, it disrupted the global percept of the object in motion. To estimate a detection threshold in each object domain, we systematically varied the number of noise dots in which the stimuli were embedded using an adaptive staircase approach. Contrary to our predictions, the experts did not show disproportionately higher sensitivity to bird motion, and both groups showed no inversion cost. However, consistent with previous work showing a robust inversion effect for human motion, both groups were more sensitive to upright human walkers than their inverted counterparts. Thus, the result suggests that real-world experience in the bird domain has little to no influence on the sensitivity to bird motion and that birds do not show the typical inversion effect seen with humans and other terrestrial movement.

One-shot learning of view-invariant object representations in newborn chicks

Can newborn brains perform one-shot learning? To address this question, we reared newborn chicks in strictly controlled environments containing a single view of a single object, then tested their object recognition performance across 24 uniformly-spaced viewpoints. We found that chicks can build view-invariant object representations from a single view of an object: a case of one-shot learning in newborn brains. Chicks can also build the same view-invariant object representation from different views of an object, showing that newborn brains converge on common object representations from different sets of sensory inputs. Finally, by rearing chicks with larger numbers of object views, we found that chicks develop enhanced recognition for familiar views. These results illuminate the earliest stages of object recognition, revealing (1) powerful one-shot learning that builds invariant object representations from the first views of an object and (2) view-based learning that enriches object representations, producing enhanced recognition for familiar views.

Learning and surface boundary feedbacks for colour natural scene perception

Boundary detection and segmentation are essential stages in object recognition and scene understanding. In this paper, we present a bio-inspired neural model of the ventral pathway for colour contour and surface perception, called LPREEN (Learning and Perceptual boundaRy rEcurrent dEtection Neural architecture). LPREEN models colour opponent processes and feedback interactions between cortical areas V1, V2, V4, and IT, which produce top-down and bottom-up information fusion. We suggest three feedback interactions that enhance and complete boundaries. Our proposed neural model contains a contour learning feedback that enhances the most probable contour positions in V1 according to a previous experience, and generates a surface perception in V4 through diffusion processes. We compared the proposed model with another bio-inspired model and two well-known contour extraction methods, using the Berkeley Segmentation Benchmark. LPREEN showed better performance than two methods and slightly worse performance than another one.

Human Detection Framework for Automated Surveillance Systems

Vision-based systems for surveillance applications have been used widely and gained more research attention. Detecting people in an image stream is challenging because of their intra-class variability, the diversity of the backgrounds, and the conditions under which the images were acquired. Existing human detection solutions suffer in their effectiveness and efficiency. In particular, the accuracy of the existing detectors is characterized by their high false positive and negative. In addition, existing detectors are slow for online surveillance systems which lead to large delay that is not suitable for surveillance systems for real-time monitoring. In this paper, a holistic framework is proposed for enhancing the performance of human detection in surveillance system. In general, the framework includes the following stages: environment modeling, motion object detection, and human object recognition. In environment modeling, modal algorithm has been suggested for background initialization and extraction. Then for effectively classifying the motion object, edge detecting and B-spline algorithm have been used for shadow detection and removal. Then, enhanced Lucas–Kanade optical flow has been used to get the area of interest for object segmentation. Finally, to enhance the segmentation, some morphological processes were performed. In the motion object recognition stage, segmentation for each blob is performed and processed to the human detector which is a complete learning-based system for detecting and localizing objects/humans in images using mixtures of deformable part models (PFF detector). Results show enhancement in each phase of the proposed framework. These enhancements are shown in the overall performance of human detection in surveillance system.

RGB-D Hand-Held Object Recognition Based on Heterogeneous Feature Fusion

Object recognition has many applications in human-machine interaction and multimedia retrieval. However, due to large intra-class variability and inter-class similarity, accurate recognition relying only on RGB data is still a big challenge. Recently, with the emergence of inexpensive RGB-D devices, this challenge can be better addressed by leveraging additional depth information. A very special yet important case of object recognition is hand-held object recognition, as manipulating objects with hands is common and intuitive in human-human and human-machine interactions. In this paper, we study this problem and introduce an effective framework to address it. This framework first detects and segments the hand-held object by exploiting skeleton information combined with depth information. In the object recognition stage, this work exploits heterogeneous features extracted from different modalities and fuses them to improve the recognition accuracy. In particular, we incorporate handcrafted and deep learned features and study several multi-step fusion variants. Experimental evaluations validate the effectiveness of the proposed method.

A saw is first identified as an object used on wood: ERP evidence for temporal differences between Thematic and Functional similarity relations

The role of functional and motor information in manipulable artifact object semantic organization is still poorly understood. In particular, several types of semantic relations involving object functional knowledge may be distinguished. Functional similarity relations group objects with similar functions at relatively specific (e.g. saw–axe, both used to cut wood) or general (saw–knife, both used to cut) levels. Thematic relations group objects based on their complementarity in events (saw used upon/with wood). Recent eye-tracking data showed distinct temporal time courses for the different semantic relations, with fastest processing of thematic relations and slowest processing of general function similarity relations. Behavioral data suggest the involvement of distinct cognitive mechanisms in manipulable artifact object semantic processing. The aim of the present study was to assess the neural correlates of thematic, and specific and general function similarity relation processing. Specifically, we investigated whether time course differences between semantic relations could be highlighted at the neurophysiological level. We used a protocol combining semantic priming with electroencephalography, and manipulated the type of semantic relation and the duration of the interval between prime and target objects. Two consistent and complementary results were shown. On N1 and P3 components, semantic priming was observed for thematic relations only. On N400 component, the type of semantic relation interacted with interval duration, and semantic priming was visible for all 3 relations after the longest interval only. Results revealed graded processing time courses for thematic, specific function similarity, and general function similarity relations at the neural level, and further indicate that thematic relations impact object processing during the early stages of object recognition. Findings suggest a hierarchical organization of three types of semantic relations based on functional knowledge. The parallel between semantic relations involving manipulable artifact objects and levels of action representations is discussed.

Proper name retrieval in temporal lobe epilepsy: Naming of famous faces and landmarks

The objective of this study was to further explore proper name (PN) retrieval and conceptual knowledge in patients with left and right temporal lobe epilepsy (69 patients with LTLE and 62 patients with RTLE) using a refined assessment procedure. Based on the performance of a large group of age- and education-matched normals, a new test of famous faces and famous landmarks was designed. Recognition, naming, and semantic knowledge were assessed consecutively, allowing for a better characterization of deficient levels in the naming system. Impairment in PN retrieval was common in the cohort with TLE. Furthermore, side of seizure onset impaired stages of name retrieval differently: LTLE impaired the lexico-phonological processing, whereas RTLE mainly impaired the perceptual–semantic stage of object recognition. In addition to deficient PN retrieval, patients with TLE had reduced conceptual knowledge regarding famous persons and landmarks.

Depth of Conceptual Knowledge Modulates Visual Processes during Word Reading

Recent evidence suggests that conceptual knowledge modulates early visual stages of object recognition. The present study investigated whether similar modulations can be observed also for the recognition of object names, that is, for symbolic representations with only arbitrary relationships between their visual features and the corresponding conceptual knowledge. In a learning paradigm, we manipulated the amount of information provided about initially unfamiliar visual objects while controlling for perceptual stimulus properties and exposure. In a subsequent test session with electroencephalographic recordings, participants performed several tasks on either the objects or their written names. For objects as well as names, knowledge effects were observed as early as about 120 msec in the P1 component of the ERP, reflecting perceptual processing in extrastriate visual cortex. These knowledge-dependent modulations of early stages of visual word recognition suggest that information about word meanings may modulate the perception of arbitrarily related visual features surprisingly early.

Crowding reveals a third stage of object recognition

Crowding reveals a third stage of object recognition

How the Camel Lost Its Hump: The Impact of Object Typicality on Event-related Potential Signals in Object Decision

Using an object decision task, event-related potentials (ERPs), and minimum norm current source estimates, we investigated early spatiotemporal aspects of cortical activation elicited by line drawings that were manipulated on two dimensions: authenticity and typicality. Authentic objects were those that match real-world experience, whereas nonauthentic objects were "doctored" by deletion or addition of features (e.g., a camel with its hump removed, a hammer with two handles). The main manipulation of interest for both authentic and nonauthentic objects was the degree of typicality in the object's structure: typical items are composed of parts that have tended to co-occur across many different objects in the perceiver's experience. The ERP pattern revealed a significant typicality effect at 116 msec after stimulus onset. Both atypical authentic objects (e.g., a camel with its hump) and atypical nonauthentic objects (e.g., a jackal with a hump) elicited stronger brain activation than did objects with typical structure. A significant effect of authenticity was observed at 480 msec, with stronger activation for the nonauthentic objects. The factors of typicality and authenticity interacted at 160 and 330 msec. The most prominent source of the typicality effect was the bilateral occipitotemporal cortex, whereas the interaction and the authenticity effects were mainly observed in the more anterior bilateral temporal cortex. These findings support the hypothesis that within the first few hundred milliseconds after stimulus presentation onset, visual-form-related perceptual and conceptual processes represent distinct but interacting stages in object recognition.

Shape representation by a network of V4-like cells

Cells in extrastriate visual cortex have been reported to be selective for various configurations of local contour shape [Pasupathy, A., & Connor, C. E. (2001). Shape representation in area V4: Position-specific tuning for boundary conformation. The Journal of Neurophysiology, 86 (5), 2505-2519; Hegdé, J., & Van Essen, D. C. (2003). Strategies of shape representation in macaque visual area V2. Visual Neuroscience, 20 (3), 313-328]. Specifically, Pasupathy and Connor found that in area V4 most cells are strongly responsive to a particular local contour conformation located at a specific position on the object's boundary. We used a population of "V4-like cells"-units sensitive to multiple shape features modeled after V4 cell behavior-to generate representations of different shapes. Standard classification algorithms (earth mover's distance, support vector machines) applied to this population representation demonstrate high recognition accuracies classifying handwritten digits in the MNIST database and objects in the MPEG-7 Shape Silhouette database. We compare the performance of the V4-like unit representation to the "shape context" representation of Belongie et al. [Belongie, S., Malik, J., & Puzicha, J. (2002). Shape matching and object recognition using shape contexts. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24 (24), 509-522]. Results show roughly comparable recognition accuracies using the two representations when tested on portions of the MNIST database. We analyze the relative contributions of various V4-like feature sensitivities to recognition accuracy and robustness to noise - feature sensitivities include curvature magnitude, direction of curvature, global orientation of the contour segment, distance of the contour segment from object center, and modulatory effect of adjacent contour regions. Among these, local curvature appears to be the most informative variable for shape recognition. Our results support the hypothesis that V4 cells function as robust shape descriptors in the early stages of object recognition.

Contour Saliency in Primary Visual Cortex

Contour integration is an important intermediate stage of object recognition, in which line segments belonging to an object boundary are perceptually linked and segmented from complex backgrounds. Contextual influences observed in primary visual cortex (V1) suggest the involvement of V1 in contour integration. Here, we provide direct evidence that, in monkeys performing a contour detection task, there was a close correlation between the responses of V1 neurons and the perceptual saliency of contours. Receiver operating characteristic analysis showed that single neuronal responses encode the presence or absence of a contour as reliably as the animal's behavioral responses. We also show that the same visual contours elicited significantly weaker neuronal responses when they were not detected in the detection task, or when they were unattended. Our results demonstrate that contextual interactions in V1 play a pivotal role in contour integration and saliency.

Transient inactivation of perirhinal cortex disrupts encoding, retrieval, and consolidation of object recognition memory.

Damage to perirhinal cortex (PRh) impairs object recognition memory in humans, monkeys, and rats when tested in tasks such as delayed nonmatching to sample, visual paired comparison, and its rodent analog, the spontaneous object recognition task. In the present study, we have capitalized on the discrete one-trial nature of the spontaneous object recognition task to investigate the role of PRh in several distinct stages of object recognition memory. In a series of experiments, transient inactivation of PRh was accomplished with bilateral infusions of lidocaine directly into PRh immediately before the sample phase (encoding), immediately before the choice phase (retrieval), or within the retention delay after the sample phase (storage-consolidation). Compared with performance on trials in which they received saline infusions, rats were significantly impaired when lidocaine was infused before the sample phase, regardless of the length of the retention delay. Similarly, delay-independent deficits were observed after immediate pre-choice infusions of lidocaine. Finally, PRh inactivation immediately and 20 min after the sample phase, but not 40, 60, or 80 min after, also disrupted subsequent object recognition when the retention delay was sufficiently long to ensure the dissipation of the actions of lidocaine during the choice phase. The effects of pre-sample and pre-choice inactivation indicate involvement of PRh in encoding and retrieval stages of object recognition, and the time course of post-sample inactivation effects suggests a role for PRh in the maintenance of the object trace during memory consolidation.

Visual object recognition deficits in early Parkinson's disease

The nature of the visual perception deficits in Parkinson's disease (PD) has remained unclear. The present study explored whether there emerge deficits in the different stages of visual object recognition in early PD. Twenty-eight patients and 14 healthy controls were studied. A set of reaction time tasks were applied to measure the different stages of object recognition. The results indicate some selective problems in both basic perceptual and semantic visual processing at an early stage of cognitive deterioration in PD.

Object Recognition in the Cerebral Hemispheres as Revealed by Visual Field Experiments

Visual object recognition contains several stages of information processing carried out along the ventral visual pathway. Brain imaging and behavioural studies have suggested hemispheric asymmetries in object recognition, but the results do not provide a coherent view about the direction of the asymmetries or about the processing stage at which the asymmetries emerge. In order to clarify the contribution of the hemispheres to object recognition, two visual field experiments using normal participants were conducted. Experiment 1 used five different decision-making tasks to study processing at different stages of object recognition. The results showed that familiar objects were recognised faster when presented to the left visual field (LVF) than to the right visual field (RVF) in tasks requiring decisions between familiar objects vs scrambled objects or between familiar objects vs coherent novel objects, suggesting that the right hemisphere is superior in matching visual stimuli to stored representations. Experiment 2 replicated the LVF advantage in object decisions with the novel objects from Experiment 1 that had an unfamiliar overall shape but basic features similar to those of familiar objects. In addition, a RVF advantage emerged with chimeric objects composed by changing parts of familiar objects. The results suggest that the right hemisphere is superior at accessing the overall shape of objects from memory whereas the left hemisphere is superior at analysing whether the parts of objects match to memory representations.