Visual Object Recognition Tasks Research Articles

Memory-related brain potentials for visual objects in early AD show impairment and compensatory mechanisms.

Impaired episodic memory is the primary feature of early Alzheimer's disease (AD), but not all memories are equally affected. Patients with AD and amnestic Mild Cognitive Impairment (aMCI) remember pictures better than words, to a greater extent than healthy elderly. We investigated neural mechanisms for visual object recognition in 30 patients (14AD, 16 aMCI) and 36 cognitively unimpaired healthy (19 in the "preclinical" stage of AD). Event-related brain potentials (ERPs) were recorded while participants performed a visual object recognition task. Hippocampal occupancy (integrity), amyloid (florbetapir) PET, and neuropsychological measures of verbal & visual memory, executive function were also collected. A right-frontal ERP recognition effect (500-700ms post-stimulus) was seen in cognitively unimpaired participants only, and significantly correlated with memory and executive function abilities. A later right-posterior negative ERP effect (700-900ms) correlated with visual memory abilities across participants with low verbal memory ability, and may reflect a compensatory mechanism. A correlation of this retrieval-related negativity with right hippocampal occupancy (r = 0.55), implicates the hippocampus in the engagement of compensatory perceptual retrieval mechanisms. Our results suggest that early AD patients are impaired in goal-directed retrieval processing, but may engage compensatory perceptual mechanisms which rely on hippocampal function.

Skills and cautiousness predict performance in difficult search.

People differ in how well they search. What are the factors that might contribute to this variability? We tested the contribution of two cognitive abilities: visual working memory (VWM) capacity and object recognition ability. Participants completed three tasks: a difficult inefficient visual search task, where they searched for a target letter T among skewed L distractors; a VWM task, where they memorized a color array and then identified whether a probed color belonged to the previous array; and the Novel Object Memory Test (NOMT), where they learnt complex novel objects and then identified them amongst objects that closely resembled them. Exploratory and confirmatory factor analyses revealed that there are two latent factors that explain the shared variance among these three tasks: a factor indicative of the level of caution participants exercised during the challenging visual search task, and a factor representing their visual cognitive abilities. People who score high on the search cautiousness tend to perform a more accurate but slower search. People who score high on the visual cognitive ability factor tend to have a higher VWM capacity, a better object recognition ability, and a faster search speed. The results reflect two points: (1) Visual search tasks share components with visual working memory and object recognition tasks. (2) Search performance is influenced not only by the search display's properties but also by individual predispositions such as caution and general visual abilities. This study introduces new factors for consideration when interpreting variations in visual search behaviors.

The oscillatory features of visual processing are altered in healthy aging.

The temporal features of visual processing were compared between young and elderly healthy participants in visual object and word recognition tasks using the technique of random temporal sampling. The target stimuli were additively combined with a white noise field and were exposed very briefly (200 ms). Target visibility oscillated randomly throughout exposure duration by manipulating the signal-to-noise ratio (SNR). Classification images (CIs) based on response accuracy were calculated to reflect processing efficiency according to the time elapsed since target onset and the power of SNR oscillations in the 5-55 Hz range. CIs differed substantially across groups whereas individuals of the same group largely shared crucial features such that a machine learning algorithm reached 100% accuracy in classifying the data patterns of individual participants into their proper group. These findings demonstrate altered perceptual oscillations in healthy aging and are consistent with previous investigations showing brain oscillation anomalies in the elderly.

Gender-Specific Interactions in a Visual Object Recognition Task in Persons with Opioid Use Disorder.

Opioid use disorder (OUD)-associated overdose deaths have reached epidemic proportions worldwide over the past two decades, with death rates for men reported at twice the rate for women. Using a controlled, cross-sectional, age-matched (18-56 y) design to better understand the cognitive neuroscience of OUD, we evaluated the electroencephalographic (EEG) responses of male and female participants with OUD vs. age- and gender-matched non-OUD controls during a simple visual object recognition Go/No-Go task. Overall, women had significantly slower reaction times (RTs) than men. In addition, EEG N200 and P300 event-related potential (ERP) amplitudes for non-OUD controls were significantly larger for men, while their latencies were significantly shorter than for women. However, while N200 and P300 amplitudes were not significantly affected by OUD for either men or women in this task, latencies were also affected differentially in men vs. women with OUD. Accordingly, for both N200 and P300, male OUD participants exhibited longer latencies while female OUD participants exhibited shorter ones than in non-OUD controls. Additionally, robust oscillations were found in all participants during a feedback message associated with performance in the task. Although alpha and beta power during the feedback message were significantly greater for men than women overall, both alpha and beta oscillations exhibited significantly lower power in all participants with OUD. Taken together, these findings suggest important gender by OUD differences in cognitive processing and reflection of performance in this simple visual task.

Discrepancy-Guided Domain-Adaptive Data Augmentation.

Data augmentation has been observed playing a crucial role in achieving better generalization in many machine learning tasks, especially in unsupervised domain adaptation (DA). It is particularly effective on visual object recognition tasks as images are high-dimensional with an enormous range of variations that can be simulated. Existing data augmentation techniques, however, are not explicitly designed to address the differences between different domains. Expert knowledge about the data is required, as well as manual efforts in finding the optimal parameters. In this article, we propose a novel domain-adaptive augmentation method by making use of a state-of-the-art style transfer method and domain discrepancy measurement. Specifically, we measure the discrepancy between source and target domains, and use it as a guide to augment the original source samples using style transferred source-to-target samples. The proposed domain-adaptive augmentation method is data and model agnostic that can be easily incorporated with state-of-the-art DA algorithms. We show empirically that, by using this domain-adaptive augmentation, we are able to gradually reduce the discrepancy between the source and target samples, and further boost the adaptation performance using different DA algorithms on three popular domain adaption datasets.

Stability of face recognition abilities after left or right anterior temporal lobectomy.

Patients with anterior temporal lobe (ATL) resection due to mesial temporal lobe epilepsy (MTLE) have difficulties at identifying familiar faces and explicitly remembering newly learned faces but their ability to individuate unfamiliar faces remains largely unknown. Moreover, the extent to which their difficulties with familiar face identity recognition and learning is truly due to the ATL resection remains unknown. Here, we report a study of 24 MTLE patients and matched healthy controls tested with an extensive set of seven face and visual object recognition tasks (including three tasks evaluating unfamiliar face individuation) before and about 6 months after unilateral (nine left, 15 right) ATL resection. We found that ATL resection has little or no effect on the patients' preserved pre-surgical ability to perform unfamiliar face individuation, both at the group and individual levels. More surprisingly, ATL resection also has little effect on the patients' performance at recognizing and naming famous faces as well as at learning new faces. A substantial proportion of right MTLE patients (33%) even improved their response times on several tasks, which may indicate a functional release of visuo-spatial processing after resection in the right ATL. Altogether this study shows that face recognition abilities are mainly unaffected by ATL resection in MTLE, either because the critical regions for face recognition are spared or because performance at some tasks is already lower than normal preoperatively. Overall, these findings urge caution when interpreting the causal effect of brain lesions on face recognition ability in patients with ATL resection due to MTLE. They also illustrate the complexity of predicting cognitive outcomes after epilepsy surgery because of the influence of many different intertwined factors.

Visual Object Recognition From Single-Trial EEG Signals Using Machine Learning Wrapper Techniques.

Responses of the human brain to different visual stimuli elicit specific patterns in electroencephalography (EEG) signals. It is confirmed that by analyzing these patterns, we can recognize the category of the visited objects. However, high levels of noise and artifacts in EEG signals and the discrepancies between the recorded data from different subjects in visual object recognition task make classification of cognitive states of subjects a serious challenge. In this research, we present a framework for evaluating machine learning and wrapper channel selection algorithms used for classifying single-trial EEG signals recorded in response to photographic stimuli. It is shown that by correctly mapping the entire EEG data space to informative feature spaces (IFS), the performance of the classification methods can improve significantly. Results outperform the state-of-the-art results and confirm efficiency of the proposed feature selection methods in capturing the most informative EEG channels. This can help to achieve high separability of object categories in single-trial visual object recognition task.

Contrasting attentional processing in visual search, object recognition, and complex tasks

Contrasting attentional processing in visual search, object recognition, and complex tasks

Updating the dual-mechanism model for cross-sensory attentional spreading: The influence of space-based visual selective attention.

Selective attention to visual stimuli can spread cross‐modally to task‐irrelevant auditory stimuli through either the stimulus‐driven binding mechanism or the representation‐driven priming mechanism. The stimulus‐driven attentional spreading occurs whenever a task‐irrelevant sound is delivered simultaneously with a spatially attended visual stimulus, whereas the representation‐driven attentional spreading occurs only when the object representation of the sound is congruent with that of the to‐be‐attended visual object. The current study recorded event‐related potentials in a space‐selective visual object‐recognition task to examine the exact roles of space‐based visual selective attention in both the stimulus‐driven and representation‐driven cross‐modal attentional spreading, which remain controversial in the literature. Our results yielded that the representation‐driven auditory Nd component (200–400 ms after sound onset) did not differ according to whether the peripheral visual representations of audiovisual target objects were spatially attended or not, but was decreased when the auditory representations of target objects were presented alone. In contrast, the stimulus‐driven auditory Nd component (200–300 ms) was decreased but still prominent when the peripheral visual constituents of audiovisual nontarget objects were spatially unattended. These findings demonstrate not only that the representation‐driven attentional spreading is independent of space‐based visual selective attention and benefits in an all‐or‐nothing manner from object‐based visual selection for actually presented visual representations of target objects, but also that although the stimulus‐driven attentional spreading is modulated by space‐based visual selective attention, attending to visual modality per se is more likely to be the endogenous determinant of the stimulus‐driven attentional spreading.

Neural Basis of Semantically Dependent and Independent Cross-Modal Boosts on the Attentional Blink.

The present study recorded event-related potentials (ERPs) in a visual object-recognition task under the attentional blink paradigm to explore the temporal dynamics of the cross-modal boost on attentional blink and whether this auditory benefit would be modulated by semantic congruency between T2 and the simultaneous sound. Behaviorally, the present study showed that not only a semantically congruent but also a semantically incongruent sound improved T2 discrimination during the attentional blink interval, whereas the enhancement was larger for the congruent sound. The ERP results revealed that the behavioral improvements induced by both the semantically congruent and incongruent sounds were closely associated with an early cross-modal interaction on the occipital N195 (192-228ms). In contrast, the lower T2 accuracy for the incongruent than congruent condition was accompanied by a larger late occurring cento-parietal N440 (424-448ms). These findings suggest that the cross-modal boost on attentional blink is hierarchical: the task-irrelevant but simultaneous sound, irrespective of its semantic relevance, firstly enables T2 to escape the attentional blink via cross-modally strengthening the early stage of visual object-recognition processing, whereas the semantic conflict of the sound begins to interfere with visual awareness only at a later stage when the representation of visual object is extracted.

Identification of White Matter Networks Engaged in Object (Face) Recognition Showing Differential Responses to Modulated Stimulus Strength.

Blood-oxygenation-level-dependent (BOLD) signals in magnetic resonance imaging indirectly reflect neural activity in cortex, but they are also detectable in white matter (WM). BOLD signals in WM exhibit strong correlations with those in gray matter (GM) in a resting state, but their interpretation and relationship to GM activity in a task are unclear. We performed a parametric visual object recognition task designed to modulate the BOLD signal response in GM regions engaged in higher order visual processing, and measured corresponding changes in specific WM tracts. Human faces embedded in different levels of random noise have previously been shown to produce graded changes in BOLD activation in for example, the fusiform gyrus, as well as in electrophysiological (N170) evoked potentials. The magnitudes of BOLD responses in both GM regions and selected WM tracts varied monotonically with the stimulus strength (noise level). In addition, the magnitudes and temporal profiles of signals in GM and WM regions involved in the task coupled strongly across different task parameters. These findings reveal the network of WM tracts engaged in object (face) recognition and confirm that WM BOLD signals may be directly affected by neural activity in GM regions to which they connect.

Detection and Monitoring of Thermal Lesions Induced by Microwave Ablation Using Ultrasound Imaging and Convolutional Neural Networks.

Microwave ablation (MWA) for cancer treatment is frequently monitored by ultrasound (US) B-mode imaging in the clinic, which often fails due to the low intrinsic contrast between the thermal lesion and normal tissue. Deep learning, especially convolutional neural network (CNN), has shown significant improvements in medical image analysis. Here, we propose and evaluate an US imaging based on a CNN architecture for the detection and monitoring of thermal lesions induced by MWA in porcine livers. Unlike dealing with images in many visual object recognition tasks, US radiofrequency (RF) data backscattered from the ablated region were utilized to capture features related to the thermal lesion. The dataset comprised of 1640 US RF envelope data matrices and their corresponding gross-pathology images, and were utilized for training and testing. After envelope detection, US B-mode, segmentation results based on CNN (SICNN), and modified CNN (SIm-CNN) for US data were simultaneously reconstructed to reveal the suitability for monitoring of MWA. The SICNN and SIm-CNN outperformed B-mode images for the detection and monitoring of MWA-induced thermal lesions. The values of the area under the receiver operating characteristic curve were 0.8728 and 0.8948 for the SICNN and Sim-CNN, respectively, which were both higher than the value of 0.6904 for B-mode images. Ablated regions that were assessed using SIm-CNN showed a good correlation (J 0.8845, r 0.8739, and E 0.410) to gross-pathology images. This study was the first to illustrate that SIm-CNN has the potential to detect and monitor thermal lesions, and may be utilized as an alternative modality for image-guided MWA treatments.

Hypergraph Neural Networks

In this paper, we present a hypergraph neural networks (HGNN) framework for data representation learning, which can encode high-order data correlation in a hypergraph structure. Confronting the challenges of learning representation for complex data in real practice, we propose to incorporate such data structure in a hypergraph, which is more flexible on data modeling, especially when dealing with complex data. In this method, a hyperedge convolution operation is designed to handle the data correlation during representation learning. In this way, traditional hypergraph learning procedure can be conducted using hyperedge convolution operations efficiently. HGNN is able to learn the hidden layer representation considering the high-order data structure, which is a general framework considering the complex data correlations. We have conducted experiments on citation network classification and visual object recognition tasks and compared HGNN with graph convolutional networks and other traditional methods. Experimental results demonstrate that the proposed HGNN method outperforms recent state-of-theart methods. We can also reveal from the results that the proposed HGNN is superior when dealing with multi-modal data compared with existing methods.

Different attentional dysfunctions in eEF2K-/- , IL1RAPL1-/- and SHANK3Δ11-/- mice.

A common feature of several psychiatric disorders is the attentional impairment. eEF2K -/- , IL1RAPL1 -/- and SHANK3Δ11 -/- mice were used as animal models consistently linked to changes in synaptic plasticity, learning and memory. All knockout (KO) mice and their corresponding littermates were submitted to the novel object recognition (NOR) and visual object recognition (VOR) tasks. In the NOR, eEF2K-/- mice exhibited a normal performance in terms of mean discrimination index, while SHANK3Δ11-/- and IL1RAPL1 -/- mice were impaired when a delay of 2 and 24 hours was introduced. Surprisingly, when submitted to VOR, where the two objects were replaced with two shapes delivered from two iPods, all the mutant mice performed worse than those in the NOR. In VOR, the application of motion to different shapes, to increase attention, improved performance in eEF2K -/- and IL1RAPL1 -/- but not in SHANK3Δ11 -/- mice. In SHANK3Δ11 -/- mice, attentional deficit was also present even if different motions were applied to the same shapes or when these mice were repeatedly exposed for 5 days to the context. Behavioral analysis showed that eEF2K-/- and IL1RAPL1 -/- mice had a good flexibility tested in the T-maze. eEF2K-/- showed normal self-grooming. On the basis of previous literature data indicating that SHANK3Δ11 -/- showed impaired flexibility and reduced sociability, we identified in this genotype the most exhaustive model showing all the core symptoms of autism spectrum disorder including a heavy visual attention deficit. These findings show the importance of VOR to identify mouse models of autism.

Behavioral testing and litter effects in the rabbit

BackgroundBehavioral testing provides an essential approach in further developing our understanding of brain structure and function. The aim of our study was to outline a more expanded approach to cognition- and anxiety-related behavior in the rabbit. MethodsTwenty-one 70-day old rabbits (13 female, 8 male) were exposed to open field test, dark-light box test and object recognition testing with variations in inter-trial-interval, olfactory recognition and object location testing. Independent T-tests were used to compare data by individual baseline characteristics, i.e. birth weight, weight at testing, sex, litter #, litter size. ResultsIn the open field test, median time spent in the center was 3.64 s (0.84–41.36) for the 9 rabbits who entered the center; median distance moved in the arena was 874.42 cm (54.20–3444.83). In the dark light box test, 12 rabbits entered the light compartment. In the object recognition task, rabbits spent significantly less time exploring the familiar object compared to the novel (0.40 s [0–2.8] vs. 3.17 s [1.30–32.69]; P = 0.003) when using a 30-min inter-trial interval, as well with a 90-min inter-trial interval: 0.87 s [0–7.8] vs. 7.65 s [0–37.6] (P = 0.008). However, recognition was lost when using a 24-h inter-trial interval (time spent exploring the familiar object: 3.33 [0–10.90]; novel object:3.87 [1.15–48.53]; n.s). In the object location task and in olfactory object recognition task, median discrimination indexes were 0.69 (−1 to 1) and 0.37 (−0.38 to 0.78) respectively, higher than level expected by chance (P < 0.001). Litter size >3 during the neonatal period was associated with increased explorative behavior in the dark light box test (P = 0.046) and in the visual object recognition task (P = 0.005), whereas body weight and sex were not. ConclusionsSettings and outcome measures for multiple behavioral tests, providing reference values and considerations for future developmental studies are reported. Discrimination and memory in the rabbit appear to relate to litter characteristics, although a larger sample size is needed to confirm our findings.

Structure preserving dimensionality reduction for visual object recognition

Robust object recognition has drawn increasing attention in the field of computer vision and machine learning with fast development in feature extraction and classification techniques, and release of public datasets, such as Caltech datasets, Pascal Visual Object Classes, and ImageNet. Recently, deep learning based object recognition systems have shown significant performance improvements in visual object recognition tasks using innovative learning methodology. However, high dimensional space searching and recognition is time consuming, so performing point and range queries in high dimension is reconsidered for object recognition. This paper proposes optimized dimensionality reduction using structured sparse principle component analysis. The proposed method retains high dimensional feature structures, removes redundant features that do not contribute to similarity, and classifies the query image in a large database. The qualitative and quantitative experimental results, including a comparison with the current state-of-the-art visual object recognition algorithms, verify that the proposed recognition algorithm performs favorably in reducing the query image dimension and number of training images.

Linear and Non-Linear Visual Feature Learning in Rat and Humans.

The visual system processes visual input in a hierarchical manner in order to extract relevant features that can be used in tasks such as invariant object recognition. Although typically investigated in primates, recent work has shown that rats can be trained in a variety of visual object and shape recognition tasks. These studies did not pinpoint the complexity of the features used by these animals. Many tasks might be solved by using a combination of relatively simple features which tend to be correlated. Alternatively, rats might extract complex features or feature combinations which are nonlinear with respect to those simple features. In the present study, we address this question by starting from a small stimulus set for which one stimulus-response mapping involves a simple linear feature to solve the task while another mapping needs a well-defined nonlinear combination of simpler features related to shape symmetry. We verified computationally that the nonlinear task cannot be trivially solved by a simple V1-model. We show how rats are able to solve the linear feature task but are unable to acquire the nonlinear feature. In contrast, humans are able to use the nonlinear feature and are even faster in uncovering this solution as compared to the linear feature. The implications for the computational capabilities of the rat visual system are discussed.

Prediction Reweighting for Domain Adaptation.

There are plenty of classification methods that perform well when training and testing data are drawn from the same distribution. However, in real applications, this condition may be violated, which causes degradation of classification accuracy. Domain adaptation is an effective approach to address this problem. In this paper, we propose a general domain adaptation framework from the perspective of prediction reweighting, from which a novel approach is derived. Different from the major domain adaptation methods, our idea is to reweight predictions of the training classifier on testing data according to their signed distance to the domain separator, which is a classifier that distinguishes training data (from source domain) and testing data (from target domain). We then propagate the labels of target instances with larger weights to ones with smaller weights by introducing a manifold regularization method. It can be proved that our reweighting scheme effectively brings the source and target domains closer to each other in an appropriate sense, such that classification in target domain becomes easier. The proposed method can be implemented efficiently by a simple two-stage algorithm, and the target classifier has a closed-form solution. The effectiveness of our approach is verified by the experiments on artificial datasets and two standard benchmarks, a visual object recognition task and a cross-domain sentiment analysis of text. Experimental results demonstrate that our method is competitive with the state-of-the-art domain adaptation algorithms.

Actions in the Eye: Dynamic Gaze Datasets and Learnt Saliency Models for Visual Recognition.

Systems based on bag-of-words models from image features collected at maxima of sparse interest point operators have been used successfully for both computer visual object and action recognition tasks. While the sparse, interest-point based approach to recognition is not inconsistent with visual processing in biological systems that operate in `saccade and fixate' regimes, the methodology and emphasis in the human and the computer vision communities remains sharply distinct. Here, we make three contributions aiming to bridge this gap. First, we complement existing state-of-the art large scale dynamic computer vision annotated datasets like Hollywood-2 [1] and UCF Sports [2] with human eye movements collected under the ecological constraints of visual action and scene context recognition tasks. To our knowledge these are the first large human eye tracking datasets to be collected and made publicly available for video, vision.imar.ro/eyetracking (497,107 frames, each viewed by 19 subjects), unique in terms of their (a) large scale and computer vision relevance, (b) dynamic, video stimuli, (c) task control, as well as free-viewing. Second, we introduce novel dynamic consistency and alignment measures, which underline the remarkable stability of patterns of visual search among subjects. Third, we leverage the significant amount of collected data in order to pursue studies and build automatic, end-to-end trainable computer vision systems based on human eye movements. Our studies not only shed light on the differences between computer vision spatio-temporal interest point image sampling strategies and the human fixations, as well as their impact for visual recognition performance, but also demonstrate that human fixations can be accurately predicted, and when used in an end-to-end automatic system, leveraging some of the advanced computer vision practice, can lead to state of the art results.

Learning Deep Generative Models

Building intelligent systems that are capable of extracting high-level representations from high-dimensional sensory data lies at the core of solving many artificial intelligence–related tasks, including object recognition, speech perception, and language understanding. Theoretical and biological arguments strongly suggest that building such systems requires models with deep architectures that involve many layers of nonlinear processing. In this article, we review several popular deep learning models, including deep belief networks and deep Boltzmann machines. We show that (a) these deep generative models, which contain many layers of latent variables and millions of parameters, can be learned efficiently, and (b) the learned high-level feature representations can be successfully applied in many application domains, including visual object recognition, information retrieval, classification, and regression tasks.