Visual Recognition Performance Research Articles

MEDIAL TEMPORAL LOBE TAU IS STRONGLY RELATED TO EPISODIC MEMORY DECLINE IN AGING

The medial temporal lobe (MTL) is the earliest site of tau accumulation. MTL dysfunction may underlie episodic memory decline in normal aging and early dementia. Postmortem data indicate that transentorhinal cortex and hippocampus (HC) tau pathology is common over 60 years-of-age, while spread to inferior temporal (IFT) regions and worsening cognition is associated with beta-amyloid (Aβ). We used PET to investigate how in vivo tau accumulation in temporal lobe regions, Aβ deposition, and MTL atrophy measures contribute to memory and memory decline in cognitively normal older adults. Seventy-five subjects (77±6yrs) underwent [18F]AV-1451 tau PET, [11C]PiB amyloid PET, T1-weighted structural MRI, and neuropsychological assessment (60 with ≥ 2 sessions). T1-images were segmented (FreeSurfer v5.3) to derive volume/thickness measures, a global cortical PiB DVR, and regional AV-1451 measures in native space (80–100min SUVR, cerebellar gray reference, partial volume corrected). Mean SUVRs were derived from MTL and IFT, and temporal lobe subregions (after automated division along the anterior-posterior axis). An episodic memory composite score was derived by averaging Z-transformed verbal and visual early recall, late recall and recognition performance. Changes in memory were estimated by linear-mixed effects models. Stepwise multiple regressions identified the best predictor(s) for memory or memory change, among demographic variables, MTL and IFT tau, global Aβ, HC volume, and entorhinal (EC) thickness. Interactive effects of MTL or IFT tau x Aβ on memory were tested by general linear models. Contributions of temporal lobe subregions were assessed by subsequent regressions. Stepwise regressions identified MTL tau as the best predictor of memory and memory decline (R2=.34 and .33, p<.001). Correlations between MTL tau and memory were seen in both PiB- and PiB+ individuals (Figure). GLMs did not reveal any significant interactive effect of MTL or IFT tau with Aβ on memory. Among temporal lobe subregions, posterior EC and parahippocampal cortex best predicted memory (R2=.32), whereas medial IFT and posterior EC best predicted memory decline (R2=.34). Consistent with neuropathological studies, our data suggest close links between MTL tau pathology (particularly in posterior EC) and age-related memory dysfunction, with stronger decline associated with involvement of IFT regions. Relationship between episodic memory at session closest to AV-1451 tau scan and tau tracer uptake in the posterior EC (PiB- subjects: rPearson=-.43; rSpearman=-.41; PiB+ subjects: rPearson=-.52; rSpearman=-.54; all p<.001). A similar relationship was seen between memory and whole MTL mean SUVR, which we defined as average across Freesurfer-defined EC, para-hippocampal gyrus, hippocampus and amygdala. Posterior EC was defined here as the part of EC/parahippocampal gyrus (Freesurfer nomenclature) that covers the same slices as the anterior half of the hippocampus.

Masking the Identities of Celebrities and Personally Familiar Individuals: Effects on Visual and Auditory Recognition Performance.

The current study compared the effectiveness of masking on recognition performance for faces and voices of celebrities and personally familiar individuals. On the basis of the theory suggesting stronger memory representations for personally familiar individuals, we expected masking to be more effective for celebrities than for personally familiar stimulus persons. Furthermore, we sought to replicate the face recognition advantage with masked stimuli. Face pixelation and electronic changes of the voice pitch were applied as masking techniques, using four masking levels for each stimulus. Thirty-one undergraduate students were presented with the masked faces and voices of 10 celebrities and 10 personally familiar fellow students. As expected, more correct recognitions occurred for faces than for voices, suggesting that masking does not counteract the mechanisms causing the face recognition advantage. Unexpectedly, masking effectiveness did not differ between celebrities and personally familiar individuals. This may be due to the type of personally familiar individuals used. Within personally familiar stimuli, increased familiarity did not predict the effectiveness of masking. Whereas the highest masking level eliminated speaker recognition, masking did not fully eliminate face recognition. From a practical perspective, the findings especially question the suitability of pixelation as a means for identity concealment.

Visual Localization by Place Recognition Based on Multifeature (D-λLBP++HOG)

Visual localization is widely used in the autonomous navigation system and Advanced Driver Assistance Systems (ADAS). This paper presents a visual localization method based on multifeature fusion and disparity information using stereo images. We integrate disparity information into complete center-symmetric local binary patterns (CSLBP) to obtain a robust global image description (D-CSLBP). In order to represent the scene in depth, multifeature fusion of D-CSLBP and HOG features provides valuable information and permits decreasing the effect of some typical problems in place recognition such as perceptual aliasing. It improves visual recognition performance by taking advantage of depth, texture, and shape information. In addition, for real-time visual localization, local sensitive hashing method (LSH) was used to compress the high-dimensional multifeature into binary vectors. It can thus speed up the process of image matching. To show its effectiveness, the proposed method is tested and evaluated using real datasets acquired in outdoor environments. Given the obtained results, our approach allows more effective visual localization compared with the state-of-the-art method FAB-MAP.

Submodular Attribute Selection for Visual Recognition

In real-world visual recognition problems, low-level features cannot adequately characterize the semantic content in images, or the spatio-temporal structure in videos. In this work, we encode objects or actions based on attributes that describe them as high-level concepts. We consider two types of attributes. One type of attributes is generated by humans, while the second type is data-driven attributes extracted from data using dictionary learning methods. Attribute-based representation may exhibit variations due to noisy and redundant attributes. We propose a discriminative and compact attribute-based representation by selecting a subset of discriminative attributes from a large attribute set. Three attribute selection criteria are proposed and formulated as a submodular optimization problem. A greedy optimization algorithm is presented and its solution is guaranteed to be at least (1-1/e)-approximation to the optimum. Experimental results on four public datasets demonstrate that the proposed attribute-based representation significantly boosts the performance of visual recognition and outperforms most recently proposed recognition approaches.

SYSTEMATIZATION OF SPACE OF STRUCTURAL FEATURES BASED ON SELF-LEARNING METHODS FOR EFFECTIVE IMAGE RECOGNITION

The work deals with issues of clustering sets of characteristic features of images. For the construction of array of the characteristic features is used method Speeded Up Robust Features. Implemented algorithms for clustering structural descriptions of images on the basis of a self-organizing Kohonen neural network and method of grouping the difference. The object of the research are clustering methods which applied to the set of structural features. The aim is to construct a vector representations of descriptions based on clustering, which increases the speed of recognition. The subject of research is systematization a set of structural features of visual objects. Discussing the results of the application of clustering methods for structural descriptions of images in the form of sets of characteristic features to improve the performance of visual recognition of objects. For systematization and compression the feature space proposed to carry out self-study using the methods of differential grouping and Kohonen networks. The simulation and experimental study of clustering methods on examples of specific sets of characteristic features were done. The research results proves the possibility of effective representation of the descriptions in the form of a vector with integer elements. This approach can be used to solve problems of recognition and retrieval of images. As a result compact vector description of etalon images is built, quantitative estimates of clustering error are estimated, efficiency of proposed method during processing of real image database is confirmed.

A Novel Locally Linear KNN Method With Applications to Visual Recognition.

A locally linear K Nearest Neighbor (LLK) method is presented in this paper with applications to robust visual recognition. Specifically, the concept of an ideal representation is first presented, which improves upon the traditional sparse representation in many ways. The objective function based on a host of criteria for sparsity, locality, and reconstruction is then optimized to derive a novel representation, which is an approximation to the ideal representation. The novel representation is further processed by two classifiers, namely, an LLK-based classifier and a locally linear nearest mean-based classifier, for visual recognition. The proposed classifiers are shown to connect to the Bayes decision rule for minimum error. Additional new theoretical analysis is presented, such as the nonnegative constraint, the group regularization, and the computational efficiency of the proposed LLK method. New methods such as a shifted power transformation for improving reliability, a coefficients' truncating method for enhancing generalization, and an improved marginal Fisher analysis method for feature extraction are proposed to further improve visual recognition performance. Extensive experiments are implemented to evaluate the proposed LLK method for robust visual recognition. In particular, eight representative data sets are applied for assessing the performance of the LLK method for various visual recognition applications, such as action recognition, scene recognition, object recognition, and face recognition.

Integrating oculomotor and perceptual training to induce a pseudofovea: A model system for studying central vision loss.

People with a central scotoma often adopt an eccentric retinal location (Preferred Retinal Locus, PRL) for fixation. Here, we proposed a novel training paradigm as a model system to study the nature of the PRL formation and its impacts on visual function. The training paradigm was designed to effectively induce a PRL at any intended retinal location by integrating oculomotor control and pattern recognition. Using a gaze-contingent display, a simulated central scotoma was induced in eight normally sighted subjects. A subject's entire peripheral visual field was blurred, except for a small circular aperture with location randomly assigned to each subject (to the left, right, above, or below the scotoma). Under this viewing condition, subjects performed a demanding oculomotor and visual recognition task. Various visual functions were tested before and after training at both PRL and nonPRL locations. After 6–10 hr of the training, all subjects formed their PRL within the clear window. Both oculomotor control and visual recognition performance significantly improved. Moreover, there was considerable improvement at PRL location in high-level function, such as trigram letter-recognition, reading, and spatial attention, but not in low-level function, such as acuity and contrast sensitivity. Our results demonstrated that within a relatively short time, a PRL could be induced at any intended retinal location in normally-sighted subjects with a simulated scotoma. Our training paradigm might not only hold promise as a model system to study the dynamic nature of the PRL formation, but also serve as a rehabilitation regimen for individuals with central vision loss.

Actions in the Eye: Dynamic Gaze Datasets and Learnt Saliency Models for Visual Recognition.

Systems based on bag-of-words models from image features collected at maxima of sparse interest point operators have been used successfully for both computer visual object and action recognition tasks. While the sparse, interest-point based approach to recognition is not inconsistent with visual processing in biological systems that operate in `saccade and fixate' regimes, the methodology and emphasis in the human and the computer vision communities remains sharply distinct. Here, we make three contributions aiming to bridge this gap. First, we complement existing state-of-the art large scale dynamic computer vision annotated datasets like Hollywood-2 [1] and UCF Sports [2] with human eye movements collected under the ecological constraints of visual action and scene context recognition tasks. To our knowledge these are the first large human eye tracking datasets to be collected and made publicly available for video, vision.imar.ro/eyetracking (497,107 frames, each viewed by 19 subjects), unique in terms of their (a) large scale and computer vision relevance, (b) dynamic, video stimuli, (c) task control, as well as free-viewing. Second, we introduce novel dynamic consistency and alignment measures, which underline the remarkable stability of patterns of visual search among subjects. Third, we leverage the significant amount of collected data in order to pursue studies and build automatic, end-to-end trainable computer vision systems based on human eye movements. Our studies not only shed light on the differences between computer vision spatio-temporal interest point image sampling strategies and the human fixations, as well as their impact for visual recognition performance, but also demonstrate that human fixations can be accurately predicted, and when used in an end-to-end automatic system, leveraging some of the advanced computer vision practice, can lead to state of the art results.

Socio-mobile landmark recognition using local features with adaptive region selection

With the fast development of mobile devices as well as the broadband wireless network, mobile devices are playing a more and more important role in people׳s daily life. Nowadays, many landmark images are captured by mobile devices. However, these images are often captured under different lightening conditions with varied poses and camera orientations. Besides, people are inherently connected by personal interests as well as various interactions. To alleviate the imaging problem with mobile devices as well as take advantage of the social information for mobile visual applications, we propose a novel socio-mobile visual recognition method using local features with adaptive region selection. We densely extract local regions and use the pixel gradients to represent each local region. Each local region is divided into 4×4 subregions to combine the spatial information. Instead of using fixed pixel numbers for each subregion, we adaptively choose the proper size of each subregion to cope with varied poses and camera orientations. The most discriminative local features are then chosen by minimizing the sparse coding loss. Besides, a geo-discriminative codebook is also generated to take advantages of images׳ location information. Moreover, we jointly consider the visual distances as well as user׳s friends׳ matching results to further boost the final visual recognition performance. We achieve the state-of-the-art performance on the Stanford mobile visual search dataset and the San Francisco landmark dataset. These experimental results demonstrate the effectiveness and efficiency of the proposed adaptive region selection based local features for socio-mobile landmark recognition.

Visual Speech Recognition Using Weighted Dynamic Time Warping

It is unclear whether Hidden Markov Model (HMM) or Dynamic Time Warping (DTW) mapping is more appropriate for visual speech recognition when only small data samples are available. In this letter, the two approaches are compared in terms of sensitivity to the amount of training samples and computing time with the objective of determining the tipping point. The limited training data problem is addressed by exploiting a straightforward template matching via weighted-DTW. The proposed framework is a refined DTW by adjusting the warping paths with judicially injected weights to ensure a smooth diagonal path for accurate alignment without added computational load. The proposed WDTW is evaluated on three databases (two in the public domain and one developed in-house) for visual recognition performance. Subsequent experiments indicate that the proposed WDTW significantly enhances the recognition rate compared to the DTW and HMM based algorithms, especially under limited data samples.

Deep neural networks rival the representation of primate IT cortex for core visual object recognition.

The primate visual system achieves remarkable visual object recognition performance even in brief presentations, and under changes to object exemplar, geometric transformations, and background variation (a.k.a. core visual object recognition). This remarkable performance is mediated by the representation formed in inferior temporal (IT) cortex. In parallel, recent advances in machine learning have led to ever higher performing models of object recognition using artificial deep neural networks (DNNs). It remains unclear, however, whether the representational performance of DNNs rivals that of the brain. To accurately produce such a comparison, a major difficulty has been a unifying metric that accounts for experimental limitations, such as the amount of noise, the number of neural recording sites, and the number of trials, and computational limitations, such as the complexity of the decoding classifier and the number of classifier training examples. In this work, we perform a direct comparison that corrects for these experimental limitations and computational considerations. As part of our methodology, we propose an extension of “kernel analysis” that measures the generalization accuracy as a function of representational complexity. Our evaluations show that, unlike previous bio-inspired models, the latest DNNs rival the representational performance of IT cortex on this visual object recognition task. Furthermore, we show that models that perform well on measures of representational performance also perform well on measures of representational similarity to IT, and on measures of predicting individual IT multi-unit responses. Whether these DNNs rely on computational mechanisms similar to the primate visual system is yet to be determined, but, unlike all previous bio-inspired models, that possibility cannot be ruled out merely on representational performance grounds.

The effect of decreased interletter spacing on orthographic processing.

There is growing interest in how perceptual factors such as the spacing between letters within words modulate performance in visual word recognition and reading aloud. Extra-large letter spacing can strongly improve the reading performance of dyslexic children, and a small increase with respect to the standard spacing seems beneficial even for skilled word recognition in adult readers. In the present study we examined the effect of decreased letter spacing on perceptual identification and lexical decision tasks. Identification in the decreased spacing condition was slower than identification of normally spaced strings, thereby confirming that the reciprocal interference among letters located in close proximity (crowding) poses critical constraints on visual word processing. Importantly, the effect of spacing was not modulated by string length, suggesting that the locus of the spacing effect is at the level of letter detectors. Moreover, the processing of crowded letters was facilitated by top-down support from orthographic lexical representation as indicated by the fact that decreased spacing affected pseudowords significantly more than words. Conversely, in the lexical decision task only word responses were affected by the spacing manipulation. Overall, our findings support the hypothesis that increased crowding is particularly harmful for phonological decoding, thereby adversely affecting reading development in dyslexic children.

Cingulum Correlates of Cognitive Functions in Patients with Mild Cognitive Impairment and Early Alzheimer’s Disease: A Diffusion Spectrum Imaging Study

Diffusion spectrum imaging (DSI) of MRI can detect neural fiber tract changes. We investigated integrity of cingulum bundle (CB) in patients with mild cognitive impairment (MCI) and early Alzheimer's disease (EAD) using DSI tractography and explored its relationship with cognitive functions. We recruited 8 patients with MCI, 9 with EAD and 15 healthy controls (HC). All subjects received a battery of neuropsychological tests to access their executive, memory and language functions. We used a 3.0-tesla MRI scanner to obtain T1- and T2-weighted images for anatomy and used a pulsed gradient twice-refocused spin-echo diffusion echo-planar imaging sequence to acquire DSI. Patients with EAD performed significantly poorer than the HC on most tests in executive and memory functions. Significantly smaller general fractional anisotropy (GFA) values were found in the posterior and inferior segments of left CB and of the anterior segment of right CB of the EAD compared with those of the HC. Spearman's correlation on the patient groups showed that GFA values of the posterior segment of the left CB were significantly negatively associated with the time used to complete Color Trails Test Part II and positively correlated with performance of the logical memory and visual reproduction. GFA values of inferior segment of bilateral CB were positively associated with the performance of visual recognition. DSI tractography demonstrates significant preferential degeneration of the CB on the left side in patients with EAD. The location-specific degeneration is associated with corresponding declines in both executive and memory functions.

Effects of a daytime nap on the recognition of neutral and emotional memories

Background: Memory can be facilitated by the emotional strength of encoded information. Offline consolidation during sleep (post-learning sleep, even as little as a nap,) also modulates declarative memory processing. Previous studies have investigated the influence of sleep, especially rapid eye movement (REM) sleep, on emotional memory facilitation in humans. However, the interaction between emotional valence asymmetry and nap-dependent memory consolidation is poorly understood. Objective: To investigate the effects of post-learning nap on the recognition of neutral and emotional memories. Method: Ten healthy male participants completed a study session involving 240 emotional (negative and positive stimuli with different arousal magnitudes) and neutral pictures of people, animals, objects, and landscapes. Participants were then immediately tested on the visual recognition performance, in which they need to make recognition judgments on a subset of previously seen (old) pictures and intermixed unseen (new) pictures containing similar emotional and semantic contents. Three hours after this initial baseline test, one-half of the participants obtained a 90-minute nap opportunity, recorded with electroencephalography (EEG), whereas the others remained awake. Participants were again tested on the remaining old and new pictures at 6 hours after learning session. Results: The results revealed a beneficial effect of delayed post-learning nap on the recognition of neutral declarative memory. The extent of neutral memory facilitation was negatively correlated with the amount of stages-2 NREM. Unlike previous studies, the recognition performance for negative emotional items with high arousal condition in the Nap group deteriorated across an offline time rich in stage-2 NREM sleep. However, both groups showed similar decrease in recognition accuracy for positive stimuli. Conclusion: Our findings suggest that declarative memories containing distinct emotional valence and arousal are consolidated differentially during wakefulness and sleep. Under certain conditions, a daytime nap rich in stage-2 NREM sleep may play an important role on these performance differences. Keywords: EEG, emotion, nap, NREM, recognition memory

Perception Isn’t So Simple

Perception Isn’t So Simple

Robust Image Analysis With Sparse Representation on Quantized Visual Features

Recent techniques based on sparse representation (SR) have demonstrated promising performance in high-level visual recognition, exemplified by the highly accurate face recognition under occlusion and other sparse corruptions. Most research in this area has focused on classification algorithms using raw image pixels, and very few have been proposed to utilize the quantized visual features, such as the popular bag-of-words feature abstraction. In such cases, besides the inherent quantization errors, ambiguity associated with visual word assignment and misdetection of feature points, due to factors such as visual occlusions and noises, constitutes the major cause of dense corruptions of the quantized representation. The dense corruptions can jeopardize the decision process by distorting the patterns of the sparse reconstruction coefficients. In this paper, we aim to eliminate the corruptions and achieve robust image analysis with SR. Toward this goal, we introduce two transfer processes (ambiguity transfer and mis-detection transfer) to account for the two major sources of corruption as discussed. By reasonably assuming the rarity of the two kinds of distortion processes, we augment the original SR-based reconstruction objective with l(0) norm regularization on the transfer terms to encourage sparsity and, hence, discourage dense distortion/transfer. Computationally, we relax the nonconvex l(0) norm optimization into a convex l(1) norm optimization problem, and employ the accelerated proximal gradient method to optimize the convergence provable updating procedure. Extensive experiments on four benchmark datasets, Caltech-101, Caltech-256, Corel-5k, and CMU pose, illumination, and expression, manifest the necessity of removing the quantization corruptions and the various advantages of the proposed framework.

An abundance of riches: cross-task comparisons of semantic richness effects in visual word recognition.

There is considerable evidence (e.g., Pexman et al., 2008) that semantically rich words, which are associated with relatively more semantic information, are recognized faster across different lexical processing tasks. The present study extends this earlier work by providing the most comprehensive evaluation to date of semantic richness effects on visual word recognition performance. Specifically, using mixed effects analyses to control for the influence of correlated lexical variables, we considered the impact of number of features, number of senses, semantic neighborhood density, imageability, and body–object interaction across five visual word recognition tasks: standard lexical decision, go/no-go lexical decision, speeded pronunciation, progressive demasking, and semantic classification. Semantic richness effects could be reliably detected in all tasks of lexical processing, indicating that semantic representations, particularly their imaginal and featural aspects, play a fundamental role in visual word recognition. However, there was also evidence that the strength of certain richness effects could be flexibly and adaptively modulated by task demands, consistent with an intriguing interplay between task-specific mechanisms and differentiated semantic processing.

Medial temporal lobe-dependent repetition suppression and enhancement due to implicit vs. explicit processing of individual repeated search displays.

Using visual search, functional magnetic resonance imaging (fMRI) and patient studies have demonstrated that medial temporal lobe (MTL) structures differentiate repeated from novel displays—even when observers are unaware of display repetitions. This suggests a role for MTL in both explicit and, importantly, implicit learning of repeated sensory information (Greene et al., 2007). However, recent behavioral studies suggest, by examining visual search and recognition performance concurrently, that observers have explicit knowledge of at least some of the repeated displays (Geyer et al., 2010). The aim of the present fMRI study was thus to contribute new evidence regarding the contribution of MTL structures to explicit vs. implicit learning in visual search. It was found that MTL activation was increased for explicit and, respectively, decreased for implicit relative to baseline displays. These activation differences were most pronounced in left anterior parahippocampal cortex (aPHC), especially when observers were highly trained on the repeated displays. The data are taken to suggest that explicit and implicit memory processes are linked within MTL structures, but expressed via functionally separable mechanisms (repetition-enhancement vs. -suppression). They further show that repetition effects in visual search would have to be investigated at the display level.

Does degree of asymmetry relate to performance? A reply to Boles and Barth

In a recent critique Boles and Barth (2011) argue that their prior study investigating asymmetry/performance relationships ( Boles, Barth, & Merrill, 2008) uncovered the “true” association (i.e., negative correlation) between lateralization of visual lexical processes and word recognition performance. They contend that our study reporting positive correlations of lexical asymmetry and reading performance ( Chiarello et al., 2009) was flawed and hence inconclusive. In this response we address the two major objections raised by Boles and Barth (2011) regarding our selection of tasks and asymmetry measures. We conclude that the Boles and Barth principle of task purity is not relevant to the stated aims of our investigation, and that our linear regression method of measuring asymmetry is valid given the high level of accuracy for the tasks we reported. Because the aims of each investigation differed, we argue that it is unwise to attempt to fit each study into the framework favored by Boles and Barth (2011).

Object recognition using a bio-inspired neuron model with bottom-up and top-down pathways

In this paper, a new artificial neural network model is proposed for visual object recognition, in which the bottom-up, sensory-driven pathway and top-down, expectation-driven pathway are fused in information processing and their corresponding weights are learned based on the fused neuron activities. During the supervised learning process, the target labels are applied to update the bottom-up synaptic weights of the neural network. Meanwhile, the hypotheses generated by the bottom-up pathway produce expectations on sensory inputs through the top-down pathway. The expectations are constrained by the real data from the sensory inputs, which can be used to update the top-down synaptic weights accordingly. To further improve the visual object recognition performance, the multi-scale histograms of oriented gradients (MS-HOG) method is proposed to extract local features of visual objects from images. Extensive experiments on different image datasets demonstrate the efficiency and robustness of the proposed neural network model with features extracted using the MS-HOG method on visual object recognition compared with other state-of-the-art methods.