Low-level Feature Representation Research Articles

Extended Spectral Regression for efficient scene recognition

This paper proposes a novel method based on Spectral Regression (SR) for efficient scene recognition. First, a new SR approach, called Extended Spectral Regression (ESR), is proposed to perform manifold learning on a huge number of data samples. Then, an efficient Bag-of-Words (BOW) based method is developed which employs ESR to encapsulate local visual features with their semantic, spatial, scale, and orientation information for scene recognition. In many applications, such as image classification and multimedia analysis, there are a huge number of low-level feature samples in a training set. It prohibits direct application of SR to perform manifold learning on such dataset. In ESR, we first group the samples into tiny clusters, and then devise an approach to reduce the size of the similarity matrix for graph learning. In this way, the subspace learning on graph Laplacian for a vast dataset is computationally feasible on a personal computer. In the ESR-based scene recognition, we first propose an enhanced low-level feature representation which combines the scale, orientation, spatial position, and local appearance of a local feature. Then, ESR is applied to embed enhanced low-level image features. The ESR-based feature embedding not only generates a low dimension feature representation but also integrates various aspects of low-level features into the compact representation. The bag-of-words is then generated from the embedded features for image classification. The comparative experiments on open benchmark datasets for scene recognition demonstrate that the proposed method outperforms baseline approaches. It is suitable for real-time applications on mobile platforms, e.g. tablets and smart phones.

Multimodal biomedical image retrieval using hierarchical classification and modality fusion

Images are frequently used in articles to convey essential information in context with correlated text. However, searching images in a task-specific way poses significant challenges. To minimize limitations of low-level feature representations in content-based image retrieval (CBIR), and to complement text-based search, we propose a multi-modal image search approach that exploits hierarchical organization of modalities and employs both intra and inter-modality fusion techniques. For the CBIR search, several visual features were extracted to represent the images. Modality-specific information was used for similarity fusion and selection of a relevant image subset. Intra-modality fusion of retrieval results was performed by searching images for specific informational elements. Our methods use text extracted from relevant components in a document to create structured representations as “enriched citations” for the text-based search approach. Finally, the multi-modal search consists of a weighted linear combination of similarity scores of independent output results from textual and visual search approaches (inter modality). Search results were evaluated using a standard ImageCLEFmed 2012 evaluation dataset of 300,000 images with associated annotations. We achieved a mean average precision (MAP) score of 0.2533, which is statistically significant, and better in performance (7 % improvement) over comparable results in ImageCLEFmed 2012.

High-level event recognition in unconstrained videos

The goal of high-level event recognition is to automatically detect complex high-level events in a given video sequence. This is a difficult task especially when videos are captured under unconstrained conditions by non-professionals. Such videos depicting complex events have limited quality control, and therefore, may include severe camera motion, poor lighting, heavy background clutter, and occlusion. However, due to the fast growing popularity of such videos, especially on the Web, solutions to this problem are in high demands and have attracted great interest from researchers. In this paper, we review current technologies for complex event recognition in unconstrained videos. While the existing solutions vary, we identify common key modules and provide detailed descriptions along with some insights for each of them, including extraction and representation of low-level features across different modalities, classification strategies, fusion techniques, etc. Publicly available benchmark datasets, performance metrics, and related research forums are also described. Finally, we discuss promising directions for future research.

Opposite Modulation of High- and Low-Level Visual Aftereffects by Perceptual Grouping

A fundamental task of visual perception is to group visual features - sometimes spatially separated and partially occluded - into coherent, unified representations of objects. Perceptual grouping can vastly simplify the description of a visual scene and is critical for our visual system to understand the three-dimensional visual world. Numerous neurophysiological and brain imaging studies have demonstrated that neural mechanisms of perceptual grouping are characterized by the enhancement of neural responses throughout the visual processing hierarchy, from lower visual areas processing grouped features to higher visual areas representing objects and shapes from grouping. In a series of psychophysical adaptation experiments, we made the counterintuitive observation that perceptual grouping amplified the shape aftereffect but meanwhile, reduced the tilt aftereffect and the threshold elevation aftereffect (TEAE). Furthermore, the modulation of perceptual grouping on the TEAE showed a partial interocular transfer. This finding suggests a 2-fold effect of perceptual grouping - enhancing the high-level shape representation and attenuating the low-level feature representation even at a monocular level. We propose that this effect is a functional manifestation of a predictive coding scheme and reflects an efficient code of visual information across lower and higher visual cortical areas.

ModelSeek: an effective 3D model retrieval system

Since object similarity is a subjective matter, the gap between low-level feature representations and high-level semantic concepts is a major problem in the field of content-based 3D model retrieval. This paper presents a novel composite model descriptor, which takes into account both visual and geometric characteristics of 3D models. It also proposes an original mapping mechanism from low-level model features to high-level semantic concepts based on the user's retrieval history, and so this method belongs to long-term relevance feedback algorithms for 3D model retrieval. Finally, an effective 3D model retrieval system "ModelSeek" has been built, and implemented with the introduced model descriptor and mapping mechanism. The experimental results show that the approaches above not only have significantly improved the retrieval performance, but have also achieved better retrieval effectiveness than the state-of-the-art techniques on the publicly available 3D model criterion that Princeton Shape Benchmark (PSB) and several standard evaluation measures.

Mechanisms of priming of pop-out: Stored representations or feature-gain modulations?

Previous research has shown that repetition of a task-relevant attention-capturing feature facilitates pop-out search. This priming of pop-out effect is due to some residual memory from recent trials. We explore two possible mechanisms of priming of pop-out: a top-down attentional benefit from a memory of the previous target representation that is stored in visual short-term memory (VSTM) and a bottom-up change of attentional gains from perceptual features of the previously attended target. We manipulated participants' ability to form a memory trace in VSTM by occupying it with a distractor task and found that occupying VSTM did not interfere with priming of pop-out. We next manipulated attentional gains associated with feature values by inserting an irrelevant task between pop-out searches. We found that the color of the target from the intervening perceptual task influenced pop-out search: The current pop-out search was facilitated when the intervening task's target matched the target color of the pop-out search. These results suggest that priming of pop-out might not be due to a memory trace of the previous targets in VSTM but, rather, might be due to changes in attentional control based on priming from relatively low-level feature representations of previously attended objects.

A new region filtering and region weighting approach to relevance feedback in content-based image retrieval

A new region filtering and region weighting method, which filters out unnecessary regions from images and learns region importance from the region size and the spatial location of regions in an image, is proposed based on region representations. It weights the regions optimally and improves the performance of the region-based retrieval system based on relevance feedback. Due to the semantic gap between the low level feature representation and the high level concept in a query image, semantically relevant images may exhibit very different visual characteristics, and may be scattered in several clusters in the feature space. Our main goal is finding semantically related clusters and their weights to reduce this semantic gap. Experimental results demonstrate the efficiency and effectiveness of the proposed region filtering and weighting method in comparison with the area percentage method and region frequency weighted by inverse image frequency method, respectively.

Interactive retrieval of video using pre-computed shot-shot similarities

A probabilistic framework for content-based interactive video retrieval is described. The developed indexing of video fragments originates from the probability of the user's positive judgment about key-frames of video shots. Initial estimates of the probabilities are obtained from low-level feature representation. Only statistically significant estimates are picked out, the rest are replaced by an appropriate constant allowing efficient access at search time without loss of search quality and leading to improvement in most experiments. With time, these probability estimates are updated from the relevance judgment of users performing searches, resulting in further substantial increases in mean average precision.

EID: a system for exploration of image databases

The goal of this paper is to describe an exploration system for large image databases in order to help the user understand the database as a whole, discover hidden relationships, and formulate insights with minimum effort. While the proposed system works with any type of low-level feature representation of images, we describe our system using color information. The system is built in three stages: the feature extraction stage in which images are represented in a way that allows efficient storage and retrieval results closer to the human perception; the second stage consists of building a hierarchy of clusters in which the cluster prototype, as the electronic identification ( eID) of that cluster, is designed to summarize the cluster in a manner that is suited for quick human comprehension of its components. In a formal definition, an electronic IDentification ( eID) is the most similar image to the other images from a corresponding cluster; that is, the image in the cluster that maximizes the sum of the squares of the similarity values to the other images of that cluster. Besides summarizing the image database to a certain level of detail, an eID image will be a way to provide access either to another set of eID images on a lower level of the hierarchy or to a group of perceptually similar images with itself. As a third stage, the multi-dimensional scaling technique is used to provide us with a tool for the visualization of the database at different levels of details. Moreover, it gives the capability for semi-automatic annotation in the sense that the image database is organized in such a way that perceptual similar images are grouped together to form perceptual contexts. As a result, instead of trying to give all possible meanings to an image, the user will interpret and annotate an image in the context in which that image appears, thus dramatically reducing the time taken to annotate large collection of images.

Image retrieval using hierarchical self-organizing feature maps

This paper presents a scheme for image retrieval that lets a user retrieve images either by exploring summary views of the image collection at different levels or by similarity retrieval using query images. The proposed scheme is based on image clustering through a hierarchy of self-organizing feature maps. While the suggested scheme can work with any kind of low-level feature representation of images, our implementation and description of the system is centered on the use of image color information. Experimental results using a database of 2100 images are presented to show the efficacy of the suggested scheme.