Spatial Information Of Visual Words Research Articles

Discovering Latent Topics With Saliency-Weighted LDA for Image Scene Understanding

In the latent Dirichlet allocation (LDA) model, each image is represented by word distributions with their latent topics. Since the previous LDA-based models are not capable of dealing with the spatial information of visual words in images, this paper focuses on discovering the latent topics of images with visual saliency. To accomplish this, a saliency-weighted LDA (swLDA) model is proposed that incorporates visual saliency into the topic distribution of visual words, in a manner similar to human perception. The topic distributions of the visual words were learned with saliency weights reflecting whether the visual words were in the salient or nonsalient regions. The experimental results demonstrate that the swLDA model effectively incorporates visual saliency as a focus of attention, mimicking human perception behavior, remarkably outperforming previous LDA models in terms of image categorization.

Content based medical image retrieval using topic and location model.

Retrieval of medical images from an anatomically diverse dataset is a challenging task. Objective of our present study is to analyse the automated medical image retrieval system incorporating topic and location probabilities to enhance the performance. In this paper, we present an automated medical image retrieval system using Topic and Location Model. The topic information is generated using Guided Latent Dirichlet Allocation (GuidedLDA) method. A novel Location Model is proposed to incorporate the spatial information of visual words. We also introduce a new metric called position weighted Precision (wPrecision) to measure the rank order of the retrieved images. Experiments on two large medical image datasets - IRMA 2009 and Multimodal dataset - revealed that the proposed method outperforms existing medical image retrieval systems in terms of Precision and Mean Average Precision. The proposed method achieved better Mean Average Precision (86.74%) compared to the recent medical image retrieval systems using the Multimodal dataset with 7200 images. The proposed system achieved better Precision (97.5%) for top ten images compared to the recent medical image retrieval systems using IRMA 2009 dataset with 14,410 images. Supplementing spatial details of visual words to the Topic Model enhances the retrieval efficiency of medical images from large repositories. Such automated medical image retrieval systems can be used to assist physician to retrieve medical images with better precision compared to the state-of-the-art retrieval systems.

The Visual Word Booster: A Spatial Layout of Words Descriptor Exploiting Contour Cues.

Although researchers have made efforts to use the spatial information of visual words to obtain better image representations, none of the studies take contour cues into account. Meanwhile, it has been shown that contour cues are important to the perception of imagery in the literature. Inspired by these studies, we propose to use the Spatial Layout of Words (SLoW) to boost visual word based image descriptors by exploiting contour cues. Essentially, the SLoW descriptor utilises contours and incorporates different types of commonly used visual words, including hand-crafted basic contour elements (referred to as "contons"), textons and Scale-Invariant Feature Transform (SIFT) words, deep convolutional words and a special type of words: LBP (Local Binary Pattern) codes. Moreover, SLoW features are combined with Spatial Pyramid Matching (SPM) or Vector of Locally Aggregated Descriptors (VLAD) features. The SLoW descriptor and its combined versions are tested in different tasks. Our results show that they are superior to, or at least comparable to, their counterparts examined in this study. In particular, the joint use of the SLoW descriptor boosts the performance of the SPM and VLAD descriptors. We attribute these results to the fact that contour cues are important to human visual perception and, the SLoW descriptor captures not only local image characteristics but also the global spatial layout of these characteristics in a more perceptually consistent way than its counterparts.

A Combination of Spatial Pyramid and Inverted Index for Large-Scale Image Retrieval

Large-scale image retrieval has been shown remarkable potential in real-life applications. The standard approach is based on Inverted Indexing, given images are represented using Bag-of-Words model. However, one major limitation of both Inverted Index and Bag-of-Words presentation is that they ignore spatial information of visual words in image presentation and comparison. As a result, retrieval accuracy is decreased. In this paper, the authors investigate an approach to integrate spatial information into Inverted Index to improve accuracy while maintaining short retrieval time. Experiments conducted on several benchmark datasets (Oxford Building 5K, Oxford Building 5K+100K and Paris 6K) demonstrate the effectiveness of our proposed approach.

Coarse-grained ancient coin classification using image-based reverse side motif recognition

In this paper, we propose a novel approach for the image-based classification of ancient coins by recognizing motifs minted on their reverse sides. Reverse motif recognition is achieved with dense sampling-based bag-of-visual-words model. As the bag-of-visual-words model lacks spatial information of visual words, we evaluate three types of tiling schemes to incorporate such information. Due to the specific geometric structures of the objects in the reverse motifs, spatial information is also important for a better motif recognition rate. Furthermore, a coin can be imaged under various rotations. We also investigate the type of spatial scheme more robust to such rotations. Other parameters such as the size of visual vocabulary, resolution of the dense sampling grid, number of features collected per image to construct the visual vocabulary and the number of tilings in each tiling scheme are also investigated.

Land-Use Scene Classification Using a Concentric Circle-Structured Multiscale Bag-of-Visual-Words Model

High-resolution remote sensing image-based land-use scene classification is a difficult task, which is to recognize the semantic category of a given land-use scene image based on priori knowledge. Land-use scenes often cover multiple land-cover classes or ground objects, which makes a scene very complex and difficult to represent and recognize. To deal with this problem, this paper applies the well-known bag-of-visual-words (BOVWs) model which has been very successful in natural image scene classification. Moreover, many existing BOVW methods only use scale-invariant feature transform (SIFT) features to construct visual vocabularies, lacking in investigation of other features or feature combinations, and they are also sensitive to the rotation of image scenes. Therefore, this paper presents a concentric circle-based spatial-rotation-invariant representation strategy for describing spatial information of visual words and proposes a concentric circle-structured multiscale BOVW method using multiple features for land-use scene classification. Experiments on public land-use scene classification datasets demonstrate that the proposed method is superior to many existing BOVW methods and is very suitable to solve the land-use scene classification problem.

Visual word spatial arrangement for image retrieval and classification

We present word spatial arrangement (WSA), an approach to represent the spatial arrangement of visual words under the bag-of-visual-words model. It lies in a simple idea which encodes the relative position of visual words by splitting the image space into quadrants using each detected point as origin. WSA generates compact feature vectors and is flexible for being used for image retrieval and classification, for working with hard or soft assignment, requiring no pre/post processing for spatial verification. Experiments in the retrieval scenario show the superiority of WSA in relation to Spatial Pyramids. Experiments in the classification scenario show a reasonable compromise between those methods, with Spatial Pyramids generating larger feature vectors, while WSA provides adequate performance with much more compact features. As WSA encodes only the spatial information of visual words and not their frequency of occurrence, the results indicate the importance of such information for visual categorization.