Visual Vocabulary Construction Research Articles

Loop Closure Detection Via Maximization of Mutual Information

An image can be described in terms of appearance frequency of visual words. This representation is implemented in bag-of-visual-words (BoVW)-based loop closure detection for its efficiency and effectiveness. However, traditional BoVW-based approaches are strongly affected by false positive loops due to scene ambiguity caused by redundant words in the vocabulary and fail to detect bidirectional loops in monocular mode. Aiming at overcoming these problems, we propose a novel vocabulary construction algorithm named hierarchical sequential information bottleneck (HsIB) by leveraging the maximization of mutual information (MMI) mechanism. First, feature descriptors are extracted from training images for visual vocabulary construction. Second, HsIB extracts discriminative yet informative visual words through the MMI mechanism in vocabulary construction, which treats feature descriptors clustering as a process of data compression. Finally, the clustering process reaches a tradeoff between compactness and discrimination and improves the performance of traditional BoVW-based loop closure detection. The proposed method is compared with state-of-the-art methods on publicly available datasets. We also create a challenging dataset to further evaluate the performance of HsIB on bidirectional loops. To the best of our knowledge, we are the first to implement information bottleneck (IB) method in visual-SLAM (vSLAM) loop closure detection, and we obtain impressive results.

Semantic-enriched visual vocabulary construction in a weakly supervised context

One of the prevalent learning tasks involving images is content-based image classification. This is a difficult task especially because the low-level features used to digitally describe images usually capture little information about the semantics of the images. In this paper, we tackle this diffic ulty by enriching the semantic content of the image representation by using external knowledge. The underlying hypothesis of our work is that creating a more semantically rich representation for images would yield higher machine learning performances, without the need to modify the learning algorithms themselves. The external semantic information is presented under the form of non-positional image labels, therefore positioning our work in a weakly supervised context. Two approaches are proposed: the first one leverages the labels into the visual vocabulary construction algorithm, the result being dedicated visual vocabularies. The second approach adds a filtering phase as a pre-processing of the vocabulary construction. Known positive and known negative sets are constructed and features that are unlikely to be associated with the objects denoted by the labels are filtered. We apply our proposition to the task of content-based image classification and we show that semantically enriching the image representation yields higher classification performances than the baseline representation.

MDPV: metric distance permutation vocabulary

Sub-image content-based similarity search forms an important operation in current image archives since it provides users with images that contain a query image as their part. Such a search can conveniently be implemented using the bag-of-features model. Its integral part is a construction of visual vocabulary. Most existing algorithms to create a visual vocabulary suffer from high computational (e.g. k-means) or supervisor-guidance (e.g. visual-bit classifier, or sparse coding) requirements. In this paper, we propose a novel approach to visual vocabulary construction called metric distance permutation vocabulary. It is based on permutations of metric distances to create compact visual words. Its major advantage over prior techniques is time and space efficiency of vocabulary construction and quantization process during querying, while achieving comparable or even better effectiveness (query result quality). Moreover, this basic concept is extended to combine more independent permutations. Both the proposals are experimented on well-known real-world data-sets and compared to other state-of-the-art techniques.

Heterogeneous Visual Codebook Integration Via Consensus Clustering for Visual Categorization

Most recent category-level object and activity recognition systems work with visual words, i.e., vector-quantized local descriptors. These visual vocabularies are usually built by using a local feature, such as SIFT, and a single clustering algorithm, such as K-means. However, very different clusterings algorithms are at our disposal, each of them discovering different structures in the data. In this paper, we explore how to combine these heterogeneous codebooks and introduce a novel approach for their integration via consensus clustering. Considering each visual vocabulary as one modal, we propose the visual word aggregation (VWA) methodology, to learn a common codebook, where the stability of the visual vocabulary construction process is increased, the size of the codebook is determined in an unsupervised integration, and more discriminative representations are obtained. With the aim of obtaining contextual visual words, we also incorporate the spatial neighboring relation between the local descriptors into the VWA process: the contextual-VWA approach. We integrate over-segmentation algorithms and spatial grids into the aggregation process to obtain a visual vocabulary that narrows the semantic gap between visual words and visual concepts. We show how the proposed codebooks perform in recognizing objects and scenes on very challenging datasets. Compared with unimodal visual codebook construction approaches, our multimodal approach always achieves superior performances.