From Bag-of-Visual-Words to Bag-of-Visual-Phrases Using n-Grams

Abstract

The Bag-of-Visual-Words has emerged as an effective modeling approach to represent local image features. It describes local image features by assigning them a visual word according to a visual dictionary. The image representation is given by the frequency of each visual word in the image, as a similar representation used in textual documents. In this paper, we present a novel approach building a high-level description using a group of words (phrases) for representing an image. We introduce the use of n-grams for image representation, based on the idea of "Bag-of-Visual-Phrases". In the field of computational linguistics, an n-gram is a phrase formed by a sequence of n-consecutive words. As analogy, we represent an image by a combination of n-consecutive visual words. We made representative experiments using three public benchmark databases of textures and nature scenes and two medical databases to demonstrate an area that can benefit from the proposed technique. Our proposed Bag-of-Visual-Phrases approach improved up to 44% the retrieval precision and up to 33% the classification rate compared to the traditional Bag-of-Visual-Words, being a valuable asset for content-based image retrieval and image classification.