Feature fusion method using BoVW framework for enhancing image retrieval

Abstract

The bag-of-visual words (BoVW) has been applied to myriad of recognition problems in computer vision such as object recognition, scene classification and image retrieval due to its scalability and high precision. However, their performance is subservient in certain datasets, especially in natural image datasets, mainly due to the lack of consideration of image cues such as colour, texture etc. which are not prime features while computing invariant descriptors, on which BoVW models are generally built on. Hence, this study describes a multi-cue fusion approach for BoVW framework, exploiting both early and late fusion methods, to improve the retrieval performance, mainly in natural image datasets. For this, a composite edge and colour descriptor is proposed to describe the local regions of the image along with the invariant feature descriptor Speeded Up Robust Features (SURF). Independent vocabularies are built based on these descriptors and images in the dataset are encoded to form two histograms using the respective vocabularies. The histograms are further fused to characterise the image. The retrieval is carried out by matching the histograms. Experimental results show that significant increment in the average precision can be attained by combining the proposed descriptor with invariant descriptors.