Fuzzy bag of words for social image description

Abstract

Rapid growth of social media resources brings huge challenges and opportunities for image description technologies. The performance of image description method directly affects the accuracy of image retrieval, image annotation and image recognition. Bag of Words (BoW) as an efficient approach to describing the images has been attracting more and more attention. However, in traditional BoW, the maps between the words in the codebook and the features extracted from the images are actually ambiguous. As the Fuzzy Sets Theory (FST) is a powerful means for dealing with uncertainty efficiently, we utilize the FST to solve the problem caused by the ambiguity between the features and words. Accordingly, we propose a new type of BoW named as FBoW to describe images based on FST. Firstly, the features are extracted from the images. Secondly, k-means is utilized to learn the codebook. Thirdly, a fuzzy membership function is designed to measure the similarity between the features and words. The optimal parameters of the fuzzy membership function are obtained by using a Genetic Algorithm (GA). The histogram is generated by adding up the fuzzy membership values of each word to describe the images. The experimental results show that the proposed FBoW outperforms traditional BoW for social image description.