Image annotation via social diffusion analysis with common interests

Abstract

Automatic annotation of images is of crucial importance in image retrieval and management systems. Most of the existing annotation methods rely on content-based approach to annotation, whose effectiveness is restricted due to the semantic gap between low-level features and semantic annotations, as well as the irrelevance between annotations and image content. Recently, social media analysis has been investigated for image annotation. Inspired by the abundant social diffusion records of images in online social networks, we propose a novel image annotation approach based on social diffusion analysis. We present a common-interest model to interpret social diffusion, i.e. different images have different social diffusion routes due to the preferences of users, and such preferences are represented as common interests of pairwise users rather than personalized interests. We propose an image annotation framework that consists of learning of common interests, feature extraction from social diffusion records, and automatic annotation by learning to rank. Experimental results on a real-world dataset show that our proposed approach outperforms content-based and user-preference-based annotation methods.