MVDF-RSC: Multi-view data fusion via robust spectral clustering for geo-tagged image tagging

Abstract

Image tag recommendation, aiming at assigning a set of relevant tags for images, is a useful way to help users organize images’ content. Early methods in image tagging mainly demonstrated using low-level visual features. However, two visually similar photos may have different concepts (semantic gap). Although different multi-view tagging methods are proposed to learn the discriminative features, they usually do not consider the geographical correlation among images. Moreover, geographical-based image tagging models generally focused on the relevance criterion, i.e., how well the suggested tags describe image content. Diversity and redundancy should be controlled to guarantee the recommendation models’ effectiveness and promote complementary information among tags. This paper proposes a robust multi-view image tagging method, termed MVDF-RSC, which considers the relevance, diversity, and redundancy criteria. Precisely, the proposed method consists of two phases: training and prediction. We propose a new robust optimization problem in the training phase to determine the similarity between data via the early fusion of multiple views of images and obtain clusters. In the prediction phase, relevant tags are recommended to each test data using a search-based method and a late fusion strategy. Comprehensive experiments on two geo-tagged image datasets demonstrate the proposed method’s effectiveness over state-of-the-art alternatives.