Image tag refinement along the ‘what’ dimension using tag categorization and neighbor voting

Abstract

Online sharing of images is increasingly becoming popular, resulting in the availability of vast collections of user-contributed images that have been annotated with usersupplied tags. However, user-supplied tags are often not related to the actual image content, affecting the performance of multimedia applications that rely on tag-based retrieval of user-contributed images. This paper proposes a modular approach towards tag refinement, taking into account the nature of tags. First, tags are automatically categorized in five categories using WordNet: ‘where’, ‘when’, ‘who’, ‘what’, and ‘how’. Next, as a start towards a full implementation of our modular tag refinement approach, we use neighbor voting to learn the relevance of tags along the ‘what’ dimension. Our experimental results show that the proposed tag refinement technique is able to successfully differentiate correct tags from noisy tags along the ‘what’ dimension. In addition, we demonstrate that the proposed tag refinement technique is able to improve the effectiveness of image tag recommendation for non-tagged images.