Querying Users as Oracles in Tag Engines for Personalized Image Tagging

Abstract

A common trend in image-tagging research is to focus on visually relevant tags; however, this tends to ignore the personal and social aspects of tags, especially on photo blogging websites such as Flickr. Previous work has correctly identified that many of the tags that users provide on images are not visually relevant (that is, representative of the salient content in the image) and they go on to treat such tags as noise, ignoring the fact that the users chose to provide those tags over others that could have been more visually relevant. Another common assumption about user-generated tags for images is that the order of these tags provides no useful information for the prediction of tags on future images. This assumption also tends to define usefulness in terms of what is visually relevant to the image. For general tagging or labeling applications that focus on providing visual information about image content, these assumptions are reasonable, but when considering personalized image-tagging applications, these assumptions are at best too rigid, ignoring user choice and preferences. This article challenges these assumptions and provides a machine learning approach to the problem of personalized image tagging. The authors reformulate the personalized image-tagging problem as a search/retrieval ranking problem and leverage the order of tags provided by the user in the past as a cue to tag preferences, similar to click data. They propose a technique to augment sparse user tag data (semisupervision) and demonstrate the efficacy of this method on a subset of Flickr images.