Simple tag-based subclass representations for visually-varied image classes

Abstract

In this paper, we present a subclass-representation approach that predicts the probability of a social image belonging to one particular class. We explore the co-occurrence of user-contributed tags to find subclasses with a strong connection to the top level class. We then project each image onto the resulting subclass space, generating a subclass representation for the image. The advantage of our tag-based subclasses is that they have a chance of being more visually stable and easier to model than top-level classes. Our contribution is to demonstrate that a simple and inexpensive method for generating sub-class representations has the ability to improve classification results in the case of tag classes that are visually highly heterogenous. The approach is evaluated on a set of 1 million photos with 10 top-level classes, from the dataset released by the ACM Multimedia 2013 Yahoo! Large-scale Flickr-tag Image Classification Grand Challenge. Experiments show that the proposed system delivers sound performance for visually diverse classes compared with methods that directly model top classes.