Semi-automatic dynamic auxiliary-tag-aided image annotation

Abstract

Image annotation is the foundation for many real-world applications. In the age of Web 2.0, image search and browsing are largely based on the tags of images. In this paper, we formulate image annotation as a multi-label learning problem, and develop a semi-automatic image annotation system. The presented system chooses proper words from a vocabulary as tags for a given image, and refines the tags with the help of the user's feedback. The refinement amounts to a novel multi-label learning framework, named Semi-Automatic Dynamic Auxiliary-Tag-Aided (SADATA), in which the classification result for one certain tag ( target tag) can be boosted by the classification results of a subset of the other tags ( auxiliary tags). The auxiliary tags, which have strong correlations with the target tag, are determined in terms of the normalized mutual information. We only select those tags whose correlations exceed a threshold as the auxiliary tags, so the auxiliary set is sparse. How much an auxiliary tag can contribute is dependent on the image, so we also build a probabilistic model conditioned on the auxiliary tag and the input image to adjust the weight of the auxiliary tag dynamically. For an given image, the user feedback on the tags corrects the outputs of the auxiliary classifiers and SADATA will recommend more proper tags next round. SADATA is evaluated on a large collection of Corel images. The experimental results validate the effectiveness of our dynamic auxiliary-tag-aided method. Furthermore, the performance also benefits from user feedbacks such that the annotation procedure can be significantly speeded up.