Image Annotation by a Hierarchical and Iterative Combination of Recognition and Segmentation

Abstract

Automatic image annotation and image segmentation are two prominent research fields of Computer Vision, that are getting higher attention these days to accomplish image analysis and scene understanding. In this work, we present an annotation algorithm based on a hierarchical image partition, that makes use of Markov Random Fields (MRFs) to model spatial and hierarchical relations among regions in the image. In this way, we can capture local, global and contextual information. Also, we combine the processes of annotation and segmentation in an iterative way so that each process benefits from the other. Furthermore, we investigate the selection of the starting segmentation level for the hierarchical annotation process, to show its relevance for the final results. We experimentally validate our approach in three well-known datasets: CorelA, Stanford Background and MSRC-21 datasets. In these datasets, we achieved better or comparable results to other state-of-the-art algorithms, improving our base classifier in all cases.