Learning image semantics with latent aspect model

Abstract

Automatic image annotation has become an important and challenging problem due to the existence of semantic gap. In this paper, we present an approach based on probabilistic latent semantic analysis (PLSA) to accomplish the tasks of semantic image annotation and retrieval. In order to model training images precisely, we employ two PLSA models to capture semantic information from visual and textual modalities respectively. Then an adaptive asymmetric learning approach is proposed to fuse aspects which are learned from both modalities. For each image document, the weight of each modality is determined by its contribution to the content of the image. Consequently, the two models are linked with the same distribution over aspects. This structure can predict semantic annotation for an unseen image because it associates visual and textual modalities properly. Finally, we compare our approach with several previous approaches on a standard Corel dataset. The experiment results show that our approach performs more effective and accurate.