Semantic Scene Classification for Image Annotation and Retrieval

Abstract

We describe an annotation and retrieval framework that uses a semantic image representation by contextual modeling of images using occurrence probabilities of concepts and objects. First, images are segmented into regions using clustering of color features and line structures. Next, each image is modeled using the histogram of the types of its regions, and Bayesian classifiers are used to obtain the occurrence probabilities of concepts and objects using these histograms. Given the observation that a single class with the highest probability is not sufficient to model image content in an unconstrained data set with a large number of semantically overlapping classes, we use the concept/object probabilities as a new representation, and perform retrieval in the semantic space for further improvement of the categorization accuracy. Experiments on the TRECVID and Corel data sets show good performance.KeywordsLine SegmentContextual ModelingImage AnnotationSemantic SpaceProbabilistic Latent Semantic AnalysisThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.