Search-based Image Annotation Research Articles

ConceptRank for search-based image annotation

Multimedia information is becoming an ubiquitous part of our lives, which brings an equally ubiquitous need for efficient multimedia retrieval. One of the possible solutions to this problem is to attach text descriptions to multimedia data objects, thus allowing users to utilize traditional text search mechanisms. Search-based annotation techniques attempt to determine the descriptive keywords by analyzing the descriptions of similar, already annotated multimedia objects, which are detected by content-based retrieval techniques. One of the main challenges of this approach is the extraction of semantically connected keywords from the possibly noisy descriptions of similar objects. In this paper, we address this challenge by proposing the ConceptRank, a new keyword ranking algorithm that exploits semantic relationships between candidate keywords and utilizes the random walk mechanism to compute the probability of individual candidates. The effectiveness of the ConceptRank algorithm is evaluated in context of web image annotation. We present a complex image annotation system that includes the ConceptRank component, and compare it to other state-of-the–art annotation techniques.

High-level semantic image annotation based on hot Internet topics

Images are complex multimedia data that contain rich semantic information. Currently, most of image annotation algorithms are only annotating the object semantics of images. There are still many challenges on high-level semantic image annotation. The major issues are the lack of effective modeling method for the high-level semantics of images and the lack of efficient dynamic update mechanism for the training set. To address these issues, we propose a high-level semantic annotation method based on hot Internet topics in this paper. There are two independent sub tasks in our method: dynamic update of the training set based on hot Internet topics and search-based image annotation. In the first sub task, we propose to model the abstract semantics of images based on three relationships: image---to---image similarity relationship, topic---to---topic co-occurrence relationship, and image---to---topic relevance relationship. Through the complex graph clustering, the hot Internet topics are extracted for images with consistent visual and semantic contents. Then the dynamic update mechanism will update the original training set with the new topics and images. It avoids the huge computing cost in traditional update methods and does not need to re-calculate the whole mapping relationship between the semantic concepts and visual features. In the second sub task, given a query image, it first searches for similar candidates in the annotated training set via visual features. Then the hypergraph modeling and spectral clustering are exploited to filter out the images with irrelevant semantics. The keywords will be extracted for annotation from the remaining images according to an annotation probability. Extensive experiments have been conducted and the results demonstrate that our algorithm could achieve better annotation performance than the state-of-the-art algorithms. And the update mechanism could extend the training set efficiently so that the coverage of the semantics in the training set wouldn't be obsolete.

Duplicate-Search-Based Image Annotation Using Web-Scale Data

Easy photo-taking and photo-sharing today make image an increasingly important type of media in people's everyday life, which arouses a growing demand for a practical image understanding technique. Traditional computer vision or machine learning methods which learn models based on a set of training data are still in the stage of tackling hundreds of object categories. Such a scale is far from practical usage. In recent years, the technique of search-based image annotation on a large-scale data set has demonstrated great success. Rather than directly mapping visual features to texts which is inevitably hindered by the semantic gap, it understands the content of an image by propagating labels of its similar images in a large-scale data set. Since similarity search is performed among homogenous data, the difficulty is greatly reduced. This paper summarizes the extensive work on web image annotation using the large-scale metadata and social information available on the Web, and introduces the Arista system, which is a nonparametric image annotation platform built upon two billion web images. We propose a highly efficient and scalable duplicate-search technique so that the Arista system can be deployed on a few servers. A few interesting applications such as building large-scale celebrity face database and text-to-image translation are also presented in this paper.

Scalable search-based image annotation

With the popularity of digital cameras, more and more people have accumulated considerable digital images on their personal devices. As a result, there are increasing needs to effectively search these personal images. Automatic image annotation may serve the goal, for the annotated keywords could facilitate the search processes. Although many image annotation methods have been proposed in recent years, their effectiveness on arbitrary personal images is constrained by their limited scalability, i.e. limited lexicon of small-scale training set. To be scalable, we propose a search-based image annotation algorithm that is analogous to information retrieval. First, content-based image retrieval technology is used to retrieve a set of visually similar images from a large-scale Web image set. Second, a text-based keyword search technique is used to obtain a ranked list of candidate annotations for each retrieved image. Third, a fusion algorithm is used to combine the ranked lists into a final candidate annotation list. Finally, the candidate annotations are re-ranked using Random Walk with Restarts and only the top ones are reserved as the final annotations. The application of both efficient search techniques and Web-scale image set guarantees the scalability of the proposed algorithm. Moreover, we provide an annotation rejection scheme to point out the images that our annotation system cannot handle well. Experimental results on U. Washington dataset show not only the effectiveness and efficiency of the proposed algorithm but also the advantage of image retrieval using annotation results over that using visual features.