Abstract
Images are complex multimedia data that contain rich semantic information. Currently, most of image annotation algorithms are only annotating the object semantics of images. There are still many challenges on high-level semantic image annotation. The major issues are the lack of effective modeling method for the high-level semantics of images and the lack of efficient dynamic update mechanism for the training set. To address these issues, we propose a high-level semantic annotation method based on hot Internet topics in this paper. There are two independent sub tasks in our method: dynamic update of the training set based on hot Internet topics and search-based image annotation. In the first sub task, we propose to model the abstract semantics of images based on three relationships: image---to---image similarity relationship, topic---to---topic co-occurrence relationship, and image---to---topic relevance relationship. Through the complex graph clustering, the hot Internet topics are extracted for images with consistent visual and semantic contents. Then the dynamic update mechanism will update the original training set with the new topics and images. It avoids the huge computing cost in traditional update methods and does not need to re-calculate the whole mapping relationship between the semantic concepts and visual features. In the second sub task, given a query image, it first searches for similar candidates in the annotated training set via visual features. Then the hypergraph modeling and spectral clustering are exploited to filter out the images with irrelevant semantics. The keywords will be extracted for annotation from the remaining images according to an annotation probability. Extensive experiments have been conducted and the results demonstrate that our algorithm could achieve better annotation performance than the state-of-the-art algorithms. And the update mechanism could extend the training set efficiently so that the coverage of the semantics in the training set wouldn't be obsolete.
Highlights
The number of images on the Internet has been growing explosively with the widespread use of smart phones, digital cameras, and other portable devices
It includes the dynamic update of the training set based on hot Internet topics and search-based image annotation
Effectiveness evaluation of the dynamic update of the training set based on hot Internet topics
Summary
The number of images on the Internet has been growing explosively with the widespread use of smart phones, digital cameras, and other portable devices. Current approaches adopt machine learning techniques or other relevance modeling methods to learn from a static annotated training set and identify the keywords for new images based on the learned model [14] This process can be viewed as mapping low-level features of images to highlevel semantic concepts. Without an effective updating mechanism, the training set only provides a fixed vocabulary for annotation and cannot grow to cover newly formed semantics (e.g. new events, temporal hot topics, etc.). We propose a dynamic update mechanism for the training set based on hot Internet topics. Through the discovery and tracking of hot Internet topics, representative keywords are selected to annotate related images in the training set. We propose a new search-based image annotation mechanism, which uses the hypergraph modeling and spectral clustering to filter out the semantically irrelevant images.
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
