Semantic-based Image Retrieval Research Articles

Combining Low-Level Features for Semantic Extraction in Image Retrieval

An object-oriented approach for semantic-based image retrieval is presented. The goal is to identify key patterns of specific objects in the training data and to use them as object signature. Two important aspects of semantic-based image retrieval are considered: retrieval of images containing a given semantic concept and fusion of different low-level features. The proposed approach splits the image into elementary image blocks to obtain block regions close in shape to the objects of interest. A multiobjective optimization technique is used to find a suitable multidescriptor space in which several low-level image primitives can be fused. The visual primitives are combined according to a concept-specific metric, which is learned from representative blocks or training data. The optimal linear combination of single descriptor metrics is estimated by applying the Pareto archived evolution strategy. An empirical assessment of the proposed technique was conducted to validate its performance with natural images.

Semantic-associative visual content labelling and retrieval: A multimodal approach

A novel framework referred to as collaterally confirmed labelling (CCL) is proposed, aiming at localising the visual semantics to regions of interest in images with textual keywords. Both the primary image and collateral textual modalities are exploited in a mutually co-referencing and complementary fashion. The collateral content and context-based knowledge is used to bias the mapping from the low-level region-based visual primitives to the high-level visual concepts defined in a visual vocabulary. We introduce the notion of collateral context, which is represented as a co-occurrence matrix of the visual keywords. A collaborative mapping scheme is devised using statistical methods like Gaussian distribution or Euclidean distance together with collateral content and context-driven inference mechanism. We introduce a novel high-level visual content descriptor that is devised for performing semantic-based image classification and retrieval. The proposed image feature vector model is fundamentally underpinned by the CCL framework. Two different high-level image feature vector models are developed based on the CCL labelling of results for the purposes of image data clustering and retrieval, respectively. A subset of the Corel image collection has been used for evaluating our proposed method. The experimental results to-date already indicate that the proposed semantic-based visual content descriptors outperform both traditional visual and textual image feature models.

Image semantics discovery from web pages for semantic-based image retrieval using self-organizing maps

Traditional content-based image retrieval (CBIR) systems often fail to meet a user’s need due to the ‘semantic gap’ between the extracted features of the systems and the user’s query. The cause of the semantic gap is the failure of extracting real semantics from an image and the query. To extract semantics of images, however, is a difficult task. Most existing techniques apply some predefined semantic categories and assign the images to appropriate categories through some learning processes. Nevertheless, these techniques always need human intervention and rely on content-based features. In this paper we propose a novel approach to bridge the semantic gap which is the major deficiency of CBIR systems. We conquer the deficiency by extracting semantics of an image from the environmental texts around it. Since an image generally co-exists with accompanying texts in various formats, we may rely on such environmental texts to discover the semantics of the image. We apply a text mining process, which adopts the self-organizing map (SOM) learning algorithm as a kernel, on the environmental texts of an image to extract the semantic information from this image. Some implicit semantic information of the images can be discovered after the text mining process. We also define a semantic relevance measure to achieve the semantic-based image retrieval task. We performed experiments on a set of images which are collected from web pages and obtained promising results.

A survey of content-based image retrieval with high-level semantics

In order to improve the retrieval accuracy of content-based image retrieval systems, research focus has been shifted from designing sophisticated low-level feature extraction algorithms to reducing the ‘semantic gap’ between the visual features and the richness of human semantics. This paper attempts to provide a comprehensive survey of the recent technical achievements in high-level semantic-based image retrieval. Major recent publications are included in this survey covering different aspects of the research in this area, including low-level image feature extraction, similarity measurement, and deriving high-level semantic features. We identify five major categories of the state-of-the-art techniques in narrowing down the ‘semantic gap’: (1) using object ontology to define high-level concepts; (2) using machine learning methods to associate low-level features with query concepts; (3) using relevance feedback to learn users’ intention; (4) generating semantic template to support high-level image retrieval; (5) fusing the evidences from HTML text and the visual content of images for WWW image retrieval. In addition, some other related issues such as image test bed and retrieval performance evaluation are also discussed. Finally, based on existing technology and the demand from real-world applications, a few promising future research directions are suggested.

Retrieve images by understanding semantic links and clustering image fragments

The main obstacle to realize real semantic-based image retrieval is that semantic description of versatile images is difficult. The basic ideas of this paper are that the semantics of an object can be refined through top–down orthogonal semantic classification and that the semantics of an object can be reflected by the semantics of relevant objects and the semantic relationships between them. To reflect the semantic relationships between images, we propose a set of primitive semantic links as the enhancement of the hyperlinks connecting Web pages. The semantic link network is the natural extension of the hyperlink network so it can inherit the existing theory and method on hyperlink network. Based on the single semantic image established upon the orthogonal semantic space and the semantic link space, the proposed image retrieval approach enables users to obtain the semantic clustering of relevant images rather than a list of isolated images as the output of the current search engine and to browse images along semantic paths with the support of semantic link reasoning. Semantic matching and reasoning for realizing intelligent semantic image retrieval is based on graph operation and matrix operation.

Analyzing scenery images by monotonic tree

Content-based image retrieval (CBIR) has been an active research area in the last ten years, and a variety of techniques have been developed. However, retrieving images on the basis of low-level features has proven unsatisfactory, and new techniques are needed to support high-level queries. Research efforts are needed to bridge the gap between high-level semantics and low-level features. In this paper, we present a novel approach to support semantics-based image retrieval. Our approach is based on the monotonic tree, a derivation of the contour tree for use with discrete data. The structural elements of an image are modeled as branches (or subtrees) of the monotonic tree. These structural elements are classified and clustered on the basis of such properties as color, spatial location, harshness and shape. Each cluster corresponds to some semantic feature. This scheme is applied to the analysis and retrieval of scenery images. Comparisons of experimental results of this approach with conventional techniques using low-level features demonstrate the effectiveness of our approach.

Color restoration and image retrieval for Dunhuang fresco preservation

Dunhuang Mogao Cave, located in Gansu province, is a typical Chinese heritage site. Well preserved, the caves contain more than 555 grottoes, 2,000 painted statues, and 5,000 square meters of wall paintings. Scientists and artists have worked for a long time on how to preserve and restore the murals' colors, but have not found satisfactory methods. Discovering how to use advanced computer techniques to preserve, restore, and reuse Dunhuang fresco art information is a very interesting and complex task. In our project, we integrate many techniques, such as image processing, information retrieval, knowledge representation, and intelligent reasoning, to deal with these problems. Restoration of the frescoes is the basic focus of our project. Because of human activity, most of the frescoes fade day-by-day, with some seriously damaged. We propose a new, intelligent approach to color restoration. Each restored fresco will be preserved in the fresco database along with the original. For many of the Dunhuang frescoes, the key to preservation is to find an effective method for organizing, storing, managing, and retrieving the fresco data. We adopt content- and semantics-based image retrieval methods to solve these problems. Our fresco database will support Dunhuang pattern creation and virtual cave travel, reusing the fresco art information from the database. In this article, we mainly focus on the color restoration and retrieval techniques we use for fresco preservation.