Conventional Content-based Image Retrieval Research Articles

Gaze-Dependent Image Re-Ranking Technique for Enhancing Content-Based Image Retrieval

Content-based image retrieval (CBIR) aims to find desired images similar to the image input by the user, and it is extensively used in the real world. Conventional CBIR methods do not consider user preferences since they only determine retrieval results by referring to the degree of resemblance or likeness between the query and potential candidate images. Because of the above reason, a “semantic gap” appears, as the model may not accurately understand the potential intention that a user has included in the query image. In this article, we propose a re-ranking method for CBIR that considers a user’s gaze trace as interactive information to help the model predict the user’s inherent attention. The proposed method uses the user’s gaze trace corresponding to the image obtained from the initial retrieval as the user’s preference information. We introduce image captioning to effectively express the relationship between images and gaze information by generating image captions based on the gaze trace. As a result, we can transform the coordinate data into a text format and explicitly express the semantic information of the images. Finally, image retrieval is performed again using the generated gaze-dependent image captions to obtain images that align more accurately with the user’s preferences or interests. The experimental results on an open image dataset with corresponding gaze traces and human-generated descriptions demonstrate the efficacy or efficiency of the proposed method. Our method considers visual information as the user’s feedback to achieve user-oriented image retrieval.

Content based word image retrieval using multi-layer perception based convolutional neural networks

Content based image retrieval (CBIR) plays the major role in real time applications likes search engines, libraries. The conventional CBIR systems are implemented by using basic image processing and machine learning models. So, they resulted in the poor performance against various situations. Therefore, this article is focused on implementation of CBIR system using multi-layer perception based convolutional neural network (MLP-CNN). Initially, principal component analysis (PCA) applied on images to perform the dimensionality reduction operation, which also extracts the content specific features. Further, MLP-CNN model is used to train the system and generates the trained features. Finally, the testing operation is performed using MLP-CNN, which generates the output as content specific images. The simulation results shows that the proposed method resulted in superior performance as compared to state of art approaches.

Region-based image retrieval in the compressed domain using shape-adaptive DCT

Content-based image retrieval (CBIR) has drawn substantial research and many traditional CBIR systems search digital images in a large database based on features, such as color, texture and shape of a given query image. A majority of images are stored in compressed format and most of compression technologies adopt different kinds of transforms to achieve compression. Therefore, features can be extracted directly from images in compressed format by using, for example, discrete cosine transform (DCT) for JPEG compressed images. Region-based image retrieval (RBIR) is an image retrieval approach which focuses on contents from regions of images, instead of the content from the entire image in early CBIR. Although RBIR approaches attempt to solve the semantic gap problem existed in global low-level features in CBIR by using local low-level features based on regions of images. This paper proposes a new RBIR approach using Shape adaptive discrete cosine transform (SA-DCT). At a bottom level, local features are constructed from the coefficients of quantized block transforms (low-level features) for each region. Quantization acts for the concentration of block-wise information in a more condense way, which is highly desirable for the retrieval tasks. At an intermediate level, histograms of local image features are used as descriptors of statistical information. Finally, at the top level, the combination of histograms from different image regions (objects) is defined as a way to incorporate high-level semantic information. In this retrieval system, an image has a prior segmentation alpha plane, which is defined exactly as in MPEG-4. Therefore, an image is represented by segmented regions, each of which is associated with a feature vector derived from DCT and SA-DCT coefficients. Users can select any region as the main theme of the query image. The similarity between a query image and any database image is ranked according to a same similarity measure computed from the selected regions between two images. For those images without distinctive objects and scenes, users can still select the whole image as the query condition. The experimental results show that the proposed approach is able to identify main objects and reduce the influence of background in the image, and thus improve the performance of image retrieval in comparison with a conventional CBIR based on DCT.

Region-based image retrieval using shape-adaptive DCT

Content-based image retrieval (CBIR) is the process of searching digital images in a large database based on features, such as color, texture and shape of a given query image. As many images are compressed by transforms, constructing the feature vector directly in transform domain is a very popular topic. Therefore, features can be extracted directly from images in compressed format by using, for example, discrete cosine transform (DCT) for JPEG compressed images. Also, region-based image retrieval (RBIR) has attracted great interest in recent years. This paper proposes a new RBIR approach using shape-adaptive discrete cosine transform (SA-DCT). In this retrieval system, an image has a prior segmentation alpha plane, which is defined exactly as in MPEG-4. Therefore, an image is represented by segmented regions, each of which is associated with a feature vector derived from DCT and SA-DCT coefficients. Users can select any region as the main theme of the query image. The similarity between a query image and any database image is ranked according to a same similarity measure computed from the selected regions between two images. For those images without distinctive objects and scenes, users can still select the whole image as the query condition. The experimental results show that the proposed approach is able to identify main objects and reduce the influence of background in the image, and thus improve the performance of image retrieval in comparison with a conventional CBIR based on DCT.

Content Based Image Retrieval Using SVM Algorithm

Conventional content-based image retrieval (CBIR) schemes employing relevance feedback may suffer from some problems in the practical applications. First, most ordinary users would like to complete their search in a single interaction especially on the web. Second, it is time consuming and difficult to label a lot of negative examples with sufficient variety. Third, ordinary users may introduce some noisy examples into the query. This correspondence explores solutions to a new issue that image retrieval using unclean positive examples. In the proposed scheme, multiple feature distances are combined to obtain image similarity using classification technology. To handle the noisy positive examples, a new two step strategy is proposed by incorporating the methods of data cleaning and noise tolerant classifier. The extensive experiments carried out on two different real image collections validate the effectiveness of the proposed scheme.

A simultaneous feature adaptation and feature selection method for content-based image retrieval systems

In content-based image retrieval (CBIR) applications, each database needs its corresponding parameter setting for feature extraction. However, most of the CBIR systems perform indexing by a set of fixed and pre-specific parameters. On the other hand, feature selection methods have currently gained considerable popularity to reduce semantic gap. In this regard, this paper is devoted to present a hybrid approach to reduce the semantic gap between low level visual features and high level semantics, through simultaneous feature adaptation and feature selection. In the proposed approach, a hybrid meta-heuristic swarm intelligence-based search technique, called mixed gravitational search algorithm (MGSA), is employed. Some feature extraction parameters (i.e. the parameters of a 6-tap parameterized orthogonal mother wavelet in texture features and quantization levels in color histogram) are optimized to reach a maximum precision of the CBIR systems. Meanwhile, feature subset selection is done for the same purpose. A comparative experimental study with the conventional CBIR system is reported on a database of 1000 images. The obtained results confirm the effectiveness of the proposed adaptive indexing method in the field of CBIR.

An alternative approach for unsupervised cluster based image retrieval

In this research, we proposed an alternative method to retrieve images from its clusters which is determined by its color histogram. In other approach, it is called cluster based image retrieval model. One common problem with conventional content based image retrieval (CBIR) system is that the output data contains too many non-relevant feedbacks due to the indistinct symmetry of semantic gap. In this paper, two clustering techniques are performed to solve the problem mentioned above; including the misconception of semantic gap, called k-means and hierarchical clustering technique. The proposed method shows a promising retrieval procedure in terms of runtime errors reduction and accuracy. The experimental results determine that this alternative image retrieval model does not only retrieve images to the homogeneity of its features but it is also practically sufficient. Key words: Content based image retrieval, color histogram, K-means, clustering image retrieval.

Generic framework for content-based stereo image/video retrieval

With the increasing number of stereo images/videos in commercial markets, the demand for content-based image retrieval (CBIR) to deal with stereo media becomes urgent. To meet this demand, a novel framework is proposed where depth cues are extracted from stereo pairs and employed in a re-ranking scheme to refine results from conventional CBIR. Experiments show the proposed method yields promising results in retrieving stereo content.

Semi-supervised distance metric learning for collaborative image retrieval and clustering

Learning a good distance metric plays a vital role in many multimedia retrieval and data mining tasks. For example, a typical content-based image retrieval (CBIR) system often relies on an effective distance metric to measure similarity between any two images. Conventional CBIR systems simply adopting Euclidean distance metric often fail to return satisfactory results mainly due to the well-known semantic gap challenge. In this article, we present a novel framework of Semi-Supervised Distance Metric Learning for learning effective distance metrics by exploring the historical relevance feedback log data of a CBIR system and utilizing unlabeled data when log data are limited and noisy. We formally formulate the learning problem into a convex optimization task and then present a new technique, named as “Laplacian Regularized Metric Learning” (LRML). Two efficient algorithms are then proposed to solve the LRML task. Further, we apply the proposed technique to two applications. One direct application is for Collaborative Image Retrieval (CIR), which aims to explore the CBIR log data for improving the retrieval performance of CBIR systems. The other application is for Collaborative Image Clustering (CIC), which aims to explore the CBIR log data for enhancing the clustering performance of image pattern clustering tasks. We conduct extensive evaluation to compare the proposed LRML method with a number of competing methods, including 2 standard metrics, 3 unsupervised metrics, and 4 supervised metrics with side information. Encouraging results validate the effectiveness of the proposed technique.

Content Based Image Retrieval Using Unclean Positive Examples

Conventional content-based image retrieval (CBIR) schemes employing relevance feedback may suffer from some problems in the practical applications. First, most ordinary users would like to complete their search in a single interaction especially on the web. Second, it is time consuming and difficult to label a lot of negative examples with sufficient variety. Third, ordinary users may introduce some noisy examples into the query. This correspondence explores solutions to a new issue that image retrieval using unclean positive examples. In the proposed scheme, multiple feature distances are combined to obtain image similarity using classification technology. To handle the noisy positive examples, a new two-step strategy is proposed by incorporating the methods of data cleaning and noise tolerant classifier. The extensive experiments carried out on two different real image collections validate the effectiveness of the proposed scheme.

Region-based image retrieval with high-level semantics using decision tree learning

Semantic-based image retrieval has attracted great interest in recent years. This paper proposes a region-based image retrieval system with high-level semantic learning. The key features of the system are: (1) it supports both query by keyword and query by region of interest. The system segments an image into different regions and extracts low-level features of each region. From these features, high-level concepts are obtained using a proposed decision tree-based learning algorithm named DT-ST. During retrieval, a set of images whose semantic concept matches the query is returned. Experiments on a standard real-world image database confirm that the proposed system significantly improves the retrieval performance, compared with a conventional content-based image retrieval system. (2) The proposed decision tree induction method DT-ST for image semantic learning is different from other decision tree induction algorithms in that it makes use of the semantic templates to discretize continuous-valued region features and avoids the difficult image feature discretization problem. Furthermore, it introduces a hybrid tree simplification method to handle the noise and tree fragmentation problems, thereby improving the classification performance of the tree. Experimental results indicate that DT-ST outperforms two well-established decision tree induction algorithms ID3 and C4.5 in image semantic learning.

A New Way for Multidimensional Medical Data Management: Volume of Interest (VOI)-Based Retrieval of Medical Images With Visual and Functional Features

The advances in digital medical imaging and storage in integrated databases are resulting in growing demands for efficient image retrieval and management. Content-based image retrieval (CBIR) refers to the retrieval of images from a database, using the visual features derived from the information in the image, and has become an attractive approach to managing large medical image archives. In conventional CBIR systems for medical images, images are often segmented into regions which are used to derive two-dimensional visual features for region-based queries. Although such approach has the advantage of including only relevant regions in the formulation of a query, medical images that are inherently multidimensional can potentially benefit from the multidimensional feature extraction which could open up new opportunities in visual feature extraction and retrieval. In this study, we present a volume of interest (VOI) based content-based retrieval of four-dimensional (three spatial and one temporal) dynamic PET images. By segmenting the images into VOIs consisting of functionally similar voxels (e.g., a tumor structure), multidimensional visual and functional features were extracted and used as region-based query features. A prototype VOI-based functional image retrieval system (VOI-FIRS) has been designed to demonstrate the proposed multidimensional feature extraction and retrieval. Experimental results show that the proposed system allows for the retrieval of related images that constitute similar visual and functional VOI features, and can find potential applications in medical data management, such as to aid in education, diagnosis, and statistical analysis.

Multiresolution descriptor matching algorithm for fast exhaustive search in norm-sorted databases

To determine the best match to a query image in a database, conventional content-based image retrieval schemes require the exhaustive search, where the descriptor of the query, e.g., the histogram, must be compared with literally all images in the database. However, the straightforward exhaustive search algorithm is computationally expensive. Thus, fast exhaustive search algorithms are required. We present a fast exhaustive search algorithm based on a multiresolution descriptor structure and a norm-sorted database. First, we derive a condition to eliminate unnecessary matching operations from the search procedure by using a norm-sorted structure of the database. Then, we propose a fast search algorithm based on the elimination condition, which guarantees an exhaustive search for either the best match or multiple best matches to a query. With a luminance histogram as a descriptor, we show that the proposed algorithm provides a search accuracy of 100% with a high search speed.

An Interactive Approach for CBIR Using a Network of Radial Basis Functions

An important requirement for constructing effective content-based image retrieval (CBIR) systems is accurate characterization of visual information. Conventional nonadaptive models, which are usually adopted for this task in simple CBIR systems, do not adequately capture all aspects of the characteristics of the human visual system. An effective way of addressing this problem is to adopt a human-computer interactive approach, where the users directly teach the system about what they regard as being significant image features and their own notions of image similarity. We propose a machine learning approach for this task, which allows users to directly modify query characteristics by specifying their attributes in the form of training examples. Specifically, we apply a radial-basis function (RBF) network for implementing an adaptive metric which progressively models the notion of image similarity through continual relevance feedback from users. Experimental results show that the proposed methods not only outperform conventional CBIR systems in terms of both accuracy and robustness, but also previously proposed interactive systems.

Re-ranking algorithm using post-retrieval clustering for content-based image retrieval

In this paper, we propose a re-ranking algorithm using post-retrieval clustering for content-based image retrieval (CBIR). In conventional CBIR systems, it is often observed that images visually dissimilar to a query image are ranked high in retrieval results. To remedy this problem, we utilize the similarity relationship of the retrieved results via post-retrieval clustering. In the first step of our method, images are retrieved using visual features such as color histogram. Next, the retrieved images are analyzed using hierarchical agglomerative clustering methods (HACM) and the rank of the results is adjusted according to the distance of a cluster from a query. In addition, we analyze the effects of clustering methods, query-cluster similarity functions, and weighting factors in the proposed method. We conducted a number of experiments using several clustering methods and cluster parameters. Experimental results show that the proposed method achieves an improvement of retrieval effectiveness of over 10% on average in the average normalized modified retrieval rank (ANMRR) measure.

Special Issue: Selected Papers from the Sixth IFIP 2.6 Working Conference on Visual Database Systems

While multimedia data, image data in particular, is an integral part of most websites and web documents, our quest for information so far is still restricted to text based search. To explore the World Wide Web more effectively, especially its rich repository of truly multimedia information, we are facing a number of challenging problems. Firstly, we face the ambiguous and highly subjective nature of defining image semantics and similarity. Secondly, multimedia data could come from highly diversified sources, as a result of automatic image capturing and generation processes. Finally, multimedia information exists in decentralised sources over the Web, making it difficult to use conventional content-based image retrieval (CBIR) techniques for effective and efficient search. In this special issue, we present a collection of five papers on visual and multimedia information management and retrieval topics, addressing some aspects of these challenges. These papers have been selected from the conference proceedings (Kluwer Academic Publishers, ISBN: 1-4020- 7060-8) of the Sixth IFIP 2.6 Working Conference on Visual Database Systems (VDB6), held in Brisbane, Australia, on 29–31 May 2002.