Image Retrieval Framework Research Articles

Image retrieval framework based on texton uniform descriptor and modified manifold ranking

Image representation and ranking are crucial parts in image retrieval. These two steps are independently constructed in most retrieval models, but the compatibility between descriptors and ranking algorithms play an important role. Inspired by human vision perception and manifold learning, we propose a novel image retrieval framework in this paper. We first propose an image representation called texton uniform descriptor, and then illustrate the preservation of the intrinsic manifold structure through visualizing the distribution of image representations on the two-dimensional manifold. This characteristic provides the foundation for subsequent manifold-based ranking. To further improve the efficiency in image retrieval, we propose modified manifold ranking (MMR) which aims at selecting small-scale images randomly as landmarks to propagate adjacent similarity among images iteratively. The extensive experiments in four public datasets demonstrate that our framework has better performance than other state-of-the-art methods in image retrieval.

Image Retrieval Framework for Medical Images using Improved SVM

Image Retrieval Framework for Medical Images using Improved SVM

User specific context construction for personalized multimedia retrieval

The onset of Web 2.0 has given the freedom of tagging to the users. The popularization of social networking and the expansion of the smartphone market in the past decade has led to an increase of data being accumulated on the social media platforms, particularly images and videos. The exponential and ever increasing data have made information retrieval cumbersome, especially for the social network users, and this has turned out to be a huge challenge in the evolution of algorithms and technologies. In this paper, we present a novel framework and techniques for retrieving user’s multimedia content like images from the user’s profile using the context of the image/media file. We apply the Logical Itemset mining on the image Metadata consisting of the textual data (Hashtags, Caption, Date and Time) associated with the images. Through this work, we intend to bridge the semantic gap between the images and the data representation that the user associated with them. Our framework also addresses the paraphrase problem of variation in words (synonyms) used to describe a context of a media file. To evaluate the applicability of our framework, we performed tests on large Instagram image dataset extracted from various user profiles containing monolingual metadata, which show promising results for real-time applications. Furthermore, we compare and evaluate our framework with another context-based image retrieval framework, Krumbs.

An image retrieval framework for real-time endoscopic image retargeting

PurposeSerial endoscopic examinations of a patient are important for early diagnosis of malignancies in the gastrointestinal tract. However, retargeting for optical biopsy is challenging due to extensive tissue variations between examinations, requiring the method to be tolerant to these changes whilst enabling real-time retargeting.MethodThis work presents an image retrieval framework for inter-examination retargeting. We propose both a novel image descriptor tolerant of long-term tissue changes and a novel descriptor matching method in real time. The descriptor is based on histograms generated from regional intensity comparisons over multiple scales, offering stability over long-term appearance changes at the higher levels, whilst remaining discriminative at the lower levels. The matching method then learns a hashing function using random forests, to compress the string and allow for fast image comparison by a simple Hamming distance metric.ResultsA dataset that contains 13 in vivo gastrointestinal videos was collected from six patients, representing serial examinations of each patient, which includes videos captured with significant time intervals. Precision-recall for retargeting shows that our new descriptor outperforms a number of alternative descriptors, whilst our hashing method outperforms a number of alternative hashing approaches.ConclusionWe have proposed a novel framework for optical biopsy in serial endoscopic examinations. A new descriptor, combined with a novel hashing method, achieves state-of-the-art retargeting, with validation on in vivo videos from six patients. Real-time performance also allows for practical integration without disturbing the existing clinical workflow.

Quantization-based hashing: a general framework for scalable image and video retrieval

Nowadays, due to the exponential growth of user generated images and videos, there is an increasing interest in learning-based hashing methods. In computer vision, the hash functions are learned in such a way that the hash codes can preserve essential properties of the original space (or label information). Then the Hamming distance of the hash codes can approximate the data similarity. On the other hand, vector quantization methods quantize the data into different clusters based on the criteria of minimal quantization error, and then perform the search using look-up tables. While hashing methods using Hamming distance can achieve faster search speed, their accuracy is often outperformed by quantization methods with the same code length, due to the low quantization error and more flexible distance lookups. To improve the effectiveness of the hashing methods, in this work, we propose Quantization-based Hashing (QBH), a general framework which incorporates the advantages of quantization error reduction methods into conventional property preserving hashing methods. The learned hash codes simultaneously preserve the properties in the original space and reduce the quantization error, and thus can achieve better performance. Furthermore, the hash functions and a quantizer can be jointly learned and iteratively updated in a unified framework, which can be readily used to generate hash codes or quantize new data points. Importantly, QBH is a generic framework that can be integrated to different property preserving hashing methods and quantization strategies, and we apply QBH to both unsupervised and supervised hashing models as showcases in this paper. Experimental results on three large-scale unlabeled datasets (i.e., SIFT1M, GIST1M, and SIFT1B), three labeled datastes (i.e., ESPGAME, IAPRTC and MIRFLICKR) and one video dataset (UQ_VIDEO) demonstrate the superior performance of our QBH over existing unsupervised and supervised hashing methods.

A multimedia semantic framework for image understanding and retrieval

On the grounds, ontologies have been shown to be a powerful resource for the interpretation and translation of the terminological and semantic relationships within domains of interest but it is still unclear how they can be applied in the context of multimedia data. In this paper, we describe a framework which can capture and manage semantic information related to the multimedia data by modeling in the ontology their features. In particular, the proposed ontology-based framework is organized in the following way: at the lower levels, spatial objects, colors, shapes are represented, and semantic relationships can be established among them; at the higher levels, objects with semantic properties are put into relationship among themselves as well as with the corresponding low-level objects. On this basis, we have designed an ontological system particularly suitable for image retrieval. We have also taken into account the inherent uncertainty related to the representation and detection of multimedia properties in this complex domain. Along this work, we have provided examples from the image domain; moreover, since ontologies provide a semantic means for the semantic comparison of objects and relationships across different formats, the system is easily extensible to other, heterogeneous data sources.

Neighborhood Matching for Image Retrieval

In the last few years, large-scale image retrieval has attracted a lot of attention from the multimedia community. Usual approaches addressing this task first generate an initial ranking of the reference images using fast approximations that do not take into consideration the spatial arrangement of local features in the image (e.g., the bag-of-words paradigm). The top positions of the rankings are then re-estimated with verification methods that deal with more complex information, such as the geometric layout of the image. This verification step allows pruning of many false positives at the expense of an increase in the computational complexity, which may prevent its application to large-scale retrieval problems. This paper describes a geometric method known as neighborhood matching (NM), which revisits the keypoint matching process by considering a neighborhood around each keypoint and improves the efficiency of a geometric verification step in the image search system. Multiple strategies are proposed and compared to incorporate NM into a large-scale image retrieval framework. A detailed analysis and comparison of these strategies and baseline methods have been investigated. The experiments show that the proposed method not only improves the computational efficiency, but also increases the retrieval performance and outperforms state-of-the-art methods in standard datasets, such as the Oxford 5 k and 105 k datasets , for which the spatial verification step has a significant impact on the system performance.

Image retrieval by information fusion based on scalable vocabulary tree and robust Hausdorff distance

In recent years, Scalable Vocabulary Tree (SVT) has been shown to be effective in image retrieval. However, for general images where the foreground is the object to be recognized while the background is cluttered, the performance of the current SVT framework is restricted. In this paper, a new image retrieval framework that incorporates a robust distance metric and information fusion is proposed, which improves the retrieval performance relative to the baseline SVT approach. First, the visual words that represent the background are diminished by using a robust Hausdorff distance between different images. Second, image matching results based on three image signature representations are fused, which enhances the retrieval precision. We conducted intensive experiments on small-scale to large-scale image datasets: Corel-9, Corel-48, and PKU-198, where the proposed Hausdorff metric and information fusion outperforms the state-of-the-art methods by about 13, 15, and 15%, respectively.

Click-boosted graph ranking for image retrieval

Graph ranking is one popular and successful technique for image retrieval, but its effectiveness is often limited by the well-known semantic gap. To bridge this gap, one of the current trends is to leverage the click-through data associated with images to facilitate the graph-based image ranking. However, the sparse and noisy properties of the image click-through data make the exploration of such resource challenging. Towards this end, this paper propose a novel click-boosted graph ranking framework for image retrieval, which consists of two coupled components. Concretely, the first one is a click predictor based on matrix factorization with visual regularization, in order to alleviate the sparseness of the click-through data. The second component is a soft-label graph ranker that conducts the image ranking by using the enriched click-through data noise-tolerantly. Extensive experiments for the tasks of click predicting and image ranking validate the effectiveness of the proposed methods in comparison to several existing approaches.

Construction, implementation and testing of an image identification system using computer vision methods for fruit flies with economic importance (Diptera: Tephritidae).

Many species of Tephritidae are damaging to fruit, which might negatively impact international fruit trade. Automatic or semi-automatic identification of fruit flies are greatly needed for diagnosing causes of damage and quarantine protocols for economically relevant insects. A fruit fly image identification system named AFIS1.0 has been developed using 74 species belonging to six genera, which include the majority of pests in the Tephritidae. The system combines automated image identification and manual verification, balancing operability and accuracy. AFIS1.0 integrates image analysis and expert system into a content-based image retrieval framework. In the the automatic identification module, AFIS1.0 gives candidate identification results. Afterwards users can do manual selection based on comparing unidentified images with a subset of images corresponding to the automatic identification result. The system uses Gabor surface features in automated identification and yielded an overall classification success rate of 87% to the species level by Independent Multi-part Image Automatic Identification Test. The system is useful for users with or without specific expertise on Tephritidae in the task of rapid and effective identification of fruit flies. It makes the application of computer vision technology to fruit fly recognition much closer to production level. © 2016 Society of Chemical Industry.

Ontology-Based Framework for Semantic Text and Image Retrieval Using Chord-length Shape Feature

Ontology-Based Framework for Semantic Text and Image Retrieval Using Chord-length Shape Feature

Integration of wavelet transform, Local Binary Patterns and moments for content-based image retrieval

The proliferation of large number of images has made it necessary to develop systems for indexing and organizing images for easy access. This has made Content-Based Image Retrieval (CBIR) an important area of research in Computer Vision. This paper proposes a combination of features in multiresolution analysis framework for image retrieval. In this work, the concept of multiresolution analysis has been exploited through the use of wavelet transform. This paper combines Local Binary Pattern (LBP) with Legendre Moments at multiple resolutions of wavelet decomposition of image. First, LBP codes of Discrete Wavelet Transform (DWT) coefficients of images are computed to extract texture feature from image. The Legendre Moments of these LBP codes are then computed to extract shape feature from texture feature for constructing feature vectors. These feature vectors are used to search and retrieve visually similar images from large database. The proposed method has been tested on five benchmark datasets, namely, Corel-1K, Olivia-2688, Corel-5K, Corel-10K, and GHIM-10K, and performance of the proposed method has been measured in terms of precision and recall. The experimental results demonstrate that the proposed method outperforms some of the other state-of-the-art methods in terms of precision and recall.

A generic framework for ontology-based information retrieval and image retrieval in web data

AbstractIn the internet era, search engines play a vital role in information retrieval from web pages. Search engines arrange the retrieved results using various ranking algorithms. Additionally, retrieval is based on statistical searching techniques or content-based information extraction methods. It is still difficult for the user to understand the abstract details of every web page unless the user opens it separately to view the web content. This key point provided the motivation to propose and display an ontology-based object-attribute-value (O-A-V) information extraction system as a web model that acts as a user dictionary to refine the search keywords in the query for subsequent attempts. This first model is evaluated using various natural language processing (NLP) queries given as English sentences. Additionally, image search engines, such as Google Images, use content-based image information extraction and retrieval of web pages against the user query. To minimize the semantic gap between the image retrieval results and the expected user results, the domain ontology is built using image descriptions. The second proposed model initially examines natural language user queries using an NLP parser algorithm that will identify the subject-predicate-object (S-P-O) for the query. S-P-O extraction is an extended idea from the ontology-based O-A-V web model. Using this S-P-O extraction and considering the complex nature of writing SPARQL protocol and RDF query language (SPARQL) from the user point of view, the SPARQL auto query generation module is proposed, and it will auto generate the SPARQL query. Then, the query is deployed on the ontology, and images are retrieved based on the auto-generated SPARQL query. With the proposed methodology above, this paper seeks answers to following two questions. First, how to combine the use of domain ontology and semantics to improve information retrieval and user experience? Second, does this new unified framework improve the standard information retrieval systems? To answer these questions, a document retrieval system and an image retrieval system were built to test our proposed framework. The web document retrieval was tested against three key-words/bag-of-words models and a semantic ontology model. Image retrieval was tested on IAPR TC-12 benchmark dataset. The precision, recall and accuracy results were then compared against standard information retrieval systems using TREC_EVAL. The results indicated improvements over the standard systems. A controlled experiment was performed by test subjects querying the retrieval system in the absence and presence of our proposed framework. The queries were measured using two metrics, time and click-count. Comparisons were made on the retrieval performed with and without our proposed framework. The results were encouraging.

An efficient fast-response content-based image retrieval framework for big data

In this paper, an efficient fast-response content-based image retrieval (CBIR) framework based on Hadoop MapReduce is proposed to operate stably with high performance targeting big data. It provides a novel bag of visual words (BOVW) technique based on a proposed chain-clustering binary search-tree (CC-BST) algorithm to build the visual statements for representing the image. As well, it introduces a proposed methodology for creating representatives for these visual statements as a solution for big-data' high-dimensionality. Further, those representatives are utilized to provide an indexing scheme for building one large file as an input for Hadoop. Moreover, an efficient-MapReduce technique is presented to exploit the created visual-representatives of the images to retrieve the top-relevant images for the input query. Empirical tests for the proposed techniques outperform the state-of-art compared techniques.

Fusion framework for color image retrieval based on bag-of-words model and color local Haar binary patterns

Recently, global and local features have demonstrated excellent performance in image retrieval. However, there are some problems in both of them: (1) Local features particularly describe the local textures or patterns. However, similar textures may confuse these local feature extraction methods and get irrelevant retrieval results. (2) Global features delineate overall feature distributions in images, and the retrieved results often appear alike but may be irrelevant. To address problems above, we propose a fusion framework through the combination of local and global features, and thus obtain higher retrieval precision for color image retrieval. Color local Haar binary patterns (CLHBP) and the bag-of-words (BoW) of local features are exploited to capture global and local information of images. The proposed fusion framework combines the ranking results of BoW and CLHBP through a graph-based fusion method. The average retrieval precision of the proposed fusion framework is 83.6% on the Corel-1000 database, and its average precision is 9.9% and 6.4% higher than BoW and CLHBP, respectively. Extensive experiments on different databases validate the feasibility of the proposed framework.

Boosting local texture descriptors with Log-Gabor filters response for improved image retrieval

In the recent past, many local texture descriptors have been proposed for the image retrieval task. In order to improve the image retrieval accuracy, quite a few of these descriptors have been implemented on Gabor filter response. However, the response of Log-Gabor filters has been proved to be better than Gabor filters with respect to their discrimination ability. In this paper, we present a framework for image retrieval that applies various local texture descriptors on Log-Gabor filters response. To evaluate the retrieval performance of the proposed framework, experiments have been conducted on standard Wang, VisTex and OT-Scene databases. Consistent improvement in the image retrieval accuracy demonstrates the effectiveness of this framework. Further, the experimental results show that the use of proposed framework with low-dimension texture descriptors such as Orthogonal Combination of Local Binary Pattern makes them a better choice over Local Binary Pattern and its high-dimensional variants when higher retrieval accuracy, small feature vector size and ease of computation is desired.

Core Failure Mitigation in Integer Sum-of-Product Computations on Cloud Computing Systems

The decreasing mean-time-to-failure estimates in cloud computing systems indicate that multimedia applications running on such environments should be able to mitigate an increasing number of core failures at runtime. We propose a new roll-forward failure-mitigation approach for integer sum-of-product computations, with emphasis on generic matrix multiplication (GEMM) and convolution/crosscorrelation (CONV) routines. Our approach is based on the production of redundant results within the numerical representation of the outputs via the use of numerical packing. This differs from all existing roll-forward solutions that require a separate set of checksum (or duplicate) results. Our proposal imposes 37.5% reduction in the maximum output bitwidth supported in comparison to integer sum-of-product realizations performed on 32-bit integer representations which is comparable to the bitwidth requirement of checksum-methods for multiple core failure mitigation. Experiments with state-of-the-art GEMM and CONV routines running on a c4.8xlarge compute-optimized instance of amazon web services elastic compute cloud (AWS EC2) demonstrate that the proposed approach is able to mitigate up to one quadcore failure while achieving processing throughput that is: 1) comparable to that of the conventional, failure-intolerant, integer GEMM and CONV routines, 2) substantially superior to that of the equivalent roll-forward failure-mitigation method based on checksum streams. Furthermore, when used within an image retrieval framework deployed over a cluster of AWS EC2 spot (i.e., low-cost albeit terminatable) instances, our proposal leads to: 1) 16%–23% cost reduction against the equivalent checksum-based method and 2) more than 70% cost reduction against conventional failure-intolerant processing on AWS EC2 on-demand (i.e., higher-cost albeit guaranteed) instances.

A Novel Modified Evolutionary Algorithm based Image Retrieval Framework: Theoretical Analysis and Applications

A Novel Modified Evolutionary Algorithm based Image Retrieval Framework: Theoretical Analysis and Applications

An Efficient Content-Based Medical Image Retrieval Using Local Vector Pattern

This paper presents image feature extraction and matching framework for image retrieval used in medical image database management and disease diagnosis applications. Since the process of retrieval has facilitated one to find out the required medical image automatically, based on its content, the proposed system is called as Content-Based Medical Image Retrieval (CBMIR) system. The goal of CBMIR system is to enhance visual information analysis by increasing the overall search capabilities to physicians. The proposed system encompasses image feature extraction scheme using Local Vector Pattern (LVP) descriptor and image matching using histogram to retrieve relevant images from a MRI brain images database. The main characteristics of the MRI images are their poor contrast, sensitive to noise and large intensity variations. These challenges are tackled by implementing LVP which was proved as best facial feature extracting approach in the existing system with the first and second order derivatives to improve the recognition performance of the system. Ultimately, results were compared with existing local pattern descriptors such as Local Binary Pattern (LBP), Local Derivative Pattern (LDP) and Local Tetra Pattern (LTrP). We proved that an average recognition rate obtained for implementing LVP in medical image database is better than the other state-of-the-art local pattern descriptor techniques.

A unified learning framework for content based medical image retrieval using a statistical model

This paper presents a unified learning framework for heterogeneous medical image retrieval based on a Full Range Autoregressive Model (FRAR) with the Bayesian approach (BA). Using the unified framework, the color autocorrelogram, edge orientation autocorrelogram (EOAC) and micro-texture information of medical images are extracted. The EOAC is constructed in HSV color space, to circumvent the loss of edges due to spectral and chromatic variations. The proposed system employed adaptive binary tree based support vector machine (ABTSVM) for efficient and fast classification of medical images in feature vector space. The Manhattan distance measure of order one is used in the proposed system to perform a similarity measure in the classified and indexed feature vector space. The precision and recall (PR) method is used as a measure of performance in the proposed system. Short-term based relevance feedback (RF) mechanism is also adopted to reduce the semantic gap. The Experimental results reveal that the retrieval performance of the proposed system for heterogeneous medical image database is better than the existing systems at low computational and storage cost.