Content-based Image Retrieval Applications Research Articles

Content-Based Image Retrieval using Local Features Descriptors and Bag-of-Visual Words

Image retrieval is still an active research topic in the computer vision field. There are existing several techniques to retrieve visual data from large databases. Bag-of-Visual Word (BoVW) is a visual feature descriptor that can be used successfully in Content-based Image Retrieval (CBIR) applications. In this paper, we present an image retrieval system that uses local feature descriptors and BoVW model to retrieve efficiently and accurately similar images from standard databases. The proposed system uses SIFT and SURF techniques as local descriptors to produce image signatures that are invariant to rotation and scale. As well as, it uses K-Means as a clustering algorithm to build visual vocabulary for the features descriptors that obtained of local descriptors techniques. To efficiently retrieve much more images relevant to the query, SVM algorithm is used. The performance of the proposed system is evaluated by calculating both precision and recall. The experimental results reveal that this system performs well on two different standard datasets.

High-order distance-based multiview stochastic learning in image classification.

How do we find all images in a larger set of images which have a specific content? Or estimate the position of a specific object relative to the camera? Image classification methods, like support vector machine (supervised) and transductive support vector machine (semi-supervised), are invaluable tools for the applications of content-based image retrieval, pose estimation, and optical character recognition. However, these methods only can handle the images represented by single feature. In many cases, different features (or multiview data) can be obtained, and how to efficiently utilize them is a challenge. It is inappropriate for the traditionally concatenating schema to link features of different views into a long vector. The reason is each view has its specific statistical property and physical interpretation. In this paper, we propose a high-order distance-based multiview stochastic learning (HD-MSL) method for image classification. HD-MSL effectively combines varied features into a unified representation and integrates the labeling information based on a probabilistic framework. In comparison with the existing strategies, our approach adopts the high-order distance obtained from the hypergraph to replace pairwise distance in estimating the probability matrix of data distribution. In addition, the proposed approach can automatically learn a combination coefficient for each view, which plays an important role in utilizing the complementary information of multiview data. An alternative optimization is designed to solve the objective functions of HD-MSL and obtain different views on coefficients and classification scores simultaneously. Experiments on two real world datasets demonstrate the effectiveness of HD-MSL in image classification.

Content-Based Image Retrieval using Color Quantization and Angle Representation

An efficient non-uniform color quantization and similarity measurement methods are proposed to enhance the content-based image retrieval (CBIR) applications. The HSV color space is selected because it is close to human visual perception system, and a non-uniform color method is proposed to quantize an image into 37 colors. The marker histogram (MH) vector of size 296 values is generated by segmenting the quantized image into 8 regions (multiplication of 45°) and count the occurrences of the quantized colors in their particular angles. To cope with rotated images, an incremental displacement to the MH is applied 7 times. To find similar images, we proposed a new similarity measurement and other 4 existing metrics. A uniform color quantization of related work is implemented too and compared to our quantization method. One-hundred test images are selected from the Corel-1000 images database. Our experimental results conclude high retrieving precision ratios compared to other techniques.

Improved Image Retrieval with Color and Angle Representation

In this research, new ideas are proposed to enhance content-based image retrieval applications by representing colored images in terms of its colors and angles as a histogram describing the number of pixels with particular color located in specific angle, then similarity is measured between the two represented histograms. The color quantization technique is a crucial stage in the CBIR system process, we made comparisons between the uniform and the non-uniform color quantization techniques, and then according to our results we used the non- uniform technique which showed higher efficiency. In our tests we used the Corel-1000 images database in addition to a Matlab code, we compared our results with other approaches like Fuzzy Club, IRM, Geometric Histogram, Signature Based CBIR and Modified ERBIR, and our proposed technique showed high retrieving precision ratios compared to the other techniques.

Image clustering by hyper-graph regularized non-negative matrix factorization

Image clustering is a critical step for the applications of content-based image retrieval, image annotation and other high-level image processing. To achieve these tasks, it is essential to obtain proper representation of the images. Non-negative Matrix Factorization (NMF) learns a part-based representation of the data, which is in accordance with how the brain recognizes objects. Due to its psychological and physiological interpretation, NMF has been successfully applied in a wide range of application such as pattern recognition, image processing and computer vision. On the other hand, manifold learning methods discover intrinsic geometrical structure of the high dimension data space. Incorporating manifold regularizer to standard NMF framework leads to novel performance. In this paper, we proposed a novel algorithm, call Hyper-graph regularized Non-negative Matrix Factorization (HNMF) for this purpose. HNMF captures intrinsic geometrical structure by constructing a hyper-graph instead of a simple graph. Hyper-graph model considers high-order relationship of samples and outperforms simple graph model. Empirical experiments demonstrate the effectiveness of the proposed algorithm in comparison to the state-of-the-art algorithms, especially some related works based on NMF.

Mixed image-keyword query adaptive hashing over multilabel images

This article defines a new hashing task motivated by real-world applications in content-based image retrieval, that is, effective data indexing and retrieval given mixed query (query image together with user-provided keywords). Our work is distinguished from state-of-the-art hashing research by two unique features: (1) Unlike conventional image retrieval systems, the input query is a combination of an exemplar image and several descriptive keywords, and (2) the input image data are often associated with multiple labels. It is an assumption that is more consistent with the realistic scenarios. The mixed image-keyword query significantly extends traditional image-based query and better explicates the user intention. Meanwhile it complicates semantics-based indexing on the multilabel data. Though several existing hashing methods can be adapted to solve the indexing task, unfortunately they all prove to suffer from low effectiveness. To enhance the hashing efficiency, we propose a novel scheme “boosted shared hashing”. Unlike prior works that learn the hashing functions on either all image labels or a single label, we observe that the hashing function can be more effective if it is designed to index over an optimal label subset. In other words, the association between labels and hash bits are moderately sparse. The sparsity of the bit-label association indicates greatly reduced computation and storage complexities for indexing a new sample, since only limited number of hashing functions will become active for the specific sample. We develop a Boosting style algorithm for simultaneously optimizing both the optimal label subsets and hashing functions in a unified formulation, and further propose a query-adaptive retrieval mechanism based on hash bit selection for mixed queries, no matter whether or not the query words exist in the training data. Moreover, we show that the proposed method can be easily extended to the case where the data similarity is gauged by nonlinear kernel functions. Extensive experiments are conducted on standard image benchmarks like CIFAR-10, NUS-WIDE and a-TRECVID. The results validate both the sparsity of the bit-label association and the convergence of the proposed algorithm, and demonstrate that the proposed hashing scheme achieves substantially superior performances over state-of-the-art methods under the same hash bit budget.

Content-based Image Retrieval using Visual Attention Point Features

One of the challenges in the development of a content-based image indexing and retrieval application is to achieve an efficient and robust indexing scheme. Color is a fundamental image feature used in content-based image retrieval (CBIR) systems. This paper proposes a robust and effective image retrieval scheme, which is based on the weighed color histogram of visual attention points. Firstly, the fully affine invariant visual attention points are extracted from the origin color image by using the Affine-SIFT (scale-invariant feature transform) detector. Secondly, according to the color complexity measure (CCM) theory, the visual weight values for the significant visual attention points are calculated to reflect the image local variation. Then, the weighed color histogram of visual attention points is constructed. Finally, the similarity between color images is computed by using the weighed color histogram of visual attention points. Experimental results show that the proposed image retrieval is not only more accurate and efficient in retrieving the user-interested images, but also yields higher retrieval accuracy than some state-of-the-art image retrieval schemes for various test DBs.

ENGLISH

The revolutionary internet and digital technologies have imposed a need to have a system to organize abundantly available digital images for easy categorization and retrieval. We determine relative importance of hue and intensity based on the saturation of an image with respect to the visual properties of the human eye. We effectively apply this method to the generation of a color histogram and use it for content based image retrieval applications. This paper presents the content based image retrieval using features like color and texture. The texture and color features are extracted through wavelet transformation and color histogram and the combination of these features is robust to scaling and translation of objects in an image. The experimental results show that this method is more efficient than the traditional CBIR method based on the single visual feature and other methods combining color and texture.

Interactive object-based image retrieval and annotation on iPad

Apple iPad is a portable tablet computer that offers users a generic platform for consumer media including games, books, and movies. Though iPad is gaining popularity very quickly, its application in content-based image retrieval and annotation is still in its infancy. This paper aims to develop an interactive system to efficiently retrieve and annotate image objects on iPad, which mainly consists of two components of the front-end GUI (graphical user interface) and the back-end retrieval model. In the first component, an iPad-based GUI is implemented, which can provide users with an efficient way to select query objects and facilitate annotations. In the second component, we propose an object-based image retrieval algorithm that combines a novel feature descriptor based on context-preserving bags-of-words (BoW) and a two-stage re-ranking technique to measure the similarity between the query image and each image in the database. The retrieval results are returned and visualized on the iPad-based GUI, and annotations offered by users can be propagated among them. The communication between the front-end GUI and the back-end module is through the use of wireless networks. Comprehensive experiments on several benchmark datasets demonstrated the effectiveness of the proposed framework.

Relevance feedback based on n-tuplewise comparison and the ELECTRE methodology and an application in content-based image retrieval

In this article we propose a method for information retrieval based on relational Multi-Criteria Decision Making. We assume that a user cannot define precise search criteria so that these criteria ...

A weighted dominant color descriptor for content-based image retrieval

Color has been extensively used in the process of image retrieval. The dominant color descriptor (DCD) that was proposed by MPEG-7 is a famous case in point. It is based on compactly describing the prominent colors of an image or a region. However, this technique suffers from some shortcomings; especially with respect to object-based image retrieval. In this paper, a new semantic feature extracted from dominant colors (weight for each DC) is proposed. The newly proposed technique helps reduce the effect of image background on image matching decision where an object’s colors receive much more focus. In addition, a modification to DC-based similarity measure is also proposed. Experimental results demonstrate that the proposed descriptor with the similarity measure modification performs better than the existing descriptor in content-based image retrieval application. The proposed descriptor considers as step forward to the object-based image retrieval.

3D CBIR with sparse coding for image-guided neurosurgery

This research takes an application-specific approach to investigate, extend and implement the state of the art in the fields of both visual information retrieval and machine learning, bridging the gap between theoretical models and real world applications. During an image-guided neurosurgery, path planning remains the foremost and hence the most important step to perform an operation and ensures the maximum resection of an intended target and minimum sacrifice of health tissues. In this investigation, the technique of content-based image retrieval (CBIR) coupled with machine learning algorithms are exploited in designing a computer aided path planning system (CAP) to assist junior doctors in planning surgical paths while sustaining the highest precision. Specifically, after evaluation of approaches of sparse coding and K-means in constructing a codebook, the model of sparse codes of 3D SIFT has been furthered and thereafter employed for retrieving, The novelty of this work lies in the fact that not only the existing algorithms for 2D images have been successfully extended into 3D space, leading to promising results, but also the application of CBIR that is mainly in a research realm, to a clinical sector can be achieved by the integration with machine learning techniques. Comparison with the other four popular existing methods is also conducted, which demonstrates that with the implementation of sparse coding, all methods give better retrieval results than without while constituting the codebook, implying the significant contribution of machine learning techniques.

Fast training procedure for Viola–Jones type object detectors using Laplacian clutter models

This paper presents a fast training strategy for the Viola---Jones (VJ) type object-detection systems. The VJ object- detection system, popular for its high accuracy at real-time testing speeds, has a drawback that it is slow to train. A face detector, for example, can take days to train. In content-based image retrieval (CBIR), where search needs to be performed instantaneously, VJ's long training time is not affordable. Therefore, VJ's method is hardly used for such applications. This paper proposes two modifications to the training algorithm of VJ-type object detection systems which reduces the training time to the order of seconds. Firstly, Laplacian clutter (non-object) models are used to train the weak classifier, thus eliminating the need to read and evaluate thousands of clutter images. Secondly, the training procedure is simplified by removing the time-consuming AdaBoost-based feature selection procedure. An object detector, trained with 500 images, approximately takes 2 s for training in a conventional 3 GHz machine. Our results show that the accuracy of the detector, built with the proposed approach, is inferior to that of VJ for difficult object class such as frontal faces. However, for objects with lesser degree of intra-class variations such as hearts, state-of-the-art accuracy can be obtained. Importantly, for CBIR applications, the fast testing speed of the VJ type object detector is maintained.

A simultaneous feature adaptation and feature selection method for content-based image retrieval systems

In content-based image retrieval (CBIR) applications, each database needs its corresponding parameter setting for feature extraction. However, most of the CBIR systems perform indexing by a set of fixed and pre-specific parameters. On the other hand, feature selection methods have currently gained considerable popularity to reduce semantic gap. In this regard, this paper is devoted to present a hybrid approach to reduce the semantic gap between low level visual features and high level semantics, through simultaneous feature adaptation and feature selection. In the proposed approach, a hybrid meta-heuristic swarm intelligence-based search technique, called mixed gravitational search algorithm (MGSA), is employed. Some feature extraction parameters (i.e. the parameters of a 6-tap parameterized orthogonal mother wavelet in texture features and quantization levels in color histogram) are optimized to reach a maximum precision of the CBIR systems. Meanwhile, feature subset selection is done for the same purpose. A comparative experimental study with the conventional CBIR system is reported on a database of 1000 images. The obtained results confirm the effectiveness of the proposed adaptive indexing method in the field of CBIR.

Quantifying the margin sharpness of lesions on radiological images for content‐based image retrieval

To develop a method to quantify the margin sharpness of lesions on CT and to evaluate it in simulations and CT scans of liver and lung lesions. The authors computed two attributes of margin sharpness: the intensity difference between a lesion and its surroundings, and the sharpness of the intensity transition across the lesion boundary. These two attributes were extracted from sigmoid curves fitted along lines automatically drawn orthogonal to the lesion margin. The authors then represented the margin characteristics for each lesion by a feature vector containing histograms of these parameters. The authors created 100 simulated CT scans of lesions over a range of intensity difference and margin sharpness, and used the concordance correlation between the known parameter and the corresponding computed feature as a measure of performance. The authors also evaluated their method in 79 liver lesions (44 patients: 23 M, 21 F, mean age 61) and 58 lung nodules (57 patients: 24 M, 33 F, mean age 66). The methodology presented takes into consideration the boundary of the liver and lung during feature extraction in clinical images to ensure that the margin feature do not get contaminated by anatomy other than the normal organ surrounding the lesions. For evaluation in these clinical images, the authors created subjective independent reference standards for pairwise margin sharpness similarity in the liver and lung cohorts, and compared rank orderings of similarity used using our sharpness feature to that expected from the reference standards using mean normalized discounted cumulative gain (NDCG) over all query images. In addition, the authors compared their proposed feature with two existing techniques for lesion margin characterization using the simulated and clinical datasets. The authors also evaluated the robustness of their features against variations in delineation of the lesion margin by simulating five types of deformations of the lesion margin. Equivalence across deformations was assessed using Schuirmann's paired two one-sided tests. In simulated images, the concordance correlation between measured gradient and actual gradient was 0.994. The mean (s.d.) and standard deviation NDCG score for the retrieval of K images, K = 5, 10, and 15, were 84% (8%), 85% (7%), and 85% (7%) for CT images containing liver lesions, and 82% (7%), 84% (6%), and 85% (4%) for CT images containing lung nodules, respectively. The authors' proposed method outperformed the two existing margin characterization methods in average NDCG scores over all K, by 1.5% and 3% in datasets containing liver lesion, and 4.5% and 5% in datasets containing lung nodules. Equivalence testing showed that the authors' feature is more robust across all margin deformations (p < 0.05) than the two existing methods for margin sharpness characterization in both simulated and clinical datasets. The authors have described a new image feature to quantify the margin sharpness of lesions. It has strong correlation with known margin sharpness in simulated images and in clinical CT images containing liver lesions and lung nodules. This image feature has excellent performance for retrieving images with similar margin characteristics, suggesting potential utility, in conjunction with other lesion features, for content-based image retrieval applications.

An Efficient Model for Content based Image Retrieval

In the recent world with the advances in multimedia technologies such as compression, display, and visualization technologies and the increased emphasis on multimedia application, the production of image information has resulted in large volume of images that need to be properly indexed for retrieval in future. Hence, there is a need for Content Based Image retrieval application which makes the retrieval process very efficient. Current systems generally make use of low level features like colour, texture, and shape. In this paper, a novel approach for generalized image retrieval based on semantic contents is presented. A combination of two feature extraction methods namely colour and edge histogram descriptor is proposed. The retrieval efficiency is computed and compared by using four methods such as k-means, colour histogram, edge histogram and sobel method. For colour, the histogram of images is computed and for edge, edge histogram descriptors (EHD) are found. For retrieval of images, a novel idea is developed based on greedy strategy to reduce computational complexity. The proposed system stores the content of database images automatically and query image’s content is extracted during runtime and it is used to match against those in database. The result of the query is a set of images that are similar to the query image.

Directional local extrema patterns: a new descriptor for content based image retrieval

In this paper, a new algorithm using directional local extrema patterns meant for content-based image retrieval application is proposed. The standard local binary pattern (LBP) encodes the relationship between reference pixel and its surrounding neighbors by comparing gray-level values. The proposed method differs from the existing LBP in a manner that it extracts the directional edge information based on local extrema in 0 $$^{\circ }$$ , 45 $$^{\circ }$$ , 90 $$^{\circ }$$ , and 135 $$^{\circ }$$ directions in an image. Performance is compared with LBP, block-based LBP (BLK_LBP), center-symmetric local binary pattern (CS-LBP), local edge patterns for segmentation (LEPSEG), local edge patterns for image retrieval (LEPINV), and other existing transform domain methods by conducting four experiments on benchmark databases viz. Corel (DB1) and Brodatz (DB2) databases. The results after being investigated show a significant improvement in terms of their evaluation measures as compared with other existing methods on respective databases.

Block Encoding of Color Histogram for Content based Image Retrieval Applications

In content-based image retrieval (CBIR) application, a large amount of floating-point data is processed. Among various low-level features, color is an important feature and represented in the form of histogram. It is essential that features required to be coded in such a way that the storage space requirement is low and processing speed is high. In this paper, we propose an encoding approach using Golomb-Rice coding, which effectively codes the floating point bin values of the color histogram. The floating point values are converted into integer values using preprocessing steps. The encoded histogram is finally represented in the form of sparse matrix and XOR based bitwise comparison is used as similarity measure to calculate the distance between the encoded and query histogram in the feature space. Based on the number of 1's, the retrieved list is ranked and the relevant images are presented. This approach is tested in CBIR application and the precision of retrieval is encouraging compared to the original color histogram and the average bit length is very low besides having fast retrieval time.

Incremental Indexing for High-Dimensional Data using Tree Structure

A suitable indexing scheme for multimedia information retrieval is the need of the hour and captured the attention of researchers. In content-based image retrieval applications, users wait for long query time to retrieve similar images from the dynamic growing database. A novel indexing scheme is proposed in this paper to achieve a fast response and higher precision of retrieval. The Dynamically Rearranged High Dimensional Data-tree is constructed using available high-dimensional data with dimensionality reduction based on the occurrences of those values. The high-dimensional data or vectors are represented in the compact form and occupy less memory space. The vectors are clustered based on the patterns to support indexing by avoiding both overlapping of vectors and extra factors for clustering. There is no indication of degradation while database size is changed and it also prevents the effect of noisy vectors during retrieval, which help to improve the retrieval speed. Incremental indexing is an important functionality of the proposed tree, which is suitable for dynamic database. For experimental purpose, the coral image database used and found that the performance of the proposed indexing scheme is encouraging.

Fast Wavelet-Based Image Characterization for Highly Adaptive Image Retrieval

Adaptive wavelet-based image characterizations have been proposed in previous works for content-based image retrieval (CBIR) applications. In these applications, the same wavelet basis was used to characterize each query image: This wavelet basis was tuned to maximize the retrieval performance in a training data set. We take it one step further in this paper: A different wavelet basis is used to characterize each query image. A regression function, which is tuned to maximize the retrieval performance in the training data set, is used to estimate the best wavelet filter, i.e., in terms of expected retrieval performance, for each query image. A simple image characterization, which is based on the standardized moments of the wavelet coefficient distributions, is presented. An algorithm is proposed to compute this image characterization almost instantly for every possible separable or nonseparable wavelet filter. Therefore, using a different wavelet basis for each query image does not considerably increase computation times. On the other hand, significant retrieval performance increases were obtained in a medical image data set, a texture data set, a face recognition data set, and an object picture data set. This additional flexibility in wavelet adaptation paves the way to relevance feedback on image characterization itself and not simply on the way image characterizations are combined.