Performance Of Content-based Image Retrieval Research Articles

Evaluation of retrieval accuracy and visual similarity in content-based image retrieval of chest CT for obstructive lung disease

The aim of our study was to assess the performance of content-based image retrieval (CBIR) for similar chest computed tomography (CT) in obstructive lung disease. This retrospective study included patients with obstructive lung disease who underwent volumetric chest CT scans. The CBIR database included 600 chest CT scans from 541 patients. To assess the system performance, follow-up chest CT scans of 50 patients were evaluated as query cases, which showed the stability of the CT findings between baseline and follow-up chest CT, as confirmed by thoracic radiologists. The CBIR system retrieved the top five similar CT scans for each query case from the database by quantifying and comparing emphysema extent and size, airway wall thickness, and peripheral pulmonary vasculatures in descending order from the database. The rates of retrieval of the same pairs of query CT scans in the top 1–5 retrievals were assessed. Two expert chest radiologists evaluated the visual similarities between the query and retrieved CT scans using a five-point scale grading system. The rates of retrieving the same pairs of query CTs were 60.0% (30/50) and 68.0% (34/50) for top-three and top-five retrievals. Radiologists rated 64.8% (95% confidence interval 58.8–70.4) of the retrieved CT scans with a visual similarity score of four or five and at least one case scored five points in 74% (74/100) of all query cases. The proposed CBIR system for obstructive lung disease integrating quantitative CT measures demonstrated potential for retrieving chest CT scans with similar imaging phenotypes. Further refinement and validation in this field would be valuable.

Content-Based Image Retrieval (CBIR): Using Combined Color and Texture Features (TriCLR and HistLBP)

Content-Based Image Retrieval (CBIR) is a broad research field in the current digital world. This paper focuses on content-based image retrieval based on visual properties, consisting of high-level semantic information. The variation between low-level and high-level features is identified as a semantic gap. The semantic gap is the biggest problem in CBIR. The visual characteristics are extracted from low-level features such as color, texture and shape. The low-level feature increases CBIRs performance level. The paper mainly focuses on an image retrieval system called combined color (TriCLR) (RGB, YCbCr, and [Formula: see text]) with the histogram of texture features in LBP (HistLBP), which, is known as a hybrid of three colors (TriCLR) with Histogram of LBP (TriCLR and HistLBP). The study also discusses the hybrid method in light of low-level features. Finally, the hybrid approach uses the (TriCLR and HistLBP) algorithm, which provides a new solution to the CBIR system that is better than the existing methods.

An evaluation metric for image retrieval systems, using entropy for grouped precision of relevant retrievals

Evaluating the performance of Content-Based Image Retrieval (CBIR) systems is a challenging and intricate task, even for experts in the field. The literature presents a vast array of CBIR systems, each applied to various image databases. Traditionally, automatic metrics employed for CBIR evaluation have been borrowed from the Text Retrieval (TR) domain, primarily precision and recall metrics. However, this paper introduces a novel quantitative metric specifically designed to address the unique characteristics of CBIR. The proposed metric revolves around the concept of grouping relevant images and utilizes the entropy of the retrieved relevant images. Grouping together relevant images holds great value from a user perspective, as it enables more coherent and meaningful results. Consequently, the metric effectively captures and incorporates the grouping of the most relevant outcomes, making it highly advantageous for CBIR evaluation. Additionally, the proposed CBIR metric excels in differentiating between results that might appear similar when assessed using other metrics. It exhibits a superior ability to discern subtle distinctions among retrieval outcomes. This enhanced discriminatory power is a significant advantage of the proposed metric. Furthermore, the proposed performance metric is designed to be straightforward to comprehend and implement. Its simplicity and ease of use contribute to its practicality for researchers and practitioners in the field of CBIR. To validate the effectiveness of our metric, we conducted a comprehensive comparative study involving prominent and well-established CBIR evaluation metrics. The results of this study demonstrate that our proposed metric exhibits robust discrimination power, outperforming existing metrics in accurately evaluating CBIR system performance.

Content-based medical image retrieval with opponent class adaptive margin loss

The increasing utilization of medical imaging technology with digital storage capabilities has facilitated the compilation of large-scale data repositories. Fast access to image samples with similar appearance to suspected cases in these repositories can help establish a consulting system for healthcare professionals, and improve diagnostic procedures while minimizing processing delays. However, manual querying of large repositories is labor intensive. Content-based image retrieval (CBIR) offers an automated solution based on quantitative assessment of image similarity based on image features in a latent space. Since conventional methods based on hand-crafted features typically show poor generalization performance, learning-based CBIR methods have received attention recently. A common framework in this domain involves classifier-guided models that are trained to detect different image classes. Similarity assessments are then performed on the features captured by the intermediate stages of the trained models. While classifier-guided methods are powerful in inter-class discrimination, they are suboptimally sensitive to within-class differences in image features. An alternative framework instead performs task-agnostic training to learn an embedding space that enforces the representational discriminability of images. Within this representational-learning framework, a powerful method is triplet-wise learning that addresses the deficiencies of point-wise and pair-wise learning in characterizing the similarity relationships between image classes. However, the traditional triplet loss enforces separation between only a subset of image samples within the triplet via a manually-set constant margin value, so it can lead to suboptimal segregation of opponent classes and limited generalization performance. To address these limitations, we introduce a triplet-learning method for automated querying of medical image repositories based on a novel Opponent Class Adaptive Margin (OCAM) loss. To maintain optimally discriminative representations, OCAM considers relationships among all image pairs within the triplet and utilizes an adaptive margin value that is automatically selected per dataset and during the course of training iterations. CBIR performance of OCAM is compared against state-of-the-art loss functions for representational learning on three public databases (gastrointestinal disease, skin lesion, lung disease). On average, OCAM shows an mAP performance of 86.30% in the KVASIR dataset, 70.30% in the ISIC 2019 dataset, and 85.57% in the X-RAY dataset. Comprehensive experiments in each application domain demonstrate the superior performance of OCAM against competing triplet-wise methods at 1.52%, classifier-guided methods at 2.29%, and non-triplet representational-learning methods at 4.56%.

A Discriminative Feature Learning Approach With Distinguishable Distance Metrics for Remote Sensing Image Classification and Retrieval

The fast data acquisition rate due to the shorter revisit periods and wider observation coverage of satellites results in large amounts of remote sensing images every day. This brings the challenge of how to accurately search the images with similar visual content as the query image. Content-based image retrieval (CBIR) is a solution to this challenge, its performance heavily depends on the effectiveness of the image representation features and similarity evaluation metrics. Ideal image feature representations have dispersed interclass, compact intraclass distribution. However, the neural networks employed by many CBIR methods are trained with cross entropy loss, which does not directly optimize the metrics that evaluates interclass variance over intraclass variance, hence, their feature representations are suboptimal. Meanwhile, the traditional distance metrics used by many CBIR methods cannot index the similarity of feature representations well in high-dimensional space. For better CBIR performance, we propose a discriminative feature learning approach with distinguishable distance metrics for remote sensing image classification and retrieval. By balancing the diagonal elements and nondiagonal elements of the within-class scatter matrix of deep linear discriminant analysis, our proposed loss function, balanced deep linear discriminant analysis, can better optimize the Rayleigh–Ritz quotient, which measures interclass variance over intraclass variance. In addition, the proposed distance metrics, reciprocal exponential distance (RED), is more capable of maintaining distance contrast in high dimensionality, therefore, it can better index similarity for feature representations in high dimensionality. Both visual interpretations and quantitative metrics of extensive experiments demonstrated the effectiveness of our approach.

Content-Based Image Retrieval for Painting Style with Convolutional Neural Network

With the availability of large paintings dataset on the online platform in recent years, it opens up a new research perspective and researcher started to investigate paintings in a new and different ways. While Convolutional Neural Networks (CNN) have been successfully applied in computer vision domains, several researchers started to investigate the possibility of utilizing CNN for artwork classification and retrieval system. Image retrieval has been one of the most difficult disciplines in digital image processing because it requires scanning a large database for images that are comparable to the query image. It is commonly known that retrieval performance is largely influenced by feature representations in trained algorithm and similarity measures. Deep Learning has recently advanced significantly, and deep features based on deep learning have been widely used because it has been demonstrated that the features have great generalisation. In this paper, a convolutional neural network (CNN) is utilised to classify painting’s style and extract deep and high-level features from the paintings. Next, were used as an image retrieval system to retrieve similar images in artwork style of content which is useful for search or recommendation systems in online art collection. Our experiments show that this strategy significantly improves the performance of content-based image retrieval for the style retrieval task of painting and beat the current state-of-art for painting style classification.

Plasticity-Stability Preserving Multi-Task Learning for Remote Sensing Image Retrieval

Deep learning-based multi-task learning (MTL) methods have recently attracted attention for content-based image retrieval (CBIR) applications in remote sensing (RS). For a given set of tasks (e.g., scene classification, semantic segmentation, and image reconstruction), existing MTL methods employ a joint optimization algorithm on the direct aggregation of task-specific loss functions. Such an approach may provide limited CBIR performance when: 1) tasks compete or even distract each other; 2) one of the tasks dominates the whole learning procedure; or 3) characterization of each task is underperformed compared to single-task learning. This is mainly due to the lack of: 1) plasticity condition (which is associated with sensitivity to new information) or 2) stability condition (which is associated with protection from radical disruptions by new information) of the whole learning procedure. To avoid this issue, as a first time, we propose a novel plasticity-stability preserving MTL (PLASTA-MTL) approach to ensure the plasticity and the stability conditions of the whole learning procedure independently of the number and type of tasks. This is achieved by defining two novel loss functions. The first loss function is the plasticity preserving loss (PPL) function that aims to enforce the global image representation space to be sensitive to new information learned with each task. This is achieved by minimizing the difference of gradient magnitudes for the global representation and task-specific embedding spaces. The second loss function is the stability preserving loss (SPL) function that aims to protect the global representation space radically disrupted by a new task. This is achieved by minimizing the angular distances between the task gradients over global representation space. To effectively employ the proposed loss functions, we also introduce a novel sequential optimization algorithm. Experimental results show the effectiveness of the proposed approach compared to the state-of-the-art MTL methods in the context of CBIR.

Implementation of early and late fusion methods for content-based image retrieval

Content-Based Image Retrieval (CBIR) systems retrieve images from the image repository or database in which they are visually similar to the query image. CBIR plays an important role in various fields such as medical diagnosis, crime prevention, web-based searching, and architecture. CBIR consists mainly of two stages: The first is the extraction of features and the second is the matching of similarities. There are several ways to improve the efficiency and performance of CBIR, such as segmentation, relevance feedback, expansion of queries, and fusion-based methods. The literature has suggested several methods for combining and fusing various image descriptors. In general, fusion strategies are typically divided into two groups, namely early and late fusion strategies. Early fusion is the combination of image features from more than one descriptor into a single vector before the similarity computation, while late fusion refers either to the combination of outputs produced by various retrieval systems or to the combination of different rankings of similarity. In this study, a group of color and texture features is proposed to be used for both methods of fusion strategies. Firstly, an early combination of eighteen color features and twelve texture features are combined into a single vector representation and secondly, the late fusion of three of the most common distance measures are used in the late fusion stage. Our experimental results on two common image datasets show that our proposed method has good performance retrieval results compared to the traditional way of using single features descriptor and also has an acceptable retrieval performance compared to some of the state-of-the-art methods. The overall accuracy of our proposed method is 60.6% and 39.07% for Corel-1K and GHIM-10K ‎datasets, respectively.

Image Retrieval based Convolutional Neural Network

At the present time, everyone is interested in dealing with images in different fields such as geographic maps, medical images, images obtaining by Camera, microscope, telescope, agricultural field photos, paintings, industrial parts drawings, space photos, etc. Content Based Image Retrieval (CBIR) is an efficient retrieval of relevant images from databases based on features extracted from the image. Follow the proposed system for retrieving images related to a query image from a large set of images, based approach to extract the texture features present in the image using statistical methods (PCA, MAD, GLCM, and Fusion) after pre-processing of images. The proposed system was trained using 1D CNN using a dataset Corel10k which widely used for experimental evaluation of CBIR performance the results of proposed system shows that the highest accuracy is 97.5% using Fusion (PCA, MAD), where the accuracy is 95% using MAD, 90% using PCA. The performance result is acceptable compared to previous work.

Content-Based Image Retrieval of Chest CT with Convolutional Neural Network for Diffuse Interstitial Lung Disease: Performance Assessment in Three Major Idiopathic Interstitial Pneumonias.

ObjectiveTo assess the performance of content-based image retrieval (CBIR) of chest CT for diffuse interstitial lung disease (DILD).Materials and MethodsThe database was comprised by 246 pairs of chest CTs (initial and follow-up CTs within two years) from 246 patients with usual interstitial pneumonia (UIP, n = 100), nonspecific interstitial pneumonia (NSIP, n = 101), and cryptogenic organic pneumonia (COP, n = 45). Sixty cases (30-UIP, 20-NSIP, and 10-COP) were selected as the queries. The CBIR retrieved five similar CTs as a query from the database by comparing six image patterns (honeycombing, reticular opacity, emphysema, ground-glass opacity, consolidation and normal lung) of DILD, which were automatically quantified and classified by a convolutional neural network. We assessed the rates of retrieving the same pairs of query CTs, and the number of CTs with the same disease class as query CTs in top 1–5 retrievals. Chest radiologists evaluated the similarity between retrieved CTs and queries using a 5-scale grading system (5-almost identical; 4-same disease; 3-likelihood of same disease is half; 2-likely different; and 1-different disease).ResultsThe rate of retrieving the same pairs of query CTs in top 1 retrieval was 61.7% (37/60) and in top 1–5 retrievals was 81.7% (49/60). The CBIR retrieved the same pairs of query CTs more in UIP compared to NSIP and COP (p = 0.008 and 0.002). On average, it retrieved 4.17 of five similar CTs from the same disease class. Radiologists rated 71.3% to 73.0% of the retrieved CTs with a similarity score of 4 or 5.ConclusionThe proposed CBIR system showed good performance for retrieving chest CTs showing similar patterns for DILD.

Content Based Image Retrieval: A Review

Image recovery was one of the most thrilling and vibrant fields of computer vision science. Content-based image retrieval systems (CBIR) are used to catalog, scan, download and access image databases automatically. Color & texture features are significant properties for content-based image recovery systems. The content-based image retrieval (CBIR) is therefore an attractive source of accurate and quick retrieval. Number of techniques has been established in recent years to improve the performance of CBIR. This paper discusses why CBIR is important nowadays along with the limitations and benefits. Apart from applications, various feature extraction techniques used in CBIR are also discussed.

Study of Content Based Image Retrieval Using Data Mining Techniques

<p>The field of image retrieval has been an active research area for several decades and has been paid more and more attention in recent years as a result of the dramatic and fast increase in the volume of digital images. Content-based image retrieval (CBIR) is a new but widely adopted method for finding images from vast and un annotated image databases. In recent years, a variety of techniques have been developed to improve the performance of CBIR. In reaction to the needs of users, who feel problems connected with traditional methods of image searching and indexing, researchers focus their interest on techniques for retrieving images on the basis of automatically-derived features, often denoted as Content-Based Image Retrieval (CBIR). CBIR systems index the media documents using salient features extracted from the actual media rather than by textual annotations. Query by content is nowadays a very active research field, with many systems being developed by industrial and academic teams. Results performed by these teams are really promising. The situation gets diametrically different when we move our attention from the usual CBIR task, i.e. the retrieval of images which are similar (as a whole) to the query image, to the task “find all images that contain the query image”. The proposed CBIR technique uses more than one clustering techniques to improve the performance of CBIR. This optimized method makes use of K-means and Hierarchical clustering technique to improve the execution time and performance of image retrieval systems in high dimensional sets. In this similarity measure is totally based on colors. In this paper more focus area is the way of combination of clustering technique in order to get faster output of images. In this paper the clustering techniques are discussed and analyzed. Also, we propose a method HDK that uses more than one clustering technique to improve the performance of CBIR. This method makes use of hierarchical and divides and conquers K-means clustering technique with equivalency and compatible relation concepts to improve the performance of the K-Means for using in high dimensional datasets. It also introduced the feature like color, texture and shape for accurate and effective retrieval system.</p>

Query-by-visual-search: multimodal framework for content-based image retrieval

Content-based image retrieval (CBIR) states the procedure of recovering images having similar visual content against a query image from image datasets. In CBIR, the selection of redundant and irrelevant features from images results in the semantic gap issue, which occurs during feature representation and machine learning process. The robust image representation for effective and efficient image retrieval mainly depends upon robust feature selection and classification, which also reduces the semantic gap problem of CBIR. This paper proposed an innovative method for effective and efficient CBIR. The method uses sparse complementary features for vigorous image representation, optimal feature selection based on locality-preserving projection, fuzzy c-means clustering, and soft label support vector machine for robust image classification. In CBIR, smaller and larger sizes of codebook improve the recall and precision (accuracy) of the system, respectively. Due to this reason, the proposed method introduces complementary features based on a larger size codebook, which is assembled using two small sizes of codebooks to increase CBIR performance. The three well-known image datasets (i.e. Corel-1000, Corel-1500, and Holidays) are used to assess the performance of the proposed method. The experimental evaluation highlights promising results as compared to recent methods of CBIR.

Effect of image partitioning on content-based image retrieval using colour and texture

The current application of content-based image retrieval (CBIR) was urgently needed to process image retrieval in large image databases. Previous research had succeeded in combining feature extraction methods using texture and colour. Combining the two methods of Gray Level Co-Occurrence matrix with Colour moment produces a higher level of precision than that produced when method is applied individually. This research examined more deeply about the effect of image partitioning using the grid partitioning method. The results of precision and recall on the CBIR method would be compared with no partitions, two partitions and three image partitions. The results obtained indicated that the addition of image partitions did not affect the precision value. Thus, it can be concluded that increasing the image partition increases the recall value of the overall CBIR performance.

Image retrieval based on gradient-structures histogram

Color perception and orientation selection are very important mechanisms of the human brain that have close relationships with feature extraction and representation. However, extracting low-level features by mimicking these mechanisms remains challenging. To address this problem, we present the gradient-structures histogram as a novel method of content-based image retrieval (CBIR). Its main highlights are: (1) a novel and easy-to-calculate local structure detector, the gradient-structures, which simulates the orientation selection mechanism based on the opponent-color space and connects it with low-level features, (2) a novel discriminative representation method that describes color, intensity and orientation features. It is convenient, as it does not require weight coefficients for color, intensity and orientation. (3) The proposed representation method has the advantages of being histogram-based and having the power to discriminate spatial layout, color and edge cues. The proposed method provides efficient CBIR performance, as demonstrated by comparative experiments in which it significantly outperformed some state-of-the-art methods, including the Bow method, local binary pattern histogram, perceptual uniform descriptor, color volume histogram, color difference histogram, some improved LBP methods and the Tree2Vector method in terms of precision/recall and AUC metrics.

Boosting the Performance of the BoVW Model Using SURF–CoHOG-Based Sparse Features with Relevance Feedback for CBIR

The content-based image retrieval (CBIR) based on sparse visual features is a challenging research problem to categorize images into semantically meaningful classes. Different robust feature representation methods have been proposed to address the semantic gap issue of CBIR. In this article, we propose a novel robust image representation for CBIR, which is based on complementary visual words integration of speeded up robust features (SURF) and co-occurrence histograms of oriented gradients (CoHOG). The SURF is a local feature descriptor, while CoHOG is a global feature descriptor. The local features give better performance for images belonging to different semantic classes and having close visual appearance among their visual contents, while global features give better performance to retrieval images at large scale. To ensure image retrieval accuracy, the proposed method build two smaller size dictionaries, each containing visual words of SURF and CoHOG descriptors, which are assimilated to form one larger size dictionary. In the CBIR system, a dictionary of smaller size produces better sensitivity, while a dictionary of larger sizes produces better specificity. The dictionary of the larger sizes also yields an overfitting problem to affect CBIR performance, which is addressed by introducing a linear discriminant analysis (LDA) and relevance feedback with the proposed method. The comparative performance analysis of the proposed method is performed with the competitor CBIR methods (i.e., feature integration of SURF–CoHOG descriptors’ based on the bag-of-visual-words (BoVW), a single feature of SURF–BoVW and CoHOG–BoVW methods). The quantitative and qualitative analysis carried out on four standard image databases proves the robust performance of the proposed method as compared to its competitor and recent CBIR methods.

Boosting content based image retrieval performance through integration of parametric & nonparametric approaches

The collection of digital images is growing at ever-increasing rate which rises the interest of mining the embedded information. The appropriate representation of an image is inconceivable by a single feature. Thus, the research addresses that point for content based image retrieval (CBIR) by fusing parametric color and shape features with nonparametric texture feature. The color moments, and moment invariants which are parametric methods and applied to describe color distribution and shapes of an image. The nonparametric ranklet transformation is performed to narrate the texture features. Experimentally these parametric and nonparametric features are integrated to propose a robust and effective algorithm. The proposed work is compared with seven existing techniques by determining statistical metrics across five image databases. Finally, a hypothesis test is carried out to establish the significance of the proposed work which, infers evaluated precision and recall values are true and accepted for the all image database.

Scene analysis and search using local features and support vector machine for effective content-based image retrieval

Despite broad investigation in content-based image retrieval (CBIR), issue to lessen the semantic gap between high-level semantics and local attributes of the image is still an important issue. The local attributes of an image such as shape, color, and texture are not sufficient for effective CBIR. Visual similarity is a principal step in CBIR and in the baseline approach. In this article, we introduce a novel approach, which relies on the fusion of visual words of scale-invariant feature transform (SIFT) and binary robust invariant scalable keypoints (BRISK) descriptors based on the visual-bag-of-words approach. The two local feature descriptors are chosen as their fusion adds complementary improvement to CBIR. The SIFT descriptor is capable of detecting objects robustly under cluttering due to its invariance to scale, rotation, noise, and illumination variance. However, SIFT descriptor does not perform well at low illumination or poorly localized keypoints within an image. Due to this reason, the discriminative power of the SIFT descriptor is lost during the quantization process, which also reduces the performance of CBIR. However, the BRISK descriptor provides scale and rotation-invariant scale-space, high quality and adaptive performance in classification based applications. It also performs better at poorly localized keypoints along the edges of an object within an image as compared to the SIFT descriptor. The suggested approach based on the fusion of visual words achieves effective results on the Corel-1K, Corel-1.5K, Corel-5K, and Caltech-256 image repositories as equated to the feature fusion of both descriptors and latest CBIR approaches with the surplus assistances of scalability and fast indexing.

Parallel big image data retrieval by conceptualised clustering and un-conceptualised clustering

Content-based image retrieval is a hot topic which has been studied for few decades. Although there have been a number of recent studies proposed on this topic, it is still hard to achieve a high retrieval performance for big image data. To aim at this issue, in this paper, we propose a parallel content-based image retrieval method that efficiently retrieves the relevant images by un-conceptualised clustering and conceptualised clustering. For un-conceptualised clustering, the un-conceptualised image data is automatically divided into a number of sets, while the conceptualised image data is divided into multiple sets by conceptualised clustering. Based on the clustering index, the depth-first-search strategy is performed to retrieve the relevant images by parallel comparisons. Through experimental evaluations on a large image dataset, the proposed approach is shown to improve the performance of content-based image retrieval substantially in terms of efficiency.

A Novel Technique Based on Visual Words Fusion Analysis of Sparse Features for Effective Content-Based Image Retrieval

Content-based image retrieval (CBIR) is a mechanism that is used to retrieve similar images from an image collection. In this paper, an effective novel technique is introduced to improve the performance of CBIR on the basis of visual words fusion of scale-invariant feature transform (SIFT) and local intensity order pattern (LIOP) descriptors. SIFT performs better on scale changes and on invariant rotations. However, SIFT does not perform better in the case of low contrast and illumination changes within an image, while LIOP performs better in such circumstances. SIFT performs better even at large rotation and scale changes, while LIOP does not perform well in such circumstances. Moreover, SIFT features are invariant to slight distortion as compared to LIOP. The proposed technique is based on the visual words fusion of SIFT and LIOP descriptors which overcomes the aforementioned issues and significantly improves the performance of CBIR. The experimental results of the proposed technique are compared with another proposed novel features fusion technique based on SIFT-LIOP descriptors as well as with the state-of-the-art CBIR techniques. The qualitative and quantitative analysis carried out on three image collections, namely, Corel-A, Corel-B, and Caltech-256, demonstrate the robustness of the proposed technique based on visual words fusion as compared to features fusion and the state-of-the-art CBIR techniques.