Content-based Image Retrieval Method Research Articles

Self-attention-driven retrieval of chest CT images for COVID-19 assessment

Numerous methods have been developed for computer-aided diagnosis (CAD) of coronavirus disease-19 (COVID-19), based on chest computed tomography (CT) images. The majority of these methods are based on deep neural networks and often act as “black boxes” that cannot easily gain the trust of medical community, whereas their result is uniformly influenced by all image regions. This work introduces a novel, self-attention-driven method for content-based image retrieval (CBIR) of chest CT images. The proposed method analyzes a query CT image and returns a classification result, as well as a list of classified images, ranked according to similarity with the query. Each CT image is accompanied by a heatmap, which is derived by gradient-weighted class activation mapping (Grad-CAM) and represents the contribution of lung tissue and lesions to COVID-19 pathology. Beyond visualization, Grad-CAM weights are employed in a self-attention mechanism, in order to strengthen the influence of the most COVID-19-related image regions on the retrieval result. Experiments on two publicly available datasets demonstrate that the binary classification accuracy obtained by means of DenseNet-201 is 81.3% and 96.4%, for COVID-CT and SARS-CoV-2 datasets, respectively, with a false negative rate which is less than 3% in both datasets. In addition, the Grad-CAM-guided CBIR framework slightly outperforms the plain CBIR in most cases, with respect to nearest neighbour (NN) and first four (FF). The proposed method could serve as a computational tool for a more transparent decision-making process that could be trusted by the medical community. In addition, the employed self-attention mechanism increases the obtained retrieval performance.

Refining Content-Based Segmentation for Prediction of Coffee Bean Quality

Coffee has substantial economic value and is a key foreign exchange source for numerous nations, including Indonesia. Moreover, it is a primary livelihood for many of the country's farmers. Recently, there have been challenges in accurately predicting the quality of coffee beans, primarily due to time, inconsistency, and imprecision issues. Consequently, this study delves into the application of region-growing segmentation and content-based image retrieval (CBIR) techniques to enhance the prediction of coffee bean quality. The proposed hybrid approach, which combines region growing and CBIR methods, aims to improve the precision for forecasting cacao bean quality. Additionally, the research introduces an automated tool that employs these hybrid techniques for quality prediction. The study conducted experiments using a dataset of 400 premium and 400 low-quality coffee beans sourced from the University of Syiah Kuala in Indonesia. The results of the experiments demonstrate a commendable precision rate of 85.4%, showcasing significant improvement compared to certain previous studies.

Feature Extraction Methods by Various Concepts using SOM

Image retrieval systems gained traction with the increased use of visual and media data. It is critical to understand and manage big data, lot of analysis done in image retrieval applications. Given the considerable difficulty involved in handling big data using a traditional approach, there is a demand for its efficient management, particularly regarding accuracy and robustness. To solve these issues, we employ content-based image retrieval (CBIR) methods within both supervised , unsupervised pictures. Self-Organizing Maps (SOM), a competitive unsupervised learning aggregation technique, are applied in our innovative multilevel fusion methodology to extract features that are categorised. The proposed methodology beat state-of-the-art algorithms with 90.3% precision, approximate retrieval precision (ARP) of 0.91, and approximate retrieval recall (ARR) of 0.82 when tested on several benchmark datasets.

A powerful method for interactive content-based image retrieval by variable compressed convolutional info neural networks

A powerful method for interactive content-based image retrieval by variable compressed convolutional info neural networks

Classification on Unsupervised Deep Hashing With Pseudo Labels Using Support Vector Machine for Scalable Image Retrieval

The content-based image retrieval (CBIR) method operates on the low-level visual features of the user input query object, which makes it difficult for users to formulate the query and also does not provide adequate retrieval results. In the past, image annotation was suggested as the best possible framework for CBIR, which works on automatically signing keywords to images that support image retrieval. The recent successes of deep learning techniques, especially Convolutional Neural Networks (CNN), in solving computer vision applications have inspired me to work on this paper to solve the problem of CBIR using a dataset of annotated images

A Diagnostic Study of Content-Based Image Retrieval Technique for Studying the CT Images of Lung Nodules and Prediction of Lung Cancer as a Biometric Tool

Content Based Medical Image Retrieval (CBMIR) can be defined as a digital image search using the contents of the images. CBMIR plays a very important part in medical applications such as retrieving CT images and more accurately diagnosing aberrant lung tissues in CT images. The Content-Based Medical Image Retrieval (CBMIR) method might aid radiotherapists in examining a patient's CT image in order to retrieve comparable pulmonary nodes more precisely by utilizing query nodes. Intending a particular query node, the CBMIR system searches a large chest CT image database for comparable nodes. The prime aim of this research is to evaluate an end-to-end method for developing a CBIR system for lung cancer diagnosis.

Gaze-Dependent Image Re-Ranking Technique for Enhancing Content-Based Image Retrieval

Content-based image retrieval (CBIR) aims to find desired images similar to the image input by the user, and it is extensively used in the real world. Conventional CBIR methods do not consider user preferences since they only determine retrieval results by referring to the degree of resemblance or likeness between the query and potential candidate images. Because of the above reason, a “semantic gap” appears, as the model may not accurately understand the potential intention that a user has included in the query image. In this article, we propose a re-ranking method for CBIR that considers a user’s gaze trace as interactive information to help the model predict the user’s inherent attention. The proposed method uses the user’s gaze trace corresponding to the image obtained from the initial retrieval as the user’s preference information. We introduce image captioning to effectively express the relationship between images and gaze information by generating image captions based on the gaze trace. As a result, we can transform the coordinate data into a text format and explicitly express the semantic information of the images. Finally, image retrieval is performed again using the generated gaze-dependent image captions to obtain images that align more accurately with the user’s preferences or interests. The experimental results on an open image dataset with corresponding gaze traces and human-generated descriptions demonstrate the efficacy or efficiency of the proposed method. Our method considers visual information as the user’s feedback to achieve user-oriented image retrieval.

Learning image representations for content-based image retrieval of radiotherapy treatment plans

Objective. In this work, we propose a content-based image retrieval (CBIR) method for retrieving dose distributions of previously planned patients based on anatomical similarity. Retrieved dose distributions from this method can be incorporated into automated treatment planning workflows in order to streamline the iterative planning process. As CBIR has not yet been applied to treatment planning, our work seeks to understand which current machine learning models are most viable in this context. Approach. Our proposed CBIR method trains a representation model that produces latent space embeddings of a patient’s anatomical information. The latent space embeddings of new patients are then compared against those of previous patients in a database for image retrieval of dose distributions. All source code for this project is available on github. Main results. The retrieval performance of various CBIR methods is evaluated on a dataset consisting of both publicly available image sets and clinical image sets from our institution. This study compares various encoding methods, ranging from simple autoencoders to more recent Siamese networks like SimSiam, and the best performance was observed for the multitask Siamese network. Significance. Our current results demonstrate that excellent image retrieval performance can be obtained through slight changes to previously developed Siamese networks. We hope to integrate CBIR into automated planning workflow in future works.

Content-based medical image retrieval with opponent class adaptive margin loss

The increasing utilization of medical imaging technology with digital storage capabilities has facilitated the compilation of large-scale data repositories. Fast access to image samples with similar appearance to suspected cases in these repositories can help establish a consulting system for healthcare professionals, and improve diagnostic procedures while minimizing processing delays. However, manual querying of large repositories is labor intensive. Content-based image retrieval (CBIR) offers an automated solution based on quantitative assessment of image similarity based on image features in a latent space. Since conventional methods based on hand-crafted features typically show poor generalization performance, learning-based CBIR methods have received attention recently. A common framework in this domain involves classifier-guided models that are trained to detect different image classes. Similarity assessments are then performed on the features captured by the intermediate stages of the trained models. While classifier-guided methods are powerful in inter-class discrimination, they are suboptimally sensitive to within-class differences in image features. An alternative framework instead performs task-agnostic training to learn an embedding space that enforces the representational discriminability of images. Within this representational-learning framework, a powerful method is triplet-wise learning that addresses the deficiencies of point-wise and pair-wise learning in characterizing the similarity relationships between image classes. However, the traditional triplet loss enforces separation between only a subset of image samples within the triplet via a manually-set constant margin value, so it can lead to suboptimal segregation of opponent classes and limited generalization performance. To address these limitations, we introduce a triplet-learning method for automated querying of medical image repositories based on a novel Opponent Class Adaptive Margin (OCAM) loss. To maintain optimally discriminative representations, OCAM considers relationships among all image pairs within the triplet and utilizes an adaptive margin value that is automatically selected per dataset and during the course of training iterations. CBIR performance of OCAM is compared against state-of-the-art loss functions for representational learning on three public databases (gastrointestinal disease, skin lesion, lung disease). On average, OCAM shows an mAP performance of 86.30% in the KVASIR dataset, 70.30% in the ISIC 2019 dataset, and 85.57% in the X-RAY dataset. Comprehensive experiments in each application domain demonstrate the superior performance of OCAM against competing triplet-wise methods at 1.52%, classifier-guided methods at 2.29%, and non-triplet representational-learning methods at 4.56%.

A Novel Hybrid Approach for a Content-Based Image Retrieval Using Feature Fusion

The multimedia content generated by devices and image processing techniques requires high computation costs to retrieve images similar to the user’s query from the database. An annotation-based traditional system of image retrieval is not coherent because pixel-wise matching of images brings significant variations in terms of pattern, storage, and angle. The Content-Based Image Retrieval (CBIR) method is more commonly used in these cases. CBIR efficiently quantifies the likeness between the database images and the query image. CBIR collects images identical to the query image from a huge database and extracts more useful features from the image provided as a query image. Then, it relates and matches these features with the database images’ features and retakes them with similar features. In this study, we introduce a novel hybrid deep learning and machine learning-based CBIR system that uses a transfer learning technique and is implemented using two pre-trained deep learning models, ResNet50 and VGG16, and one machine learning model, KNN. We use the transfer learning technique to obtain the features from the images by using these two deep learning (DL) models. The image similarity is calculated using the machine learning (ML) model KNN and Euclidean distance. We build a web interface to show the result of similar images, and the Precision is used as the performance measure of the model that achieved 100%. Our proposed system outperforms other CBIR systems and can be used in many applications that need CBIR, such as digital libraries, historical research, fingerprint identification, and crime prevention.

A novel image recognition using Fuzzy C-Means and content-based fabric image retrieval

ABSTRACT Due to the growth of image exploitation, there is a requirement of systemizing the images in several ways. Time consumption and accuracy are the most important problem in the CBIR method. Therefore, a Content-Based Fabric Image Retrieval (CBFIR) method is proposed in this paper that is based on texture and colour features extraction. The Fuzzy C-Means (FCM) algorithm creates primary clusters and the Simultaneous Orthogonal Matching Pursuit (SOMP) algorithm updates every cluster in the dictionary. To find the sparse representation dictionary, the test image is compared by the proposed method with the dictionaries generated. The input images are obtained from the fabric dataset. Several distance based similarity measures are utilized for comparison. It can be seen that the proposed method’s performance is sufficiently good concerning the rate of recall, accuracy, and precision with the values of 97%, 91%, and 88% respectively.

Shuffled-Xception-DarkNet-53: A content-based image retrieval model based on deep learning algorithm

This paper proposes Shuffled-Xception-DarkNet-53, an advanced version of DarkNet-53 for Content-Based Image Retrieval (CBIR). The proposed model introduced the notion of the Shuffled -Xception module, which uses three sets of 1 × 1, 3 × 3, and 5 × 5 size filters using a serial connection in place of a single 3 × 3 size filter to extract more refined features from the input images. Instead of the standard 2D Convolution operation, 'Group Convolution' is employed in the proposed Shuffled -Xception module to make the training process of the proposed CNN more efficient. Furthermore, 'Group Convolution' improves the co-relations among the filters of the corresponding Shuffled -Xception module, resulting in more informative features. Between every two serial Group Convolution layers of the same size, one Channel Shuffle layer is used to prevent the loss of information flow among the channels of different groups. The proposed method outperformed the twelve compared methods, including conventional and CNN-based CBIR methods, in ten standard image datasets.

A computational model for medical image retrieval using orthogonal moment

This Paper Deals With Image Retrieval Process Of Liver Computer Tomography (Ct) Scan Images Using Orthogonal Moment Features And Content Based Image Retrieval. Medical Images Are Useful Diagnostic Evidence As It Can Provide Vital Information In Anatomical Pathology. The Objective Is To Efficiently Retrieve Medical Images From The Database Using Orthogonal Moments And Content Based Image Retrieval Methods. The Orthogonal Moment Viz Discrete Racah Polynomial, Continuous Legendre Moments And Zernike Moments Are Computed For The Study. The Region Of Interest Based Segmentation And Watershed Segmentation Is Applied To The Preprocessed Input Images And Features Are Extracted Using Orthogonal Moments And Shape And Texture Features Are Extracted Using Content Based Image Retrieval (Cbir). The Performances Of Each Moment In Terms Of Accuracy And Error Rate Are Compared With Cbir.

RbQE: An Efficient Method for Content-Based Medical Image Retrieval Based on Query Expansion

Systems for retrieving and managing content-based medical images are becoming more important, especially as medical imaging technology advances and the medical image database grows. In addition, these systems can also use medical images to better grasp and gain a deeper understanding of the causes and treatments of different diseases, not just for diagnostic purposes. For achieving all these purposes, there is a critical need for an efficient and accurate content-based medical image retrieval (CBMIR) method. This paper proposes an efficient method (RbQE) for the retrieval of computed tomography (CT) and magnetic resonance (MR) images. RbQE is based on expanding the features of querying and exploiting the pre-trained learning models AlexNet and VGG-19 to extract compact, deep, and high-level features from medical images. There are two searching procedures in RbQE: a rapid search and a final search. In the rapid search, the original query is expanded by retrieving the top-ranked images from each class and is used to reformulate the query by calculating the mean values for deep features of the top-ranked images, resulting in a new query for each class. In the final search, the new query that is most similar to the original query will be used for retrieval from the database. The performance of the proposed method has been compared to state-of-the-art methods on four publicly available standard databases, namely, TCIA-CT, EXACT09-CT, NEMA-CT, and OASIS-MRI. Experimental results show that the proposed method exceeds the compared methods by 0.84%, 4.86%, 1.24%, and 14.34% in average retrieval precision (ARP) for the TCIA-CT, EXACT09-CT, NEMA-CT, and OASIS-MRI databases, respectively.

A Discriminative Feature Learning Approach With Distinguishable Distance Metrics for Remote Sensing Image Classification and Retrieval

The fast data acquisition rate due to the shorter revisit periods and wider observation coverage of satellites results in large amounts of remote sensing images every day. This brings the challenge of how to accurately search the images with similar visual content as the query image. Content-based image retrieval (CBIR) is a solution to this challenge, its performance heavily depends on the effectiveness of the image representation features and similarity evaluation metrics. Ideal image feature representations have dispersed interclass, compact intraclass distribution. However, the neural networks employed by many CBIR methods are trained with cross entropy loss, which does not directly optimize the metrics that evaluates interclass variance over intraclass variance, hence, their feature representations are suboptimal. Meanwhile, the traditional distance metrics used by many CBIR methods cannot index the similarity of feature representations well in high-dimensional space. For better CBIR performance, we propose a discriminative feature learning approach with distinguishable distance metrics for remote sensing image classification and retrieval. By balancing the diagonal elements and nondiagonal elements of the within-class scatter matrix of deep linear discriminant analysis, our proposed loss function, balanced deep linear discriminant analysis, can better optimize the Rayleigh–Ritz quotient, which measures interclass variance over intraclass variance. In addition, the proposed distance metrics, reciprocal exponential distance (RED), is more capable of maintaining distance contrast in high dimensionality, therefore, it can better index similarity for feature representations in high dimensionality. Both visual interpretations and quantitative metrics of extensive experiments demonstrated the effectiveness of our approach.

Supervised Contrastive Learning Based on Fusion of Global and Local Features for Remote Sensing Image Retrieval

With the rapid development of remote sensing sensor technology, the number of remote sensing images (RSIs) has exploded. How to effectively retrieve and manage these massive data has become an urgent problem. At present, content-based image retrieval (CBIR) methods have become a mainstream method due to their excellent performance. However, most of the existing retrieval methods only consider the global features of images, which lacks the ability to discriminate images with the same semantic information but different visual representations. To alleviate this issue, supervised contrastive learning based on the fusion of global and local features method is proposed in this paper, named SCFR. Firstly, a fusion module is designed to combine global and local features to enhance the ability of image expression. Secondly, supervised contrastive learning is introduced into the retrieval task to effectively improve the feature distribution, so that the positive sample pairs are close to each other, and the negative sample pairs are far away from each other in the feature space. Furthermore, to make the distribution of features of the same class more compact, the center contrastive loss is added to the constraints, and combines the class centers that change iteratively with the network. Experimental results on three RSI datasets show that our proposed method has more effective retrieval performance than the state-of-the-art methods. The code and models are available at https://github.com/xdplay17/SCFR.

An optimal approach for content-based image retrieval using deep learning on COVID-19 and pneumonia X-ray Images

Analyzing and comparing the multi-spectral images in the healthcare domain is a complex task as it involves features in the visible spectrum and beyond. Using Content-based image retrieval (CBIR) with deep learning in the healthcare domain provides an ease to identify similar images, which eventually helps in analysis. CBIR, with deep learning, removes the explicit definition of features and autonomously determines similarity on multiple attributes like shape, color, and texture. The proposed method uses multiple deep learning architectures fine-tuned by using max-pool overlapping pooling in all networks and placing the same number of decision layers with the same parameters to determine the result based on feature extraction layers. State-of-the-art Neural Network models VGG-16, VGG-19, Xception, InceptionResNetV2, DenseNet201, MobileNetV2, and NASNetLarge were considered for the experiment to identify the most optimal model. First, a classification process is applied to compare the performance of the state-of-the-art networks. Then, the CBIR process is applied using the feature extraction part. The experiment was conducted on the chest X-rays dataset, consisting of 21,165 images with COVID-19, pneumonia, and normal, with and without rotational invariant cases. The VGG-16 model proved to be the most optimal choice for image retrieval and achieved the highest precision of 99% and mAP of 94.34% compared to recent CBIR methods, which also used chest X-rays datasets. Rotational invariant cases were also tested and achieved mean precision of 86%.

Pre-trained convolution neural networks models for content-based medical image retrieval

Content-based image retrieval (CBIR) is a recent method used to retrieve different types of images from repositories. The traditional content-based medical image retrieval (CBMIR) methods commonly used low-level image representation features extracted from color, texture, and shape image descriptors. Since most of these CBMIR systems depend mainly on the extracted features, the methods used in the feature extraction phase are more important. Features extraction methods, which generate inaccurate features, lead to very poor performance retrieval because of semantic gap widening. Hence, there is high demand for independent domain knowledge features extraction methods, which have automatic learning capabilities from input images. Pre-trained deep convolution neural networks (CNNs), the recent generation of deep learning neural networks, could be used to extract expressive and accurate features. The main advantage of these pre-trained CNNs models is the pre-training process for huge image data of thousands of different classes, and their knowledge after the training process could easily be transferred. There are many successful models of pre-trained CNNs models used in the area of medical image retrieval, image classification, and object recognition. This study utilizes two of the most known pre-trained CNNs models; ResNet18 and SqueezeNet for the offline feature extraction stage. Additionally, the highly accurate features extracted from medical images are used for the CBMIR method of medical image retrieval. This study uses two popular medical image datasets; Kvasir and PH2 to show that the proposed methods have good retrieval results. The retrieval performance evaluation measures of our proposed method have average precision of 97.75% and 83.33% for Kvasir and PH2 medical images respectively, and outperform some of the state-of-the-art methods in this field of study because these pre-trained CNNs have well trained layers among a huge number of image types. Finally, intensive statistical analysis shows that the proposed ResNet18-based retrieval method has the best performance for enhancing both recall and precision measures for both medical images.

Research progress of content-based fabric image retrieval

The application of content-based image retrieval method aims at retrieving similar fabric images and obtaining the existing process parameters to guide production. The process of sample analysis, trial weaving, and proofing can be eliminated in sample imitation production to give full play to the advantages of historical production experience and improve the core competitiveness of enterprises. By investigating and analyzing the applications of content-based image retrieval method technology in fabric retrieval, this article provides a detailed classification and summary of the existing fabric retrieval methods using content-based image retrieval method from six aspects: image preprocessing, feature extraction, similarity measurement, retrieval strategy, dataset construction, and evaluation metrics in the common framework of content-based image retrieval method. The advantages and disadvantages of different methods are analyzed and compared. Finally, the urgent problems and future research directions of fabric image retrieval are discussed, providing ideas for scholars to further study the retrieval methods. Taking fabric as the medium, this article combs the industrial application research and development process of content-based image retrieval method technology, which is helpful to understand the application examples of computer technology and provide research ideas for the application of different computer technologies in the textile industry.

Deep learned vectors’ formation using auto-correlation, scaling, and derivations with CNN for complex and huge image retrieval

Deep learning for image retrieval has been used in this era, but image retrieval with the highest accuracy is the biggest challenge, which still lacks auto-correlation for feature extraction and description. In this paper, a novel deep learning technique for achieving highly accurate results for image retrieval is proposed, which implements a convolutional neural network with auto-correlation, gradient computation, scaling, filter, and localization coupled with state-of-the-art content-based image retrieval methods. For this purpose, novel image features are fused with signatures produced by the VGG-16. In the initial step, images from rectangular neighboring key points are auto-correlated. The image smoothing is achieved by computing intensities according to the local gradient. The result of Gaussian approximation with the lowest scale and suppression is adjusted by the by-box filter with the standard deviation adjusted to the lowest scale. The parameterized images are smoothed at different scales at various levels to achieve high accuracy. The principal component analysis has been used to reduce feature vectors and combine them with the VGG features. These features are integrated with the spatial color coordinates to represent color channels. This experimentation has been performed on Cifar-100, Cifar-10, Tropical fruits, 17 Flowers, Oxford, and Corel-1000 datasets. This study has achieved an extraordinary result for the Cifar-10 and Cifar-100 datasets. Similarly, the results of the study have shown efficient results for texture datasets of 17 Flowers and Tropical fruits. Moreover, when compared to state-of-the-art approaches, this research produced outstanding results for the Corel-1000 dataset.