Content-based Medical Image Retrieval Research Articles

Multiple semantic X-ray medical image retrieval using efficient feature vector extracted by FPN.

Content-based medical image retrieval (CBMIR) has become an important part of computer-aided diagnostics (CAD) systems. The complex medical semantic information inherent in medical images is the most difficult part to improve the accuracy of image retrieval. Highly expressive feature vectors play a crucial role in the search process. In this paper, we propose an effective deep convolutional neural network (CNN) model to extract concise feature vectors for multiple semantic X-ray medical image retrieval. We build a feature pyramid based CNN model with ResNet50V2 backbone to extract multi-level semantic information. And we use the well-known public multiple semantic annotated X-ray medical image data set IRMA to train and test the proposed model. Our method achieves an IRMA error of 32.2, which is the best score compared to the existing literature on this dataset. The proposed CNN model can effectively extract multi-level semantic information from X-ray medical images. The concise feature vectors can improve the retrieval accuracy of multi-semantic and unevenly distributed X-ray medical images.

A New Approach for Effective Retrieval of Medical Images: A Step towards Computer-Assisted Diagnosis.

The biomedical imaging field has grown enormously in the past decade. In the era of digitization, the demand for computer-assisted diagnosis is increasing day by day. The COVID-19 pandemic further emphasized how retrieving meaningful information from medical repositories can aid in improving the quality of patient's diagnosis. Therefore, content-based retrieval of medical images has a very prominent role in fulfilling our ultimate goal of developing automated computer-assisted diagnosis systems. Therefore, this paper presents a content-based medical image retrieval system that extracts multi-resolution, noise-resistant, rotation-invariant texture features in the form of a novel pattern descriptor, i.e., MsNrRiTxP, from medical images. In the proposed approach, the input medical image is initially decomposed into three neutrosophic images on its transformation into the neutrosophic domain. Afterwards, three distinct pattern descriptors, i.e., MsTrP, NrTxP, and RiTxP, are derived at multiple scales from the three neutrosophic images. The proposed MsNrRiTxP pattern descriptor is obtained by scale-wise concatenation of the joint histograms of MsTrP×RiTxP and NrTxP×RiTxP. To demonstrate the efficacy of the proposed system, medical images of different modalities, i.e., CT and MRI, from four test datasets are considered in our experimental setup. The retrieval performance of the proposed approach is exhaustively compared with several existing, recent, and state-of-the-art local binary pattern-based variants. The retrieval rates obtained by the proposed approach for the noise-free and noisy variants of the test datasets are observed to be substantially higher than the compared ones.

Content based COVID-19 image retrieval system using local histogram equalization and deep convolutional neural network

Doctors play a critical role in interpreting medical images as part of their core responsibilities. They need to find comparable examples that can assist in making informed decisions, especially when encountering ambiguous visuals. Traditionally, Systems such as content-based medical image retrieval (CBMIR) have been used for this. The proposed method employs a novel technique, local histogram equalization (LHE) for preprocessing, transfer learning-based convolutional neural network to extract the representative features with Manhattan and Euclidean distance metrics to assess how similar the query image and database image are to one another. This model is trained on a standard dataset namely Chest X-Ray images. Top-k, Precision and Recall measure is employed to assess system performance. From the results, the suggested enhanced convolutional neural network (CNN) model demonstrates significantly superior performance in the top 10 retrieval rates of 97.13% for coronavirus disease 2019 (COVID-19), 96.84% for normal, 82.63% for pneumonia-bacterial, and 81.72% for pneumonia-viral and precision@recall10 of 93.14% for COVID-19, 91.88% for normal, 77.84% for pneumonia-bacterial and 74.71% for pneumonia-viral.

SEMANTIC SEGMENTATION AND CONTENT-BASED RETRIEVAL IN MULTIMEDIA IMAGE DATABASES

Brain tumors, particularly gliomas, pose a significant threat to global health, necessitating accurate and efficient diagnostic methods. Magnetic Resonance Imaging (MRI) serves as a crucial tool for diagnosing glioma grades, but interpretation is subject to variability, hindering treatment planning. Intra and inter-observer variability in radiological image interpretation impede effective therapeutic strategies for brain tumor patients. Accessing relevant images from vast medical databases for comparison and treatment planning is cumbersome and time-consuming. This paper proposes a Content-Based Medical Image Retrieval (CBIR) system utilizing Convolutional Neural Network (CNN)-based feature extraction, specifically employing the AlexNet architecture. The system employs KNN clustering for indexing the feature map database and implements Gain-based feature selection to reduce feature vector dimensionality. The proposed system underwent evaluation using BraTS 2018 and 2020 datasets with five-fold cross-validation. Achieving state-of-the-art performance, the system demonstrated a mean Average Precision of 98% and Precision of 97%, showcasing its efficacy in accurately retrieving similar pathological MRI brain images.

Advanced Image Processing Techniques for Medical Image Retrieval Using Visual Features and Distance Measures

Recently, there has been a rapid increase in the volume of medical image repositories because of the maximized utilization of digitized image data in hospitals. As a result, there is a complexity in querying and handling such large databases, which has led to the development of a novel Content-Based Medical Image Retrieval (CBMIR) technique. In this work, the CBMIR technique is developed using three types of visual features (i.e., color, texture and shape) along with 12 distance measures, and is optimized with the Crossover-Gravitational Search Algorithm (CRO-GSA). For ease, we named this technique Content-Based Medical Image Retrieval using Visual features with CRO-GSA (CBMIRVC). The initial step of our CBMIRVC technique is to extract three types of visual features from the images. Then, each type of feature is employed with a suitable distance measure, which is used for the computation of image similarities between the medical images in the database and a provided query image. The optimization of our proposed CBMIRVC technique is done by the CRO-GSA algorithm. This algorithm optimizes the CBMIRVC technique by identifying the optimal combinations of visual features and similarity measures. Additionally, optimal weights are computed in accordance with the three types of features for the three similarities. Experimental validations were done using retrieved cancer-related and endoscopic color images from databases holding numerous image categories. The experimental outcome shows that our proposed CBMIRVC technique outperforms other conventional image retrieval techniques by effectively retrieving similar images from large databases.

Sketch-based semantic retrieval of medical images

The volume of medical images stored in hospitals is rapidly increasing; however, the utilization of these accumulated medical images remains limited. Existing content-based medical image retrieval (CBMIR) systems typically require example images, leading to practical limitations, such as the lack of customizable, fine-grained image retrieval, the inability to search without example images, and difficulty in retrieving rare cases. In this paper, we introduce a sketch-based medical image retrieval (SBMIR) system that enables users to find images of interest without the need for example images. The key concept is feature decomposition of medical images, which allows the entire feature of a medical image to be decomposed into and reconstructed from normal and abnormal features. Building on this concept, our SBMIR system provides an easy-to-use two-step graphical user interface: users first select a template image to specify a normal feature and then draw a semantic sketch of the disease on the template image to represent an abnormal feature. The system integrates both types of input to construct a query vector and retrieves reference images. For evaluation, ten healthcare professionals participated in a user test using two datasets. Consequently, our SBMIR system enabled users to overcome previous challenges, including image retrieval based on fine-grained image characteristics, image retrieval without example images, and image retrieval for rare cases. Our SBMIR system provides on-demand, customizable medical image retrieval, thereby expanding the utility of medical image databases.

Effect of Genetic Algorithm as a Feature Selection for Image Classification

Analysis of image content is important in the classification of images, identification, retrieval, and recognition processes. The medical image datasets for content-based medical image retrieval ( are large datasets that are limited by high computational costs and poor performance. The aim of the proposed method is to enhance this image retrieval and classification by using a genetic algorithm (GA) to choose the reduced features and dimensionality. This process was created in three stages. In the first stage, two algorithms are applied to extract the important features; the first algorithm is the Contrast Enhancement method and the second is a Discrete Cosine Transform algorithm. In the next stage, we used datasets of the medical images using GA-based feature selection to find feature vectors. Images from the datasets and images from the query are recognized using a correlation coefficient. The third stage of the proposed method used a diverse density algorithm feedback technique to enhance the performance of the . Images of breast cancer, brain cancer, lung cancer, thyroid cancer, etc., may be retrieved using the suggested procedure. By using a feature selection algorithm based on GA to determine the best subset of features, the challenge of system dimensionality is reduced. The suggested method has greater accuracy in precision, recall, and F-score than the other techniques.

Content-based image retrieval for medical diagnosis using fuzzy clustering and deep learning

Brain tumors are one of the most threatening causes of death worldwide. Neuroradiologists widely use Magnetic Resonance Imaging as a golden standard non-invasive imaging method for diagnosing Glioma grades. Brain tumor diagnosis based on radiology images is tedious and highly subjective to intra and inter-observer variability; it adversely affects therapeutic planning. Better medical treatment can be provided relating to similar past cases if relevant images are retrieved from an extensive medical image database. Content-Based Image Retrieval systems are potent image tools for dealing with such massive datasets. In a CBIR system, accurate classification and retrieval of similar pathological images can be effectively automated by utilising Convolutional Neural Network-based feature extraction methods. This paper presents a Content-Based Medical Image Retrieval pipeline on a medical domain using the CNN model for feature extraction and the clustering method used to index the feature map database. The proposed system applies a Multi-level Gain-based feature selection to reduce the dimensionality of the feature vectors obtained from the pre-trained CNN models. The experiment follows five-fold cross-validation on BraTS 2018 and 2020 datasets. The proposed system achieves state-of-the-art MRI brain image retrieval performance with a mean Average Precision and Precision@10 of 98.15% and 97.62%, respectively. The results prove the proposed method’s effectiveness and validate its clinical application.

WWFedCBMIR: World-Wide Federated Content-Based Medical Image Retrieval.

The paper proposes a federated content-based medical image retrieval (FedCBMIR) tool that utilizes federated learning (FL) to address the challenges of acquiring a diverse medical data set for training CBMIR models. CBMIR is a tool to find the most similar cases in the data set to assist pathologists. Training such a tool necessitates a pool of whole-slide images (WSIs) to train the feature extractor (FE) to extract an optimal embedding vector. The strict regulations surrounding data sharing in hospitals makes it difficult to collect a rich data set. FedCBMIR distributes an unsupervised FE to collaborative centers for training without sharing the data set, resulting in shorter training times and higher performance. FedCBMIR was evaluated by mimicking two experiments, including two clients with two different breast cancer data sets, namely BreaKHis and Camelyon17 (CAM17), and four clients with the BreaKHis data set at four different magnifications. FedCBMIR increases the F1 score (F1S) of each client from 96% to 98.1% in CAM17 and from 95% to 98.4% in BreaKHis, with 11.44 fewer hours in training time. FedCBMIR provides 98%, 96%, 94%, and 97% F1S in the BreaKHis experiment with a generalized model and accomplishes this in 25.53 fewer hours of training.

Content-based medical image retrieval using fractional Hartley transform with hybrid features

Content-based medical image retrieval using fractional Hartley transform with hybrid features

Medical image retrieval via nearest neighbor search on pre-trained image features

Nearest neighbor search, also known as NNS, is a technique used to locate the points in a high-dimensional space closest to a given query point. This technique has multiple applications in medicine, such as searching large medical imaging databases, disease classification, and diagnosis. However, when the number of points is significantly large, the brute-force approach for finding the nearest neighbor becomes computationally infeasible. Therefore, various approaches have been developed to make the search faster and more efficient to support the applications. With a focus on medical imaging, this paper proposes DenseLinkSearch (DLS), an effective and efficient algorithm that searches and retrieves the relevant images from heterogeneous sources of medical images. Towards this, given a medical database, the proposed algorithm builds an index that consists of pre-computed links of each point in the database. The search algorithm utilizes the index to efficiently traverse the database in search of the nearest neighbor. We also explore the role of medical image feature representation in content-based medical image retrieval tasks. We propose a Transformer-based feature representation technique that outperformed the existing pre-trained Transformer-based approaches on benchmark medical image retrieval datasets. We extensively tested the proposed NNS approach and compared the performance with state-of-the-art NNS approaches on benchmark datasets and our created medical image datasets. The proposed approach outperformed the existing approaches in terms of retrieving accurate neighbors and retrieval speed. In comparison to the existing approximate NNS approaches, our proposed DLS approach outperformed them in terms of lower average time per query and ≥99% R@10 on 11 out of 13 benchmark datasets. We also found that the proposed medical feature representation approach is better for representing medical images compared to the existing pre-trained image models. The proposed feature extraction strategy obtained an improvement of 9.37%, 7.0%, and 13.33% in terms of P@5, P@10, and P@20, respectively, in comparison to the best-performing pre-trained image model. The source code and datasets of our experiments are available at https://github.com/deepaknlp/DLS.

A Diagnostic Study of Content-Based Image Retrieval Technique for Studying the CT Images of Lung Nodules and Prediction of Lung Cancer as a Biometric Tool

Content Based Medical Image Retrieval (CBMIR) can be defined as a digital image search using the contents of the images. CBMIR plays a very important part in medical applications such as retrieving CT images and more accurately diagnosing aberrant lung tissues in CT images. The Content-Based Medical Image Retrieval (CBMIR) method might aid radiotherapists in examining a patient's CT image in order to retrieve comparable pulmonary nodes more precisely by utilizing query nodes. Intending a particular query node, the CBMIR system searches a large chest CT image database for comparable nodes. The prime aim of this research is to evaluate an end-to-end method for developing a CBIR system for lung cancer diagnosis.

An Improved and Efficient Content-Based Medical Image Retrieval Technique vs State-of-Art Texture Feature Extraction Techniques

Areas in the health sector such as X-rays, Dermatology, High-resolution Computed Tomography – HRCT, Endoscopy, Radiology, Cardiology and Magnetic Resonance Imaging – MRI heavily depend on medical images for their activities hence it has become complex for managing and accessing these images from their repositories. As a matter of concern content-based medical image retrieval-CBMIR has been the system proposed by many researchers for handling access to similar medical image(s) as the input image, yet there is a caveat that has to be addressed with respect to which technique best suits CBMIR system with respect to a given performance metric. Medical images are mostly of grayscale and therefore color feature extraction techniques may not work effectively on them. Since there is no clear indication of which of the various texture feature extraction techniques is best suited for a given performance metric, this work seeks to comparatively evaluate the performance of the following state-of-the-art texture feature extraction techniques; Local Binary Pattern (LBP), Gabor Filter, Gray-Level Co-occurrence Matrix (GLCM), Haralick Descriptor, Features from Accelerated Segment Test (FAST) and a Proposed Technique using the metrics; Precision, Recall, F1-score, Mean Squared Error (MSE), Accuracy and Time. The results showed that the proposed technique is best suited for systems focusing on precision with an average precision score of 100% over 10.5k of raw medical images (dataset) using an appreciable minimum time with time complexity of 0(n).

Content-based medical image retrieval with opponent class adaptive margin loss

The increasing utilization of medical imaging technology with digital storage capabilities has facilitated the compilation of large-scale data repositories. Fast access to image samples with similar appearance to suspected cases in these repositories can help establish a consulting system for healthcare professionals, and improve diagnostic procedures while minimizing processing delays. However, manual querying of large repositories is labor intensive. Content-based image retrieval (CBIR) offers an automated solution based on quantitative assessment of image similarity based on image features in a latent space. Since conventional methods based on hand-crafted features typically show poor generalization performance, learning-based CBIR methods have received attention recently. A common framework in this domain involves classifier-guided models that are trained to detect different image classes. Similarity assessments are then performed on the features captured by the intermediate stages of the trained models. While classifier-guided methods are powerful in inter-class discrimination, they are suboptimally sensitive to within-class differences in image features. An alternative framework instead performs task-agnostic training to learn an embedding space that enforces the representational discriminability of images. Within this representational-learning framework, a powerful method is triplet-wise learning that addresses the deficiencies of point-wise and pair-wise learning in characterizing the similarity relationships between image classes. However, the traditional triplet loss enforces separation between only a subset of image samples within the triplet via a manually-set constant margin value, so it can lead to suboptimal segregation of opponent classes and limited generalization performance. To address these limitations, we introduce a triplet-learning method for automated querying of medical image repositories based on a novel Opponent Class Adaptive Margin (OCAM) loss. To maintain optimally discriminative representations, OCAM considers relationships among all image pairs within the triplet and utilizes an adaptive margin value that is automatically selected per dataset and during the course of training iterations. CBIR performance of OCAM is compared against state-of-the-art loss functions for representational learning on three public databases (gastrointestinal disease, skin lesion, lung disease). On average, OCAM shows an mAP performance of 86.30% in the KVASIR dataset, 70.30% in the ISIC 2019 dataset, and 85.57% in the X-RAY dataset. Comprehensive experiments in each application domain demonstrate the superior performance of OCAM against competing triplet-wise methods at 1.52%, classifier-guided methods at 2.29%, and non-triplet representational-learning methods at 4.56%.

Feature Extraction Method using HoG with LTP for Content-Based Medical Image Retrieval

An accurate diagnosis is significant for the treatment of any disease in its early stage. Content-Based Medical Image Retrieval (CBMIR) is used to find similar medical images in a huge database to help radiologists in diagnosis. The main difficulty in CBMIR is semantic gaps between the lower-level visual details, captured by computer-aided tools and higher-level semantic details captured by humans. Many existing methods such as Manhattan Distance, Triplet Deep Hashing, and Transfer Learning techniques for CBMIR were developed but showed lower efficiency and the computational cost was high. To solve such issues, a new feature extraction approach is proposed using Histogram of Gradient (HoG) with Local Ternary Pattern (LTP) to automatically retrieve medical images from the Contrast-Enhanced Magnetic Resonance Imaging (CE-MRI) database. Adam optimization algorithm is utilized to select features and the Euclidean measure calculates the similarity for query images. From the experimental analysis, it is clearly showing that the proposed HoG-LTP method achieves higher accuracy of 98.8%, a sensitivity of 98.5%, and a specificity of 99.416%, which is better when compared to the existing Random Forest (RF) method which displayed an accuracy, sensitivity, and specificity of 81.1%, 81.7% and 90.5% respectively.

An Accurate and Privacy-Preserving Retrieval Scheme Over Outsourced Medical Images

With the rapid advancement in medical imaging techniques, Content-Based (medical) Image Retrieval (CBIR), which can assist in disease diagnosis, has gained much attention in both academia and industry. However, due to patients’ sensitive information involved in medical images, privacy-preserving CBIR is a challenge worth exploiting. Though several privacy-preserving CBIR schemes have been put forth, they can only resist known-background attack (KBA), and do not suffice for protecting the image privacy in outsourced settings. In this article, aiming at the above challenge, we first design a novel Privacy-preserving Mahalanobis Distance Comparison (PMDC) method to improve the accuracy of medical images retrieval. Then, combined with the Mahalanobis distance based Fuzzy C-Means (FCM-M) algorithm, a scheme named TAMMIE is proposed to achieve accurate and privacy-preserving medical image retrieval over encrypted data. With TAMMIE, an image owner can securely outsource the images and indexes to a cloud server, and query users can request retrieval services from the cloud server while keeping their queries private. Detailed security analysis shows that our proposed schemes are secure under the attack stronger than KBA. Furthermore, thorough empirical experiments conducted on two real-world and one synthetic datasets also demonstrate the efficiency of TAMMIE.

RbQE: An Efficient Method for Content-Based Medical Image Retrieval Based on Query Expansion

Systems for retrieving and managing content-based medical images are becoming more important, especially as medical imaging technology advances and the medical image database grows. In addition, these systems can also use medical images to better grasp and gain a deeper understanding of the causes and treatments of different diseases, not just for diagnostic purposes. For achieving all these purposes, there is a critical need for an efficient and accurate content-based medical image retrieval (CBMIR) method. This paper proposes an efficient method (RbQE) for the retrieval of computed tomography (CT) and magnetic resonance (MR) images. RbQE is based on expanding the features of querying and exploiting the pre-trained learning models AlexNet and VGG-19 to extract compact, deep, and high-level features from medical images. There are two searching procedures in RbQE: a rapid search and a final search. In the rapid search, the original query is expanded by retrieving the top-ranked images from each class and is used to reformulate the query by calculating the mean values for deep features of the top-ranked images, resulting in a new query for each class. In the final search, the new query that is most similar to the original query will be used for retrieval from the database. The performance of the proposed method has been compared to state-of-the-art methods on four publicly available standard databases, namely, TCIA-CT, EXACT09-CT, NEMA-CT, and OASIS-MRI. Experimental results show that the proposed method exceeds the compared methods by 0.84%, 4.86%, 1.24%, and 14.34% in average retrieval precision (ARP) for the TCIA-CT, EXACT09-CT, NEMA-CT, and OASIS-MRI databases, respectively.

A dynamic programming approach for accurate content-based retrieval of ordinary and nano-scale medical images

A dynamic programming approach for accurate content-based retrieval of ordinary and nano-scale medical images

A dynamic programming approach for accurate content-based retrieval of ordinary and nano-scale medical images

A dynamic programming approach for accurate content-based retrieval of ordinary and nano-scale medical images

Pre-trained convolution neural networks models for content-based medical image retrieval

Content-based image retrieval (CBIR) is a recent method used to retrieve different types of images from repositories. The traditional content-based medical image retrieval (CBMIR) methods commonly used low-level image representation features extracted from color, texture, and shape image descriptors. Since most of these CBMIR systems depend mainly on the extracted features, the methods used in the feature extraction phase are more important. Features extraction methods, which generate inaccurate features, lead to very poor performance retrieval because of semantic gap widening. Hence, there is high demand for independent domain knowledge features extraction methods, which have automatic learning capabilities from input images. Pre-trained deep convolution neural networks (CNNs), the recent generation of deep learning neural networks, could be used to extract expressive and accurate features. The main advantage of these pre-trained CNNs models is the pre-training process for huge image data of thousands of different classes, and their knowledge after the training process could easily be transferred. There are many successful models of pre-trained CNNs models used in the area of medical image retrieval, image classification, and object recognition. This study utilizes two of the most known pre-trained CNNs models; ResNet18 and SqueezeNet for the offline feature extraction stage. Additionally, the highly accurate features extracted from medical images are used for the CBMIR method of medical image retrieval. This study uses two popular medical image datasets; Kvasir and PH2 to show that the proposed methods have good retrieval results. The retrieval performance evaluation measures of our proposed method have average precision of 97.75% and 83.33% for Kvasir and PH2 medical images respectively, and outperform some of the state-of-the-art methods in this field of study because these pre-trained CNNs have well trained layers among a huge number of image types. Finally, intensive statistical analysis shows that the proposed ResNet18-based retrieval method has the best performance for enhancing both recall and precision measures for both medical images.