Content-based Medical Image Retrieval Research Articles

Advanced Image Processing Techniques for Medical Image Retrieval Using Visual Features and Distance Measures

Recently, there has been a rapid increase in the volume of medical image repositories because of the maximized utilization of digitized image data in hospitals. As a result, there is a complexity in querying and handling such large databases, which has led to the development of a novel Content-Based Medical Image Retrieval (CBMIR) technique. In this work, the CBMIR technique is developed using three types of visual features (i.e., color, texture and shape) along with 12 distance measures, and is optimized with the Crossover-Gravitational Search Algorithm (CRO-GSA). For ease, we named this technique Content-Based Medical Image Retrieval using Visual features with CRO-GSA (CBMIRVC). The initial step of our CBMIRVC technique is to extract three types of visual features from the images. Then, each type of feature is employed with a suitable distance measure, which is used for the computation of image similarities between the medical images in the database and a provided query image. The optimization of our proposed CBMIRVC technique is done by the CRO-GSA algorithm. This algorithm optimizes the CBMIRVC technique by identifying the optimal combinations of visual features and similarity measures. Additionally, optimal weights are computed in accordance with the three types of features for the three similarities. Experimental validations were done using retrieved cancer-related and endoscopic color images from databases holding numerous image categories. The experimental outcome shows that our proposed CBMIRVC technique outperforms other conventional image retrieval techniques by effectively retrieving similar images from large databases.

Sketch-based semantic retrieval of medical images

The volume of medical images stored in hospitals is rapidly increasing; however, the utilization of these accumulated medical images remains limited. Existing content-based medical image retrieval (CBMIR) systems typically require example images, leading to practical limitations, such as the lack of customizable, fine-grained image retrieval, the inability to search without example images, and difficulty in retrieving rare cases. In this paper, we introduce a sketch-based medical image retrieval (SBMIR) system that enables users to find images of interest without the need for example images. The key concept is feature decomposition of medical images, which allows the entire feature of a medical image to be decomposed into and reconstructed from normal and abnormal features. Building on this concept, our SBMIR system provides an easy-to-use two-step graphical user interface: users first select a template image to specify a normal feature and then draw a semantic sketch of the disease on the template image to represent an abnormal feature. The system integrates both types of input to construct a query vector and retrieves reference images. For evaluation, ten healthcare professionals participated in a user test using two datasets. Consequently, our SBMIR system enabled users to overcome previous challenges, including image retrieval based on fine-grained image characteristics, image retrieval without example images, and image retrieval for rare cases. Our SBMIR system provides on-demand, customizable medical image retrieval, thereby expanding the utility of medical image databases.

Effect of Genetic Algorithm as a Feature Selection for Image Classification

Analysis of image content is important in the classification of images, identification, retrieval, and recognition processes. The medical image datasets for content-based medical image retrieval ( are large datasets that are limited by high computational costs and poor performance. The aim of the proposed method is to enhance this image retrieval and classification by using a genetic algorithm (GA) to choose the reduced features and dimensionality. This process was created in three stages. In the first stage, two algorithms are applied to extract the important features; the first algorithm is the Contrast Enhancement method and the second is a Discrete Cosine Transform algorithm. In the next stage, we used datasets of the medical images using GA-based feature selection to find feature vectors. Images from the datasets and images from the query are recognized using a correlation coefficient. The third stage of the proposed method used a diverse density algorithm feedback technique to enhance the performance of the . Images of breast cancer, brain cancer, lung cancer, thyroid cancer, etc., may be retrieved using the suggested procedure. By using a feature selection algorithm based on GA to determine the best subset of features, the challenge of system dimensionality is reduced. The suggested method has greater accuracy in precision, recall, and F-score than the other techniques.

Content-based image retrieval for medical diagnosis using fuzzy clustering and deep learning

Brain tumors are one of the most threatening causes of death worldwide. Neuroradiologists widely use Magnetic Resonance Imaging as a golden standard non-invasive imaging method for diagnosing Glioma grades. Brain tumor diagnosis based on radiology images is tedious and highly subjective to intra and inter-observer variability; it adversely affects therapeutic planning. Better medical treatment can be provided relating to similar past cases if relevant images are retrieved from an extensive medical image database. Content-Based Image Retrieval systems are potent image tools for dealing with such massive datasets. In a CBIR system, accurate classification and retrieval of similar pathological images can be effectively automated by utilising Convolutional Neural Network-based feature extraction methods. This paper presents a Content-Based Medical Image Retrieval pipeline on a medical domain using the CNN model for feature extraction and the clustering method used to index the feature map database. The proposed system applies a Multi-level Gain-based feature selection to reduce the dimensionality of the feature vectors obtained from the pre-trained CNN models. The experiment follows five-fold cross-validation on BraTS 2018 and 2020 datasets. The proposed system achieves state-of-the-art MRI brain image retrieval performance with a mean Average Precision and Precision@10 of 98.15% and 97.62%, respectively. The results prove the proposed method’s effectiveness and validate its clinical application.

WWFedCBMIR: World-Wide Federated Content-Based Medical Image Retrieval.

The paper proposes a federated content-based medical image retrieval (FedCBMIR) tool that utilizes federated learning (FL) to address the challenges of acquiring a diverse medical data set for training CBMIR models. CBMIR is a tool to find the most similar cases in the data set to assist pathologists. Training such a tool necessitates a pool of whole-slide images (WSIs) to train the feature extractor (FE) to extract an optimal embedding vector. The strict regulations surrounding data sharing in hospitals makes it difficult to collect a rich data set. FedCBMIR distributes an unsupervised FE to collaborative centers for training without sharing the data set, resulting in shorter training times and higher performance. FedCBMIR was evaluated by mimicking two experiments, including two clients with two different breast cancer data sets, namely BreaKHis and Camelyon17 (CAM17), and four clients with the BreaKHis data set at four different magnifications. FedCBMIR increases the F1 score (F1S) of each client from 96% to 98.1% in CAM17 and from 95% to 98.4% in BreaKHis, with 11.44 fewer hours in training time. FedCBMIR provides 98%, 96%, 94%, and 97% F1S in the BreaKHis experiment with a generalized model and accomplishes this in 25.53 fewer hours of training.

Content-based medical image retrieval using fractional Hartley transform with hybrid features

Content-based medical image retrieval using fractional Hartley transform with hybrid features

Medical image retrieval via nearest neighbor search on pre-trained image features

Nearest neighbor search, also known as NNS, is a technique used to locate the points in a high-dimensional space closest to a given query point. This technique has multiple applications in medicine, such as searching large medical imaging databases, disease classification, and diagnosis. However, when the number of points is significantly large, the brute-force approach for finding the nearest neighbor becomes computationally infeasible. Therefore, various approaches have been developed to make the search faster and more efficient to support the applications. With a focus on medical imaging, this paper proposes DenseLinkSearch (DLS), an effective and efficient algorithm that searches and retrieves the relevant images from heterogeneous sources of medical images. Towards this, given a medical database, the proposed algorithm builds an index that consists of pre-computed links of each point in the database. The search algorithm utilizes the index to efficiently traverse the database in search of the nearest neighbor. We also explore the role of medical image feature representation in content-based medical image retrieval tasks. We propose a Transformer-based feature representation technique that outperformed the existing pre-trained Transformer-based approaches on benchmark medical image retrieval datasets. We extensively tested the proposed NNS approach and compared the performance with state-of-the-art NNS approaches on benchmark datasets and our created medical image datasets. The proposed approach outperformed the existing approaches in terms of retrieving accurate neighbors and retrieval speed. In comparison to the existing approximate NNS approaches, our proposed DLS approach outperformed them in terms of lower average time per query and ≥99% R@10 on 11 out of 13 benchmark datasets. We also found that the proposed medical feature representation approach is better for representing medical images compared to the existing pre-trained image models. The proposed feature extraction strategy obtained an improvement of 9.37%, 7.0%, and 13.33% in terms of P@5, P@10, and P@20, respectively, in comparison to the best-performing pre-trained image model. The source code and datasets of our experiments are available at https://github.com/deepaknlp/DLS.

A Diagnostic Study of Content-Based Image Retrieval Technique for Studying the CT Images of Lung Nodules and Prediction of Lung Cancer as a Biometric Tool

Content Based Medical Image Retrieval (CBMIR) can be defined as a digital image search using the contents of the images. CBMIR plays a very important part in medical applications such as retrieving CT images and more accurately diagnosing aberrant lung tissues in CT images. The Content-Based Medical Image Retrieval (CBMIR) method might aid radiotherapists in examining a patient's CT image in order to retrieve comparable pulmonary nodes more precisely by utilizing query nodes. Intending a particular query node, the CBMIR system searches a large chest CT image database for comparable nodes. The prime aim of this research is to evaluate an end-to-end method for developing a CBIR system for lung cancer diagnosis.

An Improved and Efficient Content-Based Medical Image Retrieval Technique vs State-of-Art Texture Feature Extraction Techniques

Areas in the health sector such as X-rays, Dermatology, High-resolution Computed Tomography – HRCT, Endoscopy, Radiology, Cardiology and Magnetic Resonance Imaging – MRI heavily depend on medical images for their activities hence it has become complex for managing and accessing these images from their repositories. As a matter of concern content-based medical image retrieval-CBMIR has been the system proposed by many researchers for handling access to similar medical image(s) as the input image, yet there is a caveat that has to be addressed with respect to which technique best suits CBMIR system with respect to a given performance metric. Medical images are mostly of grayscale and therefore color feature extraction techniques may not work effectively on them. Since there is no clear indication of which of the various texture feature extraction techniques is best suited for a given performance metric, this work seeks to comparatively evaluate the performance of the following state-of-the-art texture feature extraction techniques; Local Binary Pattern (LBP), Gabor Filter, Gray-Level Co-occurrence Matrix (GLCM), Haralick Descriptor, Features from Accelerated Segment Test (FAST) and a Proposed Technique using the metrics; Precision, Recall, F1-score, Mean Squared Error (MSE), Accuracy and Time. The results showed that the proposed technique is best suited for systems focusing on precision with an average precision score of 100% over 10.5k of raw medical images (dataset) using an appreciable minimum time with time complexity of 0(n).

Content-based medical image retrieval with opponent class adaptive margin loss

The increasing utilization of medical imaging technology with digital storage capabilities has facilitated the compilation of large-scale data repositories. Fast access to image samples with similar appearance to suspected cases in these repositories can help establish a consulting system for healthcare professionals, and improve diagnostic procedures while minimizing processing delays. However, manual querying of large repositories is labor intensive. Content-based image retrieval (CBIR) offers an automated solution based on quantitative assessment of image similarity based on image features in a latent space. Since conventional methods based on hand-crafted features typically show poor generalization performance, learning-based CBIR methods have received attention recently. A common framework in this domain involves classifier-guided models that are trained to detect different image classes. Similarity assessments are then performed on the features captured by the intermediate stages of the trained models. While classifier-guided methods are powerful in inter-class discrimination, they are suboptimally sensitive to within-class differences in image features. An alternative framework instead performs task-agnostic training to learn an embedding space that enforces the representational discriminability of images. Within this representational-learning framework, a powerful method is triplet-wise learning that addresses the deficiencies of point-wise and pair-wise learning in characterizing the similarity relationships between image classes. However, the traditional triplet loss enforces separation between only a subset of image samples within the triplet via a manually-set constant margin value, so it can lead to suboptimal segregation of opponent classes and limited generalization performance. To address these limitations, we introduce a triplet-learning method for automated querying of medical image repositories based on a novel Opponent Class Adaptive Margin (OCAM) loss. To maintain optimally discriminative representations, OCAM considers relationships among all image pairs within the triplet and utilizes an adaptive margin value that is automatically selected per dataset and during the course of training iterations. CBIR performance of OCAM is compared against state-of-the-art loss functions for representational learning on three public databases (gastrointestinal disease, skin lesion, lung disease). On average, OCAM shows an mAP performance of 86.30% in the KVASIR dataset, 70.30% in the ISIC 2019 dataset, and 85.57% in the X-RAY dataset. Comprehensive experiments in each application domain demonstrate the superior performance of OCAM against competing triplet-wise methods at 1.52%, classifier-guided methods at 2.29%, and non-triplet representational-learning methods at 4.56%.

Feature Extraction Method using HoG with LTP for Content-Based Medical Image Retrieval

An accurate diagnosis is significant for the treatment of any disease in its early stage. Content-Based Medical Image Retrieval (CBMIR) is used to find similar medical images in a huge database to help radiologists in diagnosis. The main difficulty in CBMIR is semantic gaps between the lower-level visual details, captured by computer-aided tools and higher-level semantic details captured by humans. Many existing methods such as Manhattan Distance, Triplet Deep Hashing, and Transfer Learning techniques for CBMIR were developed but showed lower efficiency and the computational cost was high. To solve such issues, a new feature extraction approach is proposed using Histogram of Gradient (HoG) with Local Ternary Pattern (LTP) to automatically retrieve medical images from the Contrast-Enhanced Magnetic Resonance Imaging (CE-MRI) database. Adam optimization algorithm is utilized to select features and the Euclidean measure calculates the similarity for query images. From the experimental analysis, it is clearly showing that the proposed HoG-LTP method achieves higher accuracy of 98.8%, a sensitivity of 98.5%, and a specificity of 99.416%, which is better when compared to the existing Random Forest (RF) method which displayed an accuracy, sensitivity, and specificity of 81.1%, 81.7% and 90.5% respectively.

An Accurate and Privacy-Preserving Retrieval Scheme Over Outsourced Medical Images

With the rapid advancement in medical imaging techniques, Content-Based (medical) Image Retrieval (CBIR), which can assist in disease diagnosis, has gained much attention in both academia and industry. However, due to patients’ sensitive information involved in medical images, privacy-preserving CBIR is a challenge worth exploiting. Though several privacy-preserving CBIR schemes have been put forth, they can only resist known-background attack (KBA), and do not suffice for protecting the image privacy in outsourced settings. In this article, aiming at the above challenge, we first design a novel Privacy-preserving Mahalanobis Distance Comparison (PMDC) method to improve the accuracy of medical images retrieval. Then, combined with the Mahalanobis distance based Fuzzy C-Means (FCM-M) algorithm, a scheme named TAMMIE is proposed to achieve accurate and privacy-preserving medical image retrieval over encrypted data. With TAMMIE, an image owner can securely outsource the images and indexes to a cloud server, and query users can request retrieval services from the cloud server while keeping their queries private. Detailed security analysis shows that our proposed schemes are secure under the attack stronger than KBA. Furthermore, thorough empirical experiments conducted on two real-world and one synthetic datasets also demonstrate the efficiency of TAMMIE.

RbQE: An Efficient Method for Content-Based Medical Image Retrieval Based on Query Expansion

Systems for retrieving and managing content-based medical images are becoming more important, especially as medical imaging technology advances and the medical image database grows. In addition, these systems can also use medical images to better grasp and gain a deeper understanding of the causes and treatments of different diseases, not just for diagnostic purposes. For achieving all these purposes, there is a critical need for an efficient and accurate content-based medical image retrieval (CBMIR) method. This paper proposes an efficient method (RbQE) for the retrieval of computed tomography (CT) and magnetic resonance (MR) images. RbQE is based on expanding the features of querying and exploiting the pre-trained learning models AlexNet and VGG-19 to extract compact, deep, and high-level features from medical images. There are two searching procedures in RbQE: a rapid search and a final search. In the rapid search, the original query is expanded by retrieving the top-ranked images from each class and is used to reformulate the query by calculating the mean values for deep features of the top-ranked images, resulting in a new query for each class. In the final search, the new query that is most similar to the original query will be used for retrieval from the database. The performance of the proposed method has been compared to state-of-the-art methods on four publicly available standard databases, namely, TCIA-CT, EXACT09-CT, NEMA-CT, and OASIS-MRI. Experimental results show that the proposed method exceeds the compared methods by 0.84%, 4.86%, 1.24%, and 14.34% in average retrieval precision (ARP) for the TCIA-CT, EXACT09-CT, NEMA-CT, and OASIS-MRI databases, respectively.

A dynamic programming approach for accurate content-based retrieval of ordinary and nano-scale medical images

A dynamic programming approach for accurate content-based retrieval of ordinary and nano-scale medical images

A dynamic programming approach for accurate content-based retrieval of ordinary and nano-scale medical images

A dynamic programming approach for accurate content-based retrieval of ordinary and nano-scale medical images

Pre-trained convolution neural networks models for content-based medical image retrieval

Content-based image retrieval (CBIR) is a recent method used to retrieve different types of images from repositories. The traditional content-based medical image retrieval (CBMIR) methods commonly used low-level image representation features extracted from color, texture, and shape image descriptors. Since most of these CBMIR systems depend mainly on the extracted features, the methods used in the feature extraction phase are more important. Features extraction methods, which generate inaccurate features, lead to very poor performance retrieval because of semantic gap widening. Hence, there is high demand for independent domain knowledge features extraction methods, which have automatic learning capabilities from input images. Pre-trained deep convolution neural networks (CNNs), the recent generation of deep learning neural networks, could be used to extract expressive and accurate features. The main advantage of these pre-trained CNNs models is the pre-training process for huge image data of thousands of different classes, and their knowledge after the training process could easily be transferred. There are many successful models of pre-trained CNNs models used in the area of medical image retrieval, image classification, and object recognition. This study utilizes two of the most known pre-trained CNNs models; ResNet18 and SqueezeNet for the offline feature extraction stage. Additionally, the highly accurate features extracted from medical images are used for the CBMIR method of medical image retrieval. This study uses two popular medical image datasets; Kvasir and PH2 to show that the proposed methods have good retrieval results. The retrieval performance evaluation measures of our proposed method have average precision of 97.75% and 83.33% for Kvasir and PH2 medical images respectively, and outperform some of the state-of-the-art methods in this field of study because these pre-trained CNNs have well trained layers among a huge number of image types. Finally, intensive statistical analysis shows that the proposed ResNet18-based retrieval method has the best performance for enhancing both recall and precision measures for both medical images.

Computer-aided diagnosis through medical image retrieval in radiology

Currently, radiologists face an excessive workload, which leads to high levels of fatigue, and consequently, to undesired diagnosis mistakes. Decision support systems can be used to prioritize and help radiologists making quicker decisions. In this sense, medical content-based image retrieval systems can be of extreme utility by providing well-curated similar examples. Nonetheless, most medical content-based image retrieval systems work by finding the most similar image, which is not equivalent to finding the most similar image in terms of disease and its severity. Here, we propose an interpretability-driven and an attention-driven medical image retrieval system. We conducted experiments in a large and publicly available dataset of chest radiographs with structured labels derived from free-text radiology reports (MIMIC-CXR-JPG). We evaluated the methods on two common conditions: pleural effusion and (potential) pneumonia. As ground-truth to perform the evaluation, query/test and catalogue images were classified and ordered by an experienced board-certified radiologist. For a profound and complete evaluation, additional radiologists also provided their rankings, which allowed us to infer inter-rater variability, and yield qualitative performance levels. Based on our ground-truth ranking, we also quantitatively evaluated the proposed approaches by computing the normalized Discounted Cumulative Gain (nDCG). We found that the Interpretability-guided approach outperforms the other state-of-the-art approaches and shows the best agreement with the most experienced radiologist. Furthermore, its performance lies within the observed inter-rater variability.

A Secure Medical Image Transmission System Based On 2D Logistic Map and Diffie-Hellman Key Exchange Mechanisms

With the tremendous growth of searchable visual media, the content-based medical image retrieval and computer-aided diagnosis systems have become popular in recent years to improve knowledge and provide facilities for radiologists. Medical images transferred throughout public networks demand a mechanism that guarantees image privacy, ownership and source of origin reliability, and image integrity verification. For this reason, secure image retrieval and diagnosis scheme have been given considerable interest due to users’ security concerns. This work proposed a secure framework based on a two-dimensional (2D) chaotic map with Diffie–Hellman key exchange protocols to ensure patient information privacy and security. Consequently, from a security and protection perspective, the objective is to provide a privacy procedure for medical image retrieval systems through image encryption technique combined with a secure key exchange procedure to minimize the possibility of secret key interception by an unauthorized person. Simulation results and security analysis show that the suggested technique could protect images with minimal time complexity and a high level of security while also resisting numerous attacks.

Content-Based Medical Image Retrieval System for Skin Melanoma Diagnosis Based on Optimized Pair-Wise Comparison Approach.

Medical image analysis for perfect diagnosis of disease has become a very challenging task. Due to improper diagnosis, required medical treatment may be skipped. Proper diagnosis is needed as suspected lesions could be missed by the physician's eye. Hence, this problem can be settled up by better means with the investigation of similar case studies present in the healthcare database. In this context, this paper substantiates an assistive system that would help dermatologists for accurate identification of 23 different kinds of melanoma. For this, 2300 dermoscopic images were used to train the skin-melanoma similar image search system. The proposed system uses feature extraction by assigning dynamic weights to the low-level features based on the individual characteristics of the searched images. Optimal weights are obtained by the newly proposed optimized pair-wise comparison (OPWC) approach. The uniqueness of the proposed approach is that it provides the dynamic weights to the features of the searched image instead of applying static weights. The proposed approach is supported by analytic hierarchy process (AHP) and meta-heuristic optimization algorithms such as particle swarm optimization (PSO), JAYA, genetic algorithm (GA), and gray wolf optimization (GWO). The proposed approach has been tested with images of 23 classes of melanoma and achieved significant precision and recall. Thus, this approach of skin melanoma image search can be used as an expert assistive system to help dermatologists/physicians for accurate identification of different types of melanomas.

Optimized Deep-Neural Network for Content-based Medical Image Retrieval in a Brownfield IoMT Network

In this paper, a brownfield Internet of Medical Things network is introduced for imaging data that can be easily scaled out depending on the objectives, functional requirements, and the number of facilities and devices connected to it. This is further used to develop a novel Content-based Medical Image Retrieval framework. The developed framework uses DenseNet-201 architecture for generating the image descriptors. Then for classification, the optimized Deep Neural Network model has been configured through a population-based metaheuristic Differential Evolution. Differential Evolution iteratively performs the joint optimization of hyperparameters and architecture of Deep Neural Networks. The competence of the proposed model is validated on three publicly available datasets: Brain Tumor MRI dataset, Covid-19 Radiography database, and Breast Cancer MRI dataset, and by comparing it with selected models over different aspects of performance evaluation. Results show that the convergence rate of the proposed framework is very fast, and it achieves at least 97.28% accuracy across all the models.