Medical Image Retrieval Research Articles

Deep hashing and attention mechanism-based image retrieval of osteosarcoma scans for diagnosis of bone cancer

BackgroundDue to its intricate nature and substantial data size, microscopic image data of osteosarcoma often present a significant obstacle to the effectiveness of conventional image retrieval methods. Therefore, this study investigates a new approach for medical image retrieval using advanced deep hashing techniques and attention mechanisms to address these challenges more effectively. MethodThe proposed algorithm significantly improves osteosarcoma cell microscopic image retrieval efficiency and accuracy using deep hashing and attention mechanisms. Image preprocessing includes adaptive histogram equalization and dataset augmentation to enhance quality and diversity. Feature extraction employs the WRN-AM model to map high-dimensional features to a low-dimensional hash code space, improving retrieval efficiency. Finally, similarity matching via Hamming distance allows rapid and precise identification of similar images. ResultsThe study shows notable advancements: the WRN-AM model achieves 93.2% classification accuracy and 97.09% mAP using 64-bit hash codes. These findings underscore the technique’s effective performance in extracting and categorizing diverse microscopic cell data efficiently and reliably. ConclusionsThis innovative approach provides a robust solution for retrieving and classifying microscopic data of osteosarcoma cells and other cell types, speeding up clinical diagnosis and medical research. It facilitates quicker access and analysis of patient image data, enhancing diagnostic precision and treatment planning for healthcare professionals. Concurrently, it supports researchers in leveraging medical image data more efficiently, fostering progress and innovation in the medical field.

ViTH-RFG: Vision Transformer Hashing With Residual Fuzzy Generation for Targeted Attack in Medical Image Retrieval

ViTH-RFG: Vision Transformer Hashing With Residual Fuzzy Generation for Targeted Attack in Medical Image Retrieval

SMedIR: secure medical image retrieval framework with ConvNeXt-based indexing and searchable encryption in the cloud

The security and privacy of medical images are crucial due to their sensitive nature and the potential for severe consequences from unauthorized modifications, including data breaches and inaccurate diagnoses. This paper introduces a method for lossless medical image retrieval from encrypted images stored on third-party clouds. The proposed approach employs a symmetric integrity-centric image encryption scheme, leveraging multiple chaotic maps and cryptographic hash techniques, to ensure lossless image reconstruction. Medical images are first encrypted by the image owners and converted into hashcodes encapsulating essential features using a deep hashing technique with the ConvNeXt network as the backbone in parallel. To ensure index privacy, these hashcodes are encrypted in a searchable manner. The encrypted medical images, along with a secure index, are subsequently uploaded to cloud storage. Authorized medical image users can request similar medical images for diagnostic purposes by submitting a query image, from which a search trapdoor is generated and sent to the cloud. The retrieval process involves a secure similar image search over the encrypted indexes, followed by decryption along with integrity verification of the retrieved images. The proposed method has been rigorously tested on three standard medical datasets, demonstrating an improvement of 5-20% in retrieval accuracy compared to standard baselines. Formal security analysis and experimental results indicate that the proposed scheme offers enhanced security and retrieval accuracy, making it an effective solution for the encrypted storage and secure retrieval of medical image data.

A comprehensive guide to content-based image retrieval algorithms with visualsift ensembling.

Content-based image retrieval (CBIR) systems are vital for managing the large volumes of data produced by medical imaging technologies. They enable efficient retrieval of relevant medical images from extensive databases, supporting clinical diagnosis, treatment planning, and medical research. This study aims to enhance CBIR systems' effectiveness in medical image analysis by introducing the VisualSift Ensembling Integration with Attention Mechanisms (VEIAM). VEIAM seeks to improve diagnostic accuracy and retrieval efficiency by integrating robust feature extraction with dynamic attention mechanisms. VEIAM combines Scale-Invariant Feature Transform (SIFT) with selective attention mechanisms to emphasize crucial regions within medical images dynamically. Implemented in Python, the model integrates seamlessly into existing medical image analysis workflows, providing a robust and accessible tool for clinicians and researchers. The proposed VEIAM model demonstrated an impressive accuracy of 97.34% in classifying and retrieving medical images. This performance indicates VEIAM's capability to discern subtle patterns and textures critical for accurate diagnostics. By merging SIFT-based feature extraction with attention processes, VEIAM offers a discriminatively powerful approach to medical image analysis. Its high accuracy and efficiency in retrieving relevant medical images make it a promising tool for enhancing diagnostic processes and supporting medical research in CBIR systems.

Integration of optimal feature descriptors and optimized CapsuleNets for medical image retrieval and classification using hybrid optimization algorithm

Integration of optimal feature descriptors and optimized CapsuleNets for medical image retrieval and classification using hybrid optimization algorithm

Development of Computed Tomography Quality Control Training Application Model Based on the American College of Radiology Phantom Integrated Website

The use of Viewdex as a radiographic image evaluation tool has limited flexibility for students, especially in the complicated process of installation and input of DICOM ACR image results. These limitations can slow down the learning process and reduce the effectiveness of education on radiological image evaluation. This research aims to develop a web-integrated ACR Phantom-based CT quality control training application, which can overcome the weaknesses of Viewdex and improve efficiency and user experience. The research method used is Research and Development (R&D), which consists of several stages: literature study, product development, field testing, evaluation, and product revision. The application developed simplifies the process of DICOM data import, question coding, and automatic evaluation, improving operational efficiency. Validation of information technology and QA/QC experts shows that the application meets high standards in various aspects with 100% validity. Performance evaluation through pre-test and post-test showed a significant improvement in participants' knowledge and skills, with the Wilcoxon test showing a p-value of 0.005. The developed application has proven to be effective and efficient in improving the usability and learning experience of students, overcoming the weaknesses of Viewdex. Its advanced features enable efficient storage, retrieval, and management of medical images, as well as access and analysis of medical data.

A New Approach for Effective Retrieval of Medical Images: A Step towards Computer-Assisted Diagnosis.

The biomedical imaging field has grown enormously in the past decade. In the era of digitization, the demand for computer-assisted diagnosis is increasing day by day. The COVID-19 pandemic further emphasized how retrieving meaningful information from medical repositories can aid in improving the quality of patient's diagnosis. Therefore, content-based retrieval of medical images has a very prominent role in fulfilling our ultimate goal of developing automated computer-assisted diagnosis systems. Therefore, this paper presents a content-based medical image retrieval system that extracts multi-resolution, noise-resistant, rotation-invariant texture features in the form of a novel pattern descriptor, i.e., MsNrRiTxP, from medical images. In the proposed approach, the input medical image is initially decomposed into three neutrosophic images on its transformation into the neutrosophic domain. Afterwards, three distinct pattern descriptors, i.e., MsTrP, NrTxP, and RiTxP, are derived at multiple scales from the three neutrosophic images. The proposed MsNrRiTxP pattern descriptor is obtained by scale-wise concatenation of the joint histograms of MsTrP×RiTxP and NrTxP×RiTxP. To demonstrate the efficacy of the proposed system, medical images of different modalities, i.e., CT and MRI, from four test datasets are considered in our experimental setup. The retrieval performance of the proposed approach is exhaustively compared with several existing, recent, and state-of-the-art local binary pattern-based variants. The retrieval rates obtained by the proposed approach for the noise-free and noisy variants of the test datasets are observed to be substantially higher than the compared ones.

Artificial intelligence in cardiovascular imaging and intervention.

Recent progress in artificial intelligence (AI) includes generative models, multimodal foundation models, and federated learning, which enable awide spectrum of novel exciting applications and scenarios for cardiac image analysis and cardiovascular interventions. The disruptive nature of these novel technologies enables concurrent text and image analysis by so-called vision-language transformer models. They not only allow for automatic derivation of image reports, synthesis of novel images conditioned on certain textual properties, and visual questioning and answering in an oral or written dialogue style, but also for the retrieval of medical images from alarge database based on adescription of the pathology or specifics of the dataset of interest. Federated learning is an additional ingredient in these novel developments, facilitating multi-centric collaborative training of AI approaches and therefore access to large clinical cohorts. In this review paper, we provide an overview of the recent developments in the field of cardiovascular imaging and intervention and offer afuture outlook.

Content based COVID-19 image retrieval system using local histogram equalization and deep convolutional neural network

Doctors play a critical role in interpreting medical images as part of their core responsibilities. They need to find comparable examples that can assist in making informed decisions, especially when encountering ambiguous visuals. Traditionally, Systems such as content-based medical image retrieval (CBMIR) have been used for this. The proposed method employs a novel technique, local histogram equalization (LHE) for preprocessing, transfer learning-based convolutional neural network to extract the representative features with Manhattan and Euclidean distance metrics to assess how similar the query image and database image are to one another. This model is trained on a standard dataset namely Chest X-Ray images. Top-k, Precision and Recall measure is employed to assess system performance. From the results, the suggested enhanced convolutional neural network (CNN) model demonstrates significantly superior performance in the top 10 retrieval rates of 97.13% for coronavirus disease 2019 (COVID-19), 96.84% for normal, 82.63% for pneumonia-bacterial, and 81.72% for pneumonia-viral and precision@recall10 of 93.14% for COVID-19, 91.88% for normal, 77.84% for pneumonia-bacterial and 74.71% for pneumonia-viral.

Multiple semantic X-ray medical image retrieval using efficient feature vector extracted by FPN.

Content-based medical image retrieval (CBMIR) has become an important part of computer-aided diagnostics (CAD) systems. The complex medical semantic information inherent in medical images is the most difficult part to improve the accuracy of image retrieval. Highly expressive feature vectors play a crucial role in the search process. In this paper, we propose an effective deep convolutional neural network (CNN) model to extract concise feature vectors for multiple semantic X-ray medical image retrieval. We build a feature pyramid based CNN model with ResNet50V2 backbone to extract multi-level semantic information. And we use the well-known public multiple semantic annotated X-ray medical image data set IRMA to train and test the proposed model. Our method achieves an IRMA error of 32.2, which is the best score compared to the existing literature on this dataset. The proposed CNN model can effectively extract multi-level semantic information from X-ray medical images. The concise feature vectors can improve the retrieval accuracy of multi-semantic and unevenly distributed X-ray medical images.

Deep Attention Fusion Hashing (DAFH) Model for Medical Image Retrieval.

To address this, we propose a novel deep learning-based hashing model, the Deep Attention Fusion Hashing (DAFH) model, which integrates advanced attention mechanisms with medical imaging data. The DAFH model enhances retrieval performance by integrating multi-modality medical imaging data and employing attention mechanisms to optimize the feature extraction process. Utilizing multimodal medical image data from the Cancer Imaging Archive (TCIA), this study constructed and trained a deep hashing network that achieves high-precision classification of various cancer types. At hash code lengths of 16, 32, and 48 bits, the model respectively attained Mean Average Precision (MAP@10) values of 0.711, 0.754, and 0.762, highlighting the potential and advantage of the DAFH model in medical image retrieval. The DAFH model demonstrates significant improvements in the efficiency and accuracy of medical image retrieval, proving to be a valuable tool in clinical settings.

VTHSC-MIR: Vision Transformer Hashing with Supervised Contrastive learning based medical image retrieval

In past few years, deep learning based medical image analysis technologies have significantly improved computer-assisted tasks like detecting, diagnosing, and predicting medical outcomes. The monitoring and diagnosis of ailments such as cancer and COVID-19 rely primarily on retrieving medical images. As the medical databases size is increasing rapidly it causes difficulty in managing and querying them. The main challenge lies in achieving superior retrieval performance while handling the intricacies of medical imaging datasets. To tackle this, we propose a novel end-to-end trainable framework for medical image retrieval using a vision transformer hashing with Supervised Contrastive (VTHSC) learning. Leveraging the self-attention mechanism of transformers and supervised contrastive loss, our model surpasses existing hashing-based retrieval methods. Focusing on the most recent COVID-19 and breast cancer datasets, we aim to efficiently retrieve relevant medical images from large datasets. Using the ImageNet-pretrained ViT as its backbone network, the VTHSC model incorporates a hashing head combined with a supervised contrastive loss and employs joint loss optimization. The VTHSC model is subsequently fine- tuned for a retrieval task, employing four different hashing frameworks: Deep Supervised Hashing (DSH), GreedyHash, Improved Deep Hashing Network (IDHN), and Deep Polarized Network (DPN). Our model achieves outstanding Mean Average Precision (MAP) scores of 98.9 for the BreakHis dataset and 96.03 for the COVID dataset. Notably, the VTHSC model surpasses several competing hashing-based retrieval methods by a substantial gain in terms of performance across various metrics such as precision, recall, and Top-6 retrieved images retrieval performance on the two benchmark medical image datasets.

Enhancing Medical Image Retrieval with UMLS-Integrated CNN-Based Text Indexing.

In recent years, Convolutional Neural Network (CNN) models have demonstrated notable advancements in various domains such as image classification and Natural Language Processing (NLP). Despite their success in image classification tasks, their potential impact on medical image retrieval, particularly in text-based medical image retrieval (TBMIR) tasks, has not yet been fully realized. This could be attributed to the complexity of the ranking process, as there is ambiguity in treating TBMIR as an image retrieval task rather than a traditional information retrieval or NLP task. To address this gap, our paper proposes a novel approach to re-ranking medical images using a Deep Matching Model (DMM) and Medical-Dependent Features (MDF). These features incorporate categorical attributes such as medical terminologies and imaging modalities. Specifically, our DMM aims to generate effective representations for query and image metadata using a personalized CNN, facilitating matching between these representations. By using MDF, a semantic similarity matrix based on Unified Medical Language System (UMLS) meta-thesaurus, and a set of personalized filters taking into account some ranking features, our deep matching model can effectively consider the TBMIR task as an image retrieval task, as previously mentioned. To evaluate our approach, we performed experiments on the medical ImageCLEF datasets from 2009 to 2012. The experimental results show that the proposed model significantly enhances image retrieval performance compared to the baseline and state-of-the-art approaches.

Performance evaluation of attention-deep hashing based medical image retrieval in brain MRI datasets

BackgroundBrain MRI images pose significant challenges due to their complexity and voluminous data, which often hinder the accuracy of traditional image retrieval methods. In response, this research delves into a novel medical image retrieval approach grounded in attention mechanisms and deep hashing techniques. MethodologyThe study enhances the network's edge perception capability by incorporating a dual mixed attention module (CBAM-MA dual attention) into the convolutional neural network architecture. This addition optimally captures edge saliency and distributional features by introducing an extra attention layer atop CBAM. Furthermore, a novel triple loss function is introduced, amalgamating Euclidean and cosine distances. This fusion exploits the strengths of both distance metrics to comprehensively consider sample geometric attributes, thereby generating more robust and expressive feature representations crucial for preserving categorical details. ResultsExperimental evaluations showcase significant performance advancements in MRI image retrieval tasks using the proposed method. The average hit rate stands at 0.7783, average precision at 0.752246, and average inverse rank at 0.956721. These metrics collectively underscore the method's efficacy in enhancing the quality and expediting the diagnosis of brain diseases. ConclusionThe research underscores the critical role of attention and deep hashing techniques in augmenting the accuracy and efficiency of medical image retrieval, particularly in the context of brain MRI images. The proposed method's notable performance improvements pave the way for enhanced diagnostic capabilities, signaling a significant step forward in improving healthcare outcomes related to brain diseases.

SEMANTIC SEGMENTATION AND CONTENT-BASED RETRIEVAL IN MULTIMEDIA IMAGE DATABASES

Brain tumors, particularly gliomas, pose a significant threat to global health, necessitating accurate and efficient diagnostic methods. Magnetic Resonance Imaging (MRI) serves as a crucial tool for diagnosing glioma grades, but interpretation is subject to variability, hindering treatment planning. Intra and inter-observer variability in radiological image interpretation impede effective therapeutic strategies for brain tumor patients. Accessing relevant images from vast medical databases for comparison and treatment planning is cumbersome and time-consuming. This paper proposes a Content-Based Medical Image Retrieval (CBIR) system utilizing Convolutional Neural Network (CNN)-based feature extraction, specifically employing the AlexNet architecture. The system employs KNN clustering for indexing the feature map database and implements Gain-based feature selection to reduce feature vector dimensionality. The proposed system underwent evaluation using BraTS 2018 and 2020 datasets with five-fold cross-validation. Achieving state-of-the-art performance, the system demonstrated a mean Average Precision of 98% and Precision of 97%, showcasing its efficacy in accurately retrieving similar pathological MRI brain images.

An Investigation of Distance Measures for Development of Effective Content Based Tumor Image Retrieval System

Background/Objectives: The MRI has proven to be extremely effective in detecting tumors, with millions of images created each day throughout the world. To find similar images from a vast collection, Content-Based Tumor Image Retrieval (CBTIR) technology has been used to analysis the medical image. In the traditional retrieval methods, retrieving a similar image from the large database is crucial task. To overcome this issue we developed deep learning based retrieval method. Methods: This research offers a retrieval approach based on predefined ResNet models for quick and accurate image retrieval. We tested various prominent ResNet models with different distance similarity metrics, and the best option was determined by this work. Findings: After the various evaluation of ResNet models with varied distance measures on the CE-MRI data set, ResNet50 model applied with Hamming distance yields 99.33% of retrieval precision. Novelty: This work used predefined ResNet models with the combination of Distance similarity metrics to achieve more accurate results on medical image retrieval compared to the other conventional methods. Keywords: Content Based Image Retrieval, Tumor Retrieval, Hamming Distance, Euclidean Distance, Minkowski

Advanced Image Processing Techniques for Medical Image Retrieval Using Visual Features and Distance Measures

Recently, there has been a rapid increase in the volume of medical image repositories because of the maximized utilization of digitized image data in hospitals. As a result, there is a complexity in querying and handling such large databases, which has led to the development of a novel Content-Based Medical Image Retrieval (CBMIR) technique. In this work, the CBMIR technique is developed using three types of visual features (i.e., color, texture and shape) along with 12 distance measures, and is optimized with the Crossover-Gravitational Search Algorithm (CRO-GSA). For ease, we named this technique Content-Based Medical Image Retrieval using Visual features with CRO-GSA (CBMIRVC). The initial step of our CBMIRVC technique is to extract three types of visual features from the images. Then, each type of feature is employed with a suitable distance measure, which is used for the computation of image similarities between the medical images in the database and a provided query image. The optimization of our proposed CBMIRVC technique is done by the CRO-GSA algorithm. This algorithm optimizes the CBMIRVC technique by identifying the optimal combinations of visual features and similarity measures. Additionally, optimal weights are computed in accordance with the three types of features for the three similarities. Experimental validations were done using retrieved cancer-related and endoscopic color images from databases holding numerous image categories. The experimental outcome shows that our proposed CBMIRVC technique outperforms other conventional image retrieval techniques by effectively retrieving similar images from large databases.

APPLICATION OF NEW RETRIEVAL METHODS IN MICROSCOPIC MEDICAL IMAGES AND CT IMAGES

Medical image retrieval is of great significance for forming correct medical judgments during the diagnosis and treatment process. In response to the fact that there are rich types of features in medical images, this paper constructs a multi-feature fusion model and applies it to medical image retrieval. This multi-feature fusion model utilizes three features. Color information is characterized using three different moments, texture information is obtained using the LBP (Local Binary Pattern) operator, and shape information is described using Hu moments. After collecting three features of medical images, they are fused into a similarity measure model through a self-set weight structure for comparison in medical image retrieval. Experiments show that our method has satisfactory image retrieval result for medical images such as endoscopic images and computed tomography (CT) images. Despite the continuous improvement of recall indicators, the proposed method still has high precision.

A DEEP LEARNING TECHNIQUE FOR EFFICIENT MULTIMEDIA FOR DATA COMPRESSION

Medical image compression plays a pivotal role in efficient data storage and transmission, crucial for modern healthcare systems. This research proposes a convolutional transfer learning technique scheme tailored for multimedia data compression, specifically targeting medical images. In the background, the growing volume of medical imaging data and the demand for efficient storage and transmission underscore the need for innovative compression methods. Leveraging transfer learning from pre-trained convolutional neural networks (CNNs) designed for image recognition tasks, our methodology optimizes the compression process for medical images. The proposed scheme utilizes a pre-trained CNN’s feature extraction capabilities to capture relevant patterns in medical images, followed by fine-tuning on a specialized dataset. This approach capitalizes on the inherent ability of CNNs to learn hierarchical representations, enhancing the compression model’s adaptability to medical imaging nuances. The contribution of this research lies in the development of a tailored transfer learning scheme that effectively balances generic feature extraction and domain-specific adaptation for medical images. Results demonstrate significant improvements in compression efficiency, preserving diagnostic information while achieving substantial data reduction. The proposed scheme showcases promise for enhancing medical image storage, transmission, and retrieval systems, contributing to the advancement of healthcare technology.

Multiple Pseudo-Siamese Network with Supervised Contrast Learning for Medical Multi-modal Retrieval

Medical multi-modal retrieval aims to provide doctors with similar medical images from different modalities, which can greatly promote the efficiency and accuracy of clinical diagnosis. However, most existing medical retrieval methods hardly support the retrieval of multi-modal medical images, i.e., the number of modalities is greater than 2, and just convert retrieval to classification or clustering. It futilely breaks the gap between the visual information and the semantic information in different medical image modalities. To solve the problem, a S upervised C ontrast L earning method based on a M ultiple P seudo- S iamese network (SCL-MPS) is proposed for multi-modal medical image retrieval. In order to make the samples with semantic similarity close neighbors on Riemann manifold, the multiple constraints based on semantic consistency and modal invariance are designed in different forward stages of SCL-MPS. We theoretically demonstrate the feasibility of the designed constraints. Finally, experiments on four benchmark datasets (ADNI1, ADNI2, ADNI3, and OASIS3) show that SCL-MPS achieves state-of-the-art performance compared to 15 retrieval methods. Especially, SCL-MPS achieves a 100% mAP score in medical cross-modal retrieval on ADNI1.