Medical Image Retrieval System Research Articles

Sketch-based semantic retrieval of medical images

The volume of medical images stored in hospitals is rapidly increasing; however, the utilization of these accumulated medical images remains limited. Existing content-based medical image retrieval (CBMIR) systems typically require example images, leading to practical limitations, such as the lack of customizable, fine-grained image retrieval, the inability to search without example images, and difficulty in retrieving rare cases. In this paper, we introduce a sketch-based medical image retrieval (SBMIR) system that enables users to find images of interest without the need for example images. The key concept is feature decomposition of medical images, which allows the entire feature of a medical image to be decomposed into and reconstructed from normal and abnormal features. Building on this concept, our SBMIR system provides an easy-to-use two-step graphical user interface: users first select a template image to specify a normal feature and then draw a semantic sketch of the disease on the template image to represent an abnormal feature. The system integrates both types of input to construct a query vector and retrieves reference images. For evaluation, ten healthcare professionals participated in a user test using two datasets. Consequently, our SBMIR system enabled users to overcome previous challenges, including image retrieval based on fine-grained image characteristics, image retrieval without example images, and image retrieval for rare cases. Our SBMIR system provides on-demand, customizable medical image retrieval, thereby expanding the utility of medical image databases.

Performance Evaluation of State-of-the-Art Texture Feature Extraction Techniques on Medical Imagery Tasks

Interpreting medical images is certainly a complex task which requires extensive knowledge [1]. According to Computer Aided Diagnosis (CAD) serves as a second opinion that will help radiologists in diagnosis and on the other hand Content-based Image Retrieval uses visual content to help users browse, search and retrieve similar medical images from a database based on the user’s interest [2-4]. The competency of the CBMIR system depends on feature extraction methods [5]. The textural features are very important to determine the content of a medical image. Textural features provide scenic depth, the spatial distribution of tonal variation, and surface orientation [6]. Therefore, this study seeks to compare and evaluate some of the hand-crafted texture feature extraction techniques in CBMIR. This is to help those concerned in enhancing CBIR systems to make informed decisions concerning the selection of the best textural feature extraction techniques. Since there is no clear indication of which of the various texture feature extraction techniques is best suited for a given performance metric when considering which of the techniques to choose for a particular study in CBMIR systems. The objective of this work, therefore, is to comparatively evaluate the performance of the following texture feature extraction techniques; Local Binary Pattern (LBP), Gabor Filter, Gray-Level Co-occurrence Matrix (GLCM), Haralick Descriptor, Features from Accelerated Segment Test (FAST) and Features from Accelerated Segment Test and Binary Robust Independent Elementary Features (FAST &BRIEF) using the metrics; precision, recall, f1-score, mean squared error (MSE), accuracy and time. These techniques are coupled with specific similarity measure to obtain results. The results showed that LBP, Haralick Descriptor, FAST, and GLCM had the best results in terms of (Precision and Accuracy), Time, F1-Score, and Recall respectively. LBP had 82.05% and 88.23% scores for precision and accuracy respectively. The following scores represent the performance of the Haralick descriptor, FAST, and GLCM models respectively; 0.88s, 38.7%, and 44.82%. These test scores are obtained from datasets ranging from 1k-10.5k. Aside from LBP outperforming the other 5 models mentioned, it still outperformed the following proposed models ' [7]’, ‘Tamura texture feature and wavelet transform combined with Hausdorff distance - [8]’, ‘ [9]’ in terms of (precision, accuracy, and recall) and (precision and recall) respectively and probably f1-score (since f1-score is the weighted average of precision and recall). It is believed that an ensemble of LBP, Haralick descriptors, and Support Vector Machine (SVM) can represent a robust system for both medical image retrieval and classification.

Computer-aided diagnosis through medical image retrieval in radiology

Currently, radiologists face an excessive workload, which leads to high levels of fatigue, and consequently, to undesired diagnosis mistakes. Decision support systems can be used to prioritize and help radiologists making quicker decisions. In this sense, medical content-based image retrieval systems can be of extreme utility by providing well-curated similar examples. Nonetheless, most medical content-based image retrieval systems work by finding the most similar image, which is not equivalent to finding the most similar image in terms of disease and its severity. Here, we propose an interpretability-driven and an attention-driven medical image retrieval system. We conducted experiments in a large and publicly available dataset of chest radiographs with structured labels derived from free-text radiology reports (MIMIC-CXR-JPG). We evaluated the methods on two common conditions: pleural effusion and (potential) pneumonia. As ground-truth to perform the evaluation, query/test and catalogue images were classified and ordered by an experienced board-certified radiologist. For a profound and complete evaluation, additional radiologists also provided their rankings, which allowed us to infer inter-rater variability, and yield qualitative performance levels. Based on our ground-truth ranking, we also quantitatively evaluated the proposed approaches by computing the normalized Discounted Cumulative Gain (nDCG). We found that the Interpretability-guided approach outperforms the other state-of-the-art approaches and shows the best agreement with the most experienced radiologist. Furthermore, its performance lies within the observed inter-rater variability.

A comprehensive review of content-based image retrieval systems using deep learning and hand-crafted features in medical imaging: Research challenges and future directions

Computer-based medical image retrieval (CBMIR) system helps practitioners to enhance their diagnostic abilities, speeds up accurate diagnosis, and minimizes intra-and inter-observer variability. We investigate in-depth insights on content-based image retrieval systems over the past 20 years, to enable improvements with an imperative feature set to enhance the performance of the proposed CBMIR. After a comprehensive survey, we have made some inferences from the literature survey which is beneficial for researchers and medical practitioners. Existing studies, consider patch sizes of 32 only where 16 and 64 are ignored. We conduct experiments using 16, 32, and 64 patch sizes and two retrieval approaches i.e. simple distance-based retrieval (SDR) and customized query-based approach (CQA). SDR exhibits high inter-class ambiguity, in which some of the retrieved images are from different classes posing similar feature values, which negatively impacts the retrieval performance. To tackle this, a framework has been proposed based on CQA-CBMIR which utilizes wavelet-based Riesz features and improves the retrieval outcomes. The proposed framework is compared with the state-of-the-art (SOTA) methods and obtained f1-score and accuracy i.e. 86.5% and 86.8% respectively. Furthermore, this study illustrates some of the unresolved issues in the literature that demand future researchers' utmost attention to investigate new pathways in this discipline.

A Secure Medical Image Transmission System Based On 2D Logistic Map and Diffie-Hellman Key Exchange Mechanisms

With the tremendous growth of searchable visual media, the content-based medical image retrieval and computer-aided diagnosis systems have become popular in recent years to improve knowledge and provide facilities for radiologists. Medical images transferred throughout public networks demand a mechanism that guarantees image privacy, ownership and source of origin reliability, and image integrity verification. For this reason, secure image retrieval and diagnosis scheme have been given considerable interest due to users’ security concerns. This work proposed a secure framework based on a two-dimensional (2D) chaotic map with Diffie–Hellman key exchange protocols to ensure patient information privacy and security. Consequently, from a security and protection perspective, the objective is to provide a privacy procedure for medical image retrieval systems through image encryption technique combined with a secure key exchange procedure to minimize the possibility of secret key interception by an unauthorized person. Simulation results and security analysis show that the suggested technique could protect images with minimal time complexity and a high level of security while also resisting numerous attacks.

Content-Based Medical Image Retrieval System for Skin Melanoma Diagnosis Based on Optimized Pair-Wise Comparison Approach.

Medical image analysis for perfect diagnosis of disease has become a very challenging task. Due to improper diagnosis, required medical treatment may be skipped. Proper diagnosis is needed as suspected lesions could be missed by the physician's eye. Hence, this problem can be settled up by better means with the investigation of similar case studies present in the healthcare database. In this context, this paper substantiates an assistive system that would help dermatologists for accurate identification of 23 different kinds of melanoma. For this, 2300 dermoscopic images were used to train the skin-melanoma similar image search system. The proposed system uses feature extraction by assigning dynamic weights to the low-level features based on the individual characteristics of the searched images. Optimal weights are obtained by the newly proposed optimized pair-wise comparison (OPWC) approach. The uniqueness of the proposed approach is that it provides the dynamic weights to the features of the searched image instead of applying static weights. The proposed approach is supported by analytic hierarchy process (AHP) and meta-heuristic optimization algorithms such as particle swarm optimization (PSO), JAYA, genetic algorithm (GA), and gray wolf optimization (GWO). The proposed approach has been tested with images of 23 classes of melanoma and achieved significant precision and recall. Thus, this approach of skin melanoma image search can be used as an expert assistive system to help dermatologists/physicians for accurate identification of different types of melanomas.

A Modified LBP Operator-Based Optimized Fuzzy Art Map Medical Image Retrieval System for Disease Diagnosis and Prediction.

Medical records generated in hospitals are treasures for academic research and future references. Medical Image Retrieval (MIR) Systems contribute significantly to locating the relevant records required for a particular diagnosis, analysis, and treatment. An efficient classifier and effective indexing technique are required for the storage and retrieval of medical images. In this paper, a retrieval framework is formulated by adopting a modified Local Binary Pattern feature (AvN-LBP) for indexing and an optimized Fuzzy Art Map (FAM) for classifying and searching medical images. The proposed indexing method extracts LBP considering information from neighborhood pixels and is robust to background noise. The FAM network is optimized using the Differential Evaluation (DE) algorithm (DEFAMNet) with a modified mutation operation to minimize the size of the network without compromising the classification accuracy. The performance of the proposed DEFAMNet is compared with that of other classifiers and descriptors; the classification accuracy of the proposed AvN-LBP operator with DEFAMNet is higher. The experimental results on three benchmark medical image datasets provide evidence that the proposed framework classifies the medical images faster and more efficiently with lesser computational cost.

Penuras Terbitan Gaussian Berorientasi untuk Peruasan Imej Paru-Paru Radiograf Mesin Pegun dan Mudah Alih

Unsupervised lung segmentation method is one of the mandatory processes in order to develop a Content Based Medical Image Retrieval System (CBMIRS) of CXR. There is limited study found on segmentation of mobile chest radiographs, that is relatively important especially for very sick patients whenever their radiographs will be taken using portable X-Ray machine.The purpose of the study is to present a solution for lung segmentation of standard and mobile chest radiographs using fully automated unsupervised method, based on oriented Gaussian derivatives filter with seven orientations, combined with Fuzzy C-Means clustering and thresholding to refine the lung region. A new algorithm to automatically generate a threshold value for each Gaussian response is also proposed. The algorithms are applied to both PA and AP chest radiographs from both public JSRT and private datasets from collaborative hospital. Two pre-processing blocks are introduced to standardize the images from different machines. Comparisons with the previous works found in the literature on JSRT dataset shows that our method gives a reasonably good result. Performance measures (accuracy, F-score, precision, sensitivity and specificity) for the segmentation of lung in public JSRT dataset are above 0.90 except for the overlap measure is 0.87. The median of overlap score for the private image database is 0.83 (standard machine) and 0.75 (mobile machines). The algorithm is also fast, with the average execution time of 12.5s. Our proposed method is fully automated, unsupervised, with no training or learning stage is necessary to segment the lungs taken using both a standard and mobile machines, and useful for the application of the CBMIRS.

Content-based medical image retrieval system for lung diseases using deep CNNs.

Content-based image retrieval (CBIR) systems are designed to retrieve images that are relevant, based on detailed analysis of latent image characteristics, thus eliminating the dependency of natural language tags, text descriptions, or keywords associated with the images. A CBIR system maintains high-level image visuals in the form of feature vectors, which the retrieval engine leverages for similarity-based matching and ranking for a given query image. In this paper, a CBIR system is proposed for the retrieval of medical images (CBMIR) for enabling the early detection and classification of lung diseases based on lung X-ray images. The proposed CBMIR system is built on the predictive power of deep neural models for the identification and classification of disease-specific features using transfer learning based models trained on standard COVID-19 Chest X-ray image datasets. Experimental evaluation on the standard dataset revealed that the proposed approach achieved an improvement of 49.71% in terms of precision, averaging across various distance metrics. Also, an improvement of 26.55% was observed in the area under precision-recall curve (AUPRC) values across all subclasses.

EFFECTIVE SEMANTIC FEATURES TO IMPROVE RETRIEVAL OF LUNG NODULES IN CT SCAN IMAGES

Successful treatment of a patient depends on the accurate determination of the disease type. The advent of big data facilitates the retrieving of medical images and helps physicians in reliable diagnoses using content-based medical image retrieval systems (CBMIR). They consist of a feature extraction module and a distance metric. The extracted textural or deep-based features identify different types of diseases. In the proposed retrieval algorithm, we use the gray level cooccurrence matrix as the common textural characteristics and integrate them with semantic attributes. The semantic features are the geometric characteristics of the tumor that a radiologist employ to distinguish between benign and malignant tumors. These high-level attributes include the Euler number, margin smoothness, and the aspect ratio of the lesion’s size. We used the Minkowski distance measure for computing the similarity of images and applied the proposed algorithm to 200 CT-scan data containing lung lesions obtained from the LIDC database. The types of lesions were benign and malignant. Employing an ablation study, we proved the effectiveness of the semantic feature. The precision of the retrieval results is 93% which is promising compared to recent studies. In the future, we plan to define other kinds of semantic attributes to distinguish stages 1–5 of lung tumors as well.

A comparative approach for classifying retinal OCT images based on deep learning framework

Convolutional Networks are category of deep optimizing networks used to interpret images in Deep Learning concepts. Image recognition and medical image analysis are two areas where they are useful. The increasing scale of clinical feature spaces is raising a significant obstacle, creating issues with extensive database management, and afterward compiling those repositories for retrieval and storage, that could only be addressed using content based medical image retrieval systems. The objective of this paper is to demonstrate a deep CNN architecture for retrieving research and clinical images quickly and efficiently for identifying multi-class retinal disease objects. To train the network, the datasets used are inter-modal and divided into 4 groups. The transfer learning method is used for the multi-classification of retinal images. Another augmentation technique is used for comparing the accuracy, precision, and evaluation metrics with the transfer learning method. The accuracy of 97.1%, with a recall of 97.2%, and a precision of 97.0% was achieved in research that is higher when compared with the previous methods that were deployed. With the augmentation technique, it achieved an accuracy of 94.0% with a 94.6% precision and a recall of 95.1% for the testing data which suggests that decreasing the size of data did not impact the accuracy of the model. The proposed model helps diagnose various categories of medical images for the development of a comprehensive system that can work better than the human experts and help to detect and diagnose various diseases in the medical and clinical fields.

Effective CBMIR System Using Hybrid Features-Based Independent Condensed Nearest Neighbor Model.

In recent times, a large number of medical images are generated, due to the evolution of digital imaging modalities and computer vision application. Due to variation in the shape and size of the images, the retrieval task becomes more tedious in the large medical databases. So, it is essential in designing an effective automated system for medical image retrieval. In this research study, the input medical images are acquired from new Pap smear dataset, and then, the visible quality of acquired medical images is improved by applying image normalization technique. Furthermore, the hybrid feature extraction is accomplished using histogram of oriented gradients and modified local binary pattern to extract the color and texture feature vectors that significantly reduces the semantic gap between the feature vectors. The obtained feature vectors are fed to the independent condensed nearest neighbor classifier to classify the seven classes of cell images. Finally, relevant medical images are retrieved using chi square distance measure. Simulation results confirmed that the proposed model obtained effective performance in image retrieval in light of specificity, recall, precision, accuracy, and f-score. The proposed model almost achieved 98.88% of retrieval accuracy, which is better compared to other deep learning models such as long short-term memory network, deep neural network, and convolutional neural network.

Iterative brain tumor retrieval for MR images based on user’s intention model

Generally, medical content-based image retrieval (CBIR) systems select low-level visual features as image descriptors. However, these descriptors fail to provide clues for understanding the content of medical images in a similar way as a human expert, which makes the retrieval results inconsistent with the user’s intention. To solve this problem, we propose a closed-loop brain tumor retrieval system for MR images with an eye-tracking based relevance feedback mechanism. In our method, we first model the intention of the user by training a convolutional neural network based on the temporal and spatial features extracted from his/her eye-tracking data collected when inspecting the relevance between different images. Upon using visual features as a bridge, the relevancy degree to the query image of any of the database images is computed with our user’s intention model by transferring to it the eye movement data from the most visually similar image amongst images iteratively accumulated in the canvas. Our proposed retrieval system is implemented in an iterative manner. In each round of iteration, user’s eye movement data when inspecting the system returns are collected and the canvas collection of images is also updated by appending to it the user inspected system returns. With the updated canvas collections, the relevancy degree of database images can be recomputed and the system can begin a new round search of the most relevant images. Extensive experiments have been performed on a publicly available T1-weighted contrast-enhanced magnetic resonance image (CE-MRI) dataset that consists of three types of brain tumors (glioma, meningioma, and pituitary tumor) collected from 233 patients with a total of 3064 images across the axial, coronal, and sagittal views. Experimental results of 22 volunteers (11 males and 11 females, with an average age of 24.4 years) from our medical school show that upon implicit involvement of users in the brain tumor retrieving process, our proposed system significantly outperforms state-of-the-art methods and achieves Prec@10 to 99.94%, mAP to 97.95% after the third round of iteration.

Integration of optimized local directional weber pattern with faster region convolutional neural network for enhanced medical image retrieval and classification

AbstractWith a broad usage of digital imaging data in hospitals, the medical image repository size is growing in a very rapid manner. This creates difficulty in querying and managing these vast databases that in turn leads to the requirement of Content‐Based Medical Image Retrieval (CBMIR) systems. The CBMIR is considered as the major ambiguous and challenging task for reducing the semantic gap among the human queries and images in the datasets having more information content. The main intention of this paper is to enhance medical image retrieval and classification using the improved pattern descriptor and deep learning. For the medical image retrieval, this paper develops the Optimized Local Directional Weber Pattern with a multi‐objective similarity function. This similarity function focuses on the measures like Structural Similarity Index Measure, Peak Signal‐to‐Noise Ratio, Mean Squared Error MSE, and correlation. Likewise, the development of an improved Faster‐Region Convolutional Neural Network (Faster‐RCNN) is adopted in the classification phase. Both the retrieval and classification are enhanced by the Modified Wind Speed‐based Deer Hunting Optimization Algorithm. Considering classification and retrieval tasks, the experiments on benchmark datasets reveal an enhancement from the introduced algorithm when compared over the existing methods.

Optimal IoT Based Improved Deep Learning Model for Medical Image Classification

Recently medical image classification plays a vital role in medical image retrieval and computer-aided diagnosis system. Despite deep learning has proved to be superior to previous approaches that depend on handcrafted features; it remains difficult to implement because of the high intra-class variance and inter-class similarity generated by the wide range of imaging modalities and clinical diseases. The Internet of Things (IoT) in healthcare systems is quickly becoming a viable alternative for delivering high-quality medical treatment in today’s e-healthcare systems. In recent years, the Internet of Things (IoT) has been identified as one of the most interesting research subjects in the field of health care, notably in the field of medical image processing. For medical picture analysis, researchers used a combination of machine and deep learning techniques as well as artificial intelligence. These newly discovered approaches are employed to determine diseases, which may aid medical specialists in disease diagnosis at an earlier stage, giving precise, reliable, efficient, and timely results, and lowering death rates. Based on this insight, a novel optimal IoT-based improved deep learning model named optimization-driven deep belief neural network (ODBNN) is proposed in this article. In context, primarily image quality enhancement procedures like noise removal and contrast normalization are employed. Then the pre-processed image is subjected to feature extraction techniques in which intensity histogram, an average pixel of RGB channels, first-order statistics, Grey Level Co-Occurrence Matrix, Discrete Wavelet Transform, and Local Binary Pattern measures are extracted. After extracting these sets of features, the May Fly optimization technique is adopted to select the most relevant features. The selected features are fed into the proposed classification algorithm in terms of classifying similar input images into similar classes. The proposed model is evaluated in terms of accuracy, precision, recall, and f-measure. The investigation evident the performance of incorporating optimization techniques for medical image classification is better than conventional techniques.

Content Based Medical Image Retrieval Based on Salient Regions Combined with Deep Learning

In traditional text based medical image retrieval system, it is hard to find visually similar images in large medical image database. Content-based image retrieval is developed to retrieve similar images and it is based on visual attributes describing the content of an image. Developing a method for content based medical image retrieval is a challenging task. This paper proposed a new method for content-based medical image retrieval based on salient regions and deep learning. The proposed method includes two stages: an offline task to extract local object features and an online task for content-based image retrieval in database. In first stage, we extract local object features of medical image depending on shape, texture and intensity, and features extracted by deep learning applied in saliency of decomposition. Secondly, we make online task for content-based image retrieval in database. The user gives query image as an input and the system will retrieve n top most similar images by similarity comparison with bag of code words feature values obtained in the first stage. Evaluation of the proposed method is based on Precision and Recall values. Our dataset includes 5 groups of medical images with their quality varying from low to high. With the best medical image quality group, the accuray can be 91.61% for Precision and 89.61% for Recall. Comparing the average values with others methods, the results of the proposed method are more than 2 to 5% better.

Class-driven content-based medical image retrieval using hash codes of deep features

Medical imaging provides the convenience of physicians to analyze the disease by providing visual data of the body parts required for clinical research and treatment. Today, increasing medical images following technological developments are stored for a better understanding of diseases and future diagnoses. Effective medical image indexing and retrieval systems are required to use these images from storage repositories in real-time. In this quest, this paper provides an effective indexing and retrieval framework using deep features for MR and CT image indexing and searching. The proposed system aims to produce the most effective and least parameterized hash codes by using image features. For this reason, deep features are obtained from medical images using the convolutional neural network (CNN) architecture, which is the most effective automatic feature extraction method. The length of the acquired raw deep feature vectors for an image is relatively inefficient for retrieval speed. Feature reduction methods are used for the most effective reduction of the length of the deep feature vector. The most effective feature reduction algorithm is determined in this study. The main reason for producing a reduced class-driven hash code with feature selection algorithms is the drawbacks of medical image datasets. These drawbacks prevent the CNN output from being used directly as hash-code. The performance of the proposed method is tested on NEMA MRI and NEMA CT datasets. The proposed method is able to outperform the other state-of-the-art algorithms in terms of average precision performance.

Radiological image retrieval technique using multi-resolution texture and shape features

Medical image analysis plays a very indispensable role in providing the best possible medical support to a patient. With the rapid advancements in modern medical systems, these digital images are growing exponentially and reside in discrete places. These images help a medical practitioner in understanding the problem and then the best suitable treatment. Radiological images are very often found to be the critical constituent of medical images. So, in health care, manual retrieval of visually similar images becomes a very tedious task. To address this issue, we have suggested a content-based medical image retrieval (CBMIR) system that effectively analyzes a Radiological image’s primitive visual features. Since radiological images are in gray-scale form, these images contain rich texture and shape features only. So, we have suggested a novel multi-resolution radiological image retrieval system that uses texture and shape features for content analysis. Here, we have employed a multi-resolution modified block difference of inverse probability (BDIP) and block-level variance of local variance (BVLC) for shape and texture features, respectively. Our proposed scheme uses a multi-resolution and variable window size feature extraction strategy to maintain the block-level co-relation and extract more salient visual features. Further, we have used the MURA x-ray image dataset, which has 40561 images captured from 12173 different patients to demonstrate the proposed scheme’s retrieval performance. We have also performed and compared image retrieval experiments on Brodatz and STex texture, Corel-1K, and GHIM-10K natural image datasets to demonstrate the robustness and improvement over other contemporaries.

An Efficient Gabor Walsh-Hadamard Transform Based Approach for Retrieving Brain Tumor Images From MRI

Brain tumors are a serious and death-defying disease for human life. Discovering an appropriate brain tumor image from a magnetic resonance imaging (MRI) archive is a challenging job for the radiologist. Most search engines retrieve images on the basis of traditional text-based approaches. The main challenge in the MRI image analysis is that low-level visual information captured by the MRI machine and the high-level information identified by the assessor. This semantic gap is addressed in this study by designing a new feature extraction technique. In this paper, we introduce Content-Based Medical Image retrieval (CBMIR) system for retrieval of brain tumor images from the large data. Firstly, we remove noise from MRI images employing several filtering techniques. Afterward, we design a feature extraction scheme combining Gabor filtering technique (which is mainly focused on specific frequency content at the image region) and Walsh-Hadamard transform (WHT) (conquer technique for easy configuration of image) for discovering representative features from MRI images. After that, for retrieving the accurate and reliable image, we employ Fuzzy C-Means clustering Minkowski distance metric that can evaluate the similarity between the query image and database images. The proposed methodology design was tested on a publicly available brain tumor MRI image database. The experimental results demonstrate that our proposed approach outperforms most of the existing techniques like Gabor, wavelet, and Hough transform in detecting brain tumors and also take less time. The proposed approach will be beneficial for radiologists and also for technologists to build an automatic decision support system that will produce reproducible and objective results with high accuracy.

Medical Image Retrieval Using Empirical Mode Decomposition with Deep Convolutional Neural Network.

Content-based medical image retrieval (CBMIR) systems attempt to search medical image database to narrow the semantic gap in medical image analysis. The efficacy of high-level medical information representation using features is a major challenge in CBMIR systems. Features play a vital role in the accuracy and speed of the search process. In this paper, we propose a deep convolutional neural network- (CNN-) based framework to learn concise feature vector for medical image retrieval. The medical images are decomposed into five components using empirical mode decomposition (EMD). The deep CNN is trained in a supervised way with multicomponent input, and the learned features are used to retrieve medical images. The IRMA dataset, containing 11,000 X-ray images, 116 classes, is used to validate the proposed method. We achieve a total IRMA error of 43.21 and a mean average precision of 0.86 for retrieval task and IRMA error of 68.48 and F1 measure of 0.66 on classification task, which is the best result compared with existing literature for this dataset.