Image Retrieval In Medical Applications Research Articles

Computer aided diagnosis system for breast density classification in mammograms

Breast cancer is a deadly disease affecting women around the globe. Mass detection in the breast tissue at an early stage can lessen the mortality rate occurring due to breast cancer. Through mammograms, the presence of masses can be detected at an early stage, however, it’s sensitivity and specificity are limited in the case of dense tissues. Identification of the breast density type prior to the detection of mass can lessen the chance of misclassifying a breast tissue as normal or abnormal, which eventually decreases the false negative and false positive rate. The proposed system classifies breast density on the basis of Breast Imaging Reporting and Data System (BI-RADS). The proposed method has explored the effect of local descriptors on breast density classification and various feature-classifier combinations have also been explored for the classification. The proposed method validated on 624 mammograms from the Image Retrieval in Medical Applications (IRMA) version of the Digital Database for Screening Mammography (DDSM) database has produced an accuracy of 73% for multi-class breast density classification using the speeded-up robust features (SURF) and support vector machine (SVM) classifier.

Improved search space shrinking for medical image retrieval using capsule architecture and decision fusion

Medical diagnosis is a challenging procedure that involves issues such as data imbalance, insufficient labels, obscure images, redundancy and lack of effective model training directions to shrink the semantic gap between human knowledge and computer algorithms. Due to privacy norms, sometimes medical images are difficult to access and therefore, retrieval of identical existing images from an existing repository is quite useful. This paper proposes a search space to narrow down the identical images in an archive by using (i) Capsule Networks, followed by a (ii) decision fusion with Wavelet-Discrete Cosine Transform (W-DCT) and Radon Barcodes (RBC). Empirical case study has been applied on IRMA (Image Retrieval in Medical Applications) dataset, ImageCLEFMed-2009, containing 14,410 X-ray images, but the proposed method is generic, reproducible and scalable. Subjective and quantitative performance has been compared with the state-of-art and it has been found superior to yield accuracy of 92.83% and IRMA error of 124.25 for 193 class-code category. Thus, the proof-of-concept helps to improves diagnosis efficiency for automatic image retrieval and annotation by clustering similar images from the underlying repository.

Assessing and mitigating the effects of class imbalance in machine learning with application to X-ray imaging.

Machine learning (ML) algorithms are well known to exhibit variations in prediction accuracy when provided with imbalanced training sets typically seen in medical imaging (MI) due to the imbalanced ratio of pathological and normal cases. This paper presents a thorough investigation of the effects of class imbalance and methods for mitigating class imbalance in ML algorithms applied to MI. We first selected five classes from the Image Retrieval in Medical Applications (IRMA) dataset, performed multiclass classification using the random forest model (RFM), and then performed binary classification using convolutional neural network (CNN) on a chest X-ray dataset. An imbalanced class was created in the training set by varying the number of images in that class. Methods tested to mitigate class imbalance included oversampling, undersampling, and changing class weights of the RFM. Model performance was assessed by overall classification accuracy, overall F1 score, and specificity, recall, and precision of the imbalanced class. A close-to-balanced training set resulted in the best model performance, and a large imbalance with overrepresentation was more detrimental to model performance than underrepresentation. Oversampling and undersampling methods were both effective in mitigating class imbalance, and efficacy of oversampling techniques was class specific. This study systematically demonstrates the effect of class imbalance on two public X-ray datasets on RFM and CNN, making these findings widely applicable as a reference. Furthermore, the methods employed here can guide researchers in assessing and addressing the effects of class imbalance, while considering the data-specific characteristics to optimize imbalance mitigating methods.

Annotation of enhanced radiographs for medical image retrieval with deep convolutional neural networks.

The number of images taken per patient scan has rapidly increased due to advances in software, hardware and digital imaging in the medical domain. There is the need for medical image annotation systems that are accurate as manual annotation is impractical, time-consuming and prone to errors. This paper presents modeling approaches performed to automatically classify and annotate radiographs using several classification schemes, which can be further applied for automatic content-based image retrieval (CBIR) and computer-aided diagnosis (CAD). Different image preprocessing and enhancement techniques were applied to augment grayscale radiographs by virtually adding two extra layers. The Image Retrieval in Medical Applications (IRMA) Code, a mono-hierarchical multi-axial code, served as a basis for this work. To extensively evaluate the image enhancement techniques, five classification schemes including the complete IRMA code were adopted. The deep convolutional neural network systems Inception-v3 and Inception-ResNet-v2, and Random Forest models with 1000 trees were trained using extracted Bag-of-Keypoints visual representations. The classification model performances were evaluated using the ImageCLEF 2009 Medical Annotation Task test set. The applied visual enhancement techniques proved to achieve better annotation accuracy in all classification schemes.

Comparison of rotation invariant local frequency, LBP and SFTA methods for breast abnormality classification

Breast cancer is the second most prominent cancer diagnosed among women. Digital mammography is one of the effective imaging modalities used to detect breast cancer in early stages. Computer-aided detection systems help radiologists to detect and diagnose abnormalities earlier and faster in a mammogram. In this paper, a comprehensive study is carried out on different feature extraction methods for classification of abnormal areas in a mammogram. The prominent techniques used for feature extraction in this study are local binary pattern (LBP), rotation invariant local frequency (RILF) and segmented fractal texture analysis (SFTA). Features extracted from these techniques are then fed to a support vector machine (SVM) classifier for further classification via 10-fold cross-validation method. The evaluation is performed using image retrieval in medical applications (IRMA) database for feature extraction. Our statistical analysis shows that the RILF technique outperforms the LBP and SFTA techniques.

A New Feature Ensemble with a Multistage Classification Scheme for Breast Cancer Diagnosis.

A new and effective feature ensemble with a multistage classification is proposed to be implemented in a computer-aided diagnosis (CAD) system for breast cancer diagnosis. A publicly available mammogram image dataset collected during the Image Retrieval in Medical Applications (IRMA) project is utilized to verify the suggested feature ensemble and multistage classification. In achieving the CAD system, feature extraction is performed on the mammogram region of interest (ROI) images which are preprocessed by applying a histogram equalization followed by a nonlocal means filtering. The proposed feature ensemble is formed by concatenating the local configuration pattern-based, statistical, and frequency domain features. The classification process of these features is implemented in three cases: a one-stage study, a two-stage study, and a three-stage study. Eight well-known classifiers are used in all cases of this multistage classification scheme. Additionally, the results of the classifiers that provide the top three performances are combined via a majority voting technique to improve the recognition accuracy on both two- and three-stage studies. A maximum of 85.47%, 88.79%, and 93.52% classification accuracies are attained by the one-, two-, and three-stage studies, respectively. The proposed multistage classification scheme is more effective than the single-stage classification for breast cancer diagnosis.

Interactive radiographic image retrieval system

Background and ObjectiveContent based medical image retrieval (CBMIR) systems enable fast diagnosis through quantitative assessment of the visual information and is an active research topic over the past few decades. Most of the state-of-the-art CBMIR systems suffer from various problems: computationally expensive due to the usage of high dimensional feature vectors and complex classifier/clustering schemes. Inability to properly handle the “semantic gap” and the high intra-class versus inter-class variability problem of the medical image database (like radiographic image database). This yields an exigent demand for developing highly effective and computationally efficient retrieval system. MethodsWe propose a novel interactive two-stage CBMIR system for diverse collection of medical radiographic images. Initially, Pulse Coupled Neural Network based shape features are used to find out the most probable (similar) image classes using a novel “similarity positional score” mechanism. This is followed by retrieval using Non-subsampled Contourlet Transform based texture features considering only the images of the pre-identified classes. Maximal information compression index is used for unsupervised feature selection to achieve better results. To reduce the semantic gap problem, the proposed system uses a novel fuzzy index based relevance feedback mechanism by incorporating subjectivity of human perception in an analytic manner. ResultsExtensive experiments were carried out to evaluate the effectiveness of the proposed CBMIR system on a subset of Image Retrieval in Medical Applications (IRMA)-2009 database consisting of 10,902 labeled radiographic images of 57 different modalities. We obtained overall average precision of around 98% after only 2–3 iterations of relevance feedback mechanism. We assessed the results by comparisons with some of the state-of-the-art CBMIR systems for radiographic images. ConclusionsUnlike most of the existing CBMIR systems, in the proposed two-stage hierarchical framework, main importance is given on constructing efficient and compact feature vector representation, search-space reduction and handling the “semantic gap” problem effectively, without compromising the retrieval performance. Experimental results and comparisons show that the proposed system performs efficiently in the radiographic medical image retrieval field.

COLOR HISTOGRAM BASED MEDICAL IMAGE RETRIEVAL SYSTEM

This paper aims to focus on the feature extraction, selection and database creation of brain images for image retrieval which will aid for computer assisted diagnosis. The impact of content-based access to medical images is frequently reported but existing systems are designed for only a particular context of diagnosis. But, our concept of image retrieval in medical applications aims at a general structure for semantic content analysis that is suitable for numerous applications in case-based reasoning. By using the features, the database created for comparison. The color histogram is used to measure the similarity between the stored database image and the query image. The image which is more similar to the query image is retrieved as the resultant image. If the query

Towards a Repository for Standardized Medical Image and Signal Case Data Annotated with Ground Truth

Validation of medical signal and image processing systems requires quality-assured, representative and generally acknowledged databases accompanied by appropriate reference (ground truth) and clinical metadata, which are composed laboriously for each project and are not shared with the scientific community. In our vision, such data will be stored centrally in an open repository. We propose an architecture for a standardized case data and ground truth information repository supporting the evaluation and analysis of computer-aided diagnosis based on (a) the Reference Model for an Open Archival Information System (OAIS) provided by the NASA Consultative Committee for Space Data Systems (ISO 14721:2003), (b) the Dublin Core Metadata Initiative (DCMI) Element Set (ISO 15836:2009), (c) the Open Archive Initiative (OAI) Protocol for Metadata Harvesting, and (d) the Image Retrieval in Medical Applications (IRMA) framework. In our implementation, a portal bunches all of the functionalities that are needed for data submission and retrieval. The complete life cycle of the data (define, create, store, sustain, share, use, and improve) is managed. Sophisticated search tools make it easier to use the datasets, which may be merged from different providers. An integrated history record guarantees reproducibility. A standardized creation report is generated with a permanent digital object identifier. This creation report must be referenced by all of the data users. Peer-reviewed e-publishing of these reports will create a reputation for the data contributors and will form de-facto standards regarding image and signal datasets. Good practice guidelines for validation methodology complement the concept of the case repository. This procedure will increase the comparability of evaluation studies for medical signal and image processing methods and applications.

Towards case-based medical learning in radiological decision making using content-based image retrieval

BackgroundRadiologists' training is based on intensive practice and can be improved with the use of diagnostic training systems. However, existing systems typically require laboriously prepared training cases and lack integration into the clinical environment with a proper learning scenario. Consequently, diagnostic training systems advancing decision-making skills are not well established in radiological education.MethodsWe investigated didactic concepts and appraised methods appropriate to the radiology domain, as follows: (i) Adult learning theories stress the importance of work-related practice gained in a team of problem-solvers; (ii) Case-based reasoning (CBR) parallels the human problem-solving process; (iii) Content-based image retrieval (CBIR) can be useful for computer-aided diagnosis (CAD). To overcome the known drawbacks of existing learning systems, we developed the concept of image-based case retrieval for radiological education (IBCR-RE). The IBCR-RE diagnostic training is embedded into a didactic framework based on the Seven Jump approach, which is well established in problem-based learning (PBL). In order to provide a learning environment that is as similar as possible to radiological practice, we have analysed the radiological workflow and environment.ResultsWe mapped the IBCR-RE diagnostic training approach into the Image Retrieval in Medical Applications (IRMA) framework, resulting in the proposed concept of the IRMAdiag training application. IRMAdiag makes use of the modular structure of IRMA and comprises (i) the IRMA core, i.e., the IRMA CBIR engine; and (ii) the IRMAcon viewer. We propose embedding IRMAdiag into hospital information technology (IT) infrastructure using the standard protocols Digital Imaging and Communications in Medicine (DICOM) and Health Level Seven (HL7). Furthermore, we present a case description and a scheme of planned evaluations to comprehensively assess the system.ConclusionsThe IBCR-RE paradigm incorporates a novel combination of essential aspects of diagnostic learning in radiology: (i) Provision of work-relevant experiences in a training environment integrated into the radiologist's working context; (ii) Up-to-date training cases that do not require cumbersome preparation because they are provided by routinely generated electronic medical records; (iii) Support of the way adults learn while remaining suitable for the patient- and problem-oriented nature of medicine. Future work will address unanswered questions to complete the implementation of the IRMAdiag trainer.

Generic integration of content-based image retrieval in computer-aided diagnosis

Content-based image retrieval (CBIR) offers approved benefits for computer-aided diagnosis (CAD), but is still not well established in radiological routine yet. An essential factor is the integration gap between CBIR systems and clinical information systems. The international initiative Integrating the Healthcare Enterprise (IHE) aims at improving interoperability of medical computer systems. We took into account deficiencies in IHE compliance of current picture archiving and communication systems (PACS), and developed an intermediate integration scheme based on the IHE post-processing workflow integration profile (PWF) adapted to CBIR in CAD. The Image Retrieval in Medical Applications (IRMA) framework was used to apply our integration scheme exemplarily, resulting in the application called IRMAcon. The novel IRMAcon scheme provides a generic, convenient and reliable integration of CBIR systems into clinical systems and workflows. Based on the IHE PWF and designed to grow at a pace with the IHE compliance of the particular PACS, it provides sustainability and fosters CBIR in CAD.

MammoSys: A content-based image retrieval system using breast density patterns

In this paper, we present a content-based image retrieval system designed to retrieve mammographies from large medical image database. The system is developed based on breast density, according to the four categories defined by the American College of Radiology, and is integrated to the database of the Image Retrieval in Medical Applications (IRMA) project, that provides images with classification ground truth. Two-dimensional principal component analysis is used in breast density texture characterization, in order to effectively represent texture and allow for dimensionality reduction. A support vector machine is used to perform the retrieval process. Average precision rates are in the range from 83% to 97% considering a data set of 5024 images. The results indicate the potential of the system as the first stage of a computer-aided diagnosis framework.

Content-based Image Retrieval for Scientific Literature Access

An increasing number of articles are published electronically in the scientific literature, but access is limited to alphanumerical search on title, author, or abstract, and may disregard numerous figures. In this paper, we estimate the benefits of using content-based image retrieval (CBIR) on article figures to augment traditional access to articles. We selected four high-impact journals from the Journal Citations Report (JCR) 2005. Figures were automatically extracted from the PDF article files, and manually classified on their content and number of sub-figure panels. We make a quantitative estimate by projecting from data from the Cross-Language Evaluation Forum (ImageCLEF) campaigns, and qualitatively validate it through experiments using the Image Retrieval in Medical Applications (IRMA) project. Based on 2077 articles with 11,753 pages, 4493 figures, and 11,238 individual images, the predicted accuracy for article retrieval may reach 97.08%. Therefore, CBIR potentially has a high impact in medical literature search and retrieval.

A framework and baseline results for the CLEF medical automatic annotation task

The medical automatic annotation task issued by the cross language evaluation forum (CLEF) aims at a fair comparison of state-of-the art algorithms for medical content-based image retrieval (CBIR). The contribution of this work is twofold: at first, a logical decomposition of the CBIR task is presented, and key elements to support the relevant steps are identified: (i) implementation of algorithms for feature extraction, feature comparison, and classifier combination, (ii) visualization of extracted features and retrieval results, (iii) generic evaluation of retrieval algorithms, and (iv) optimization of the parameters for the retrieval algorithms and their combination. Data structures and tools to address these key elements are integrated into an existing framework for image retrieval in medical applications (IRMA). Secondly, baseline results for the CLEF annotation tasks 2005–2007 are provided applying the IRMA framework, where global features and corresponding distance measures are combined within a nearest neighbor approach. Using identical classifier parameters and combination weights for each year shows that the task difficulty decreases over the years. The declining rank of the baseline submission also indicates the overall advances in CBIR concepts. Furthermore, a rough comparison between participants who submitted in only one of the years becomes possible.

Effectiveness of global features for automatic medical image classification and retrieval – The experiences of OHSU at ImageCLEFmed

In 2006 and 2007, Oregon Health and Science University (OHSU) participated in the automatic image annotation task for medical images at ImageCLEF, an annual international benchmarking event that is part of the cross language evaluation forum (CLEF). The goal of the automatic annotation task was to classify 1000 test images based on the image retrieval in medical applications (IRMA) code, given a set of 10,000 training images. There were 116 distinct classes in 2006 and 2007. We evaluated the efficacy of a variety of primarily global features for this classification task. These included features based on histograms, gray level correlation matrices and the gist technique. A multitude of classifiers including k-nearest neighbors, two-level neural networks, support vector machines, and maximum likelihood classifiers were evaluated. Our official error rates for the 1000 test images were 26% in 2006 using the flat classification structure. The error count in 2007 was 67.8 using the hierarchical classification error computation based on the IRMA code in 2007. Confusion matrices as well as clustering experiments were used to identify visually similar classes. The use of the IRMA code did not help us in the classification task as the semantic hierarchy of the IRMA classes did not correspond well with the hierarchy based on clustering of image features that we used. Our most frequent misclassification errors were along the view axis. Subsequent experiments based on a two-stage classification system decreased our error rate to 19.8% for the 2006 dataset and our error count to 55.4 for the 2007 data.

Sensor, Signal and Image Informatics

The number of articles published annually in the fields of biomedical signal and image acquisition and processing is increasing. Based on selected examples, this survey aims at comprehensively demonstrating the recent trends and developments. Four articles are selected for biomedical data acquisition covering topics such as dose saving in CT, C-arm X-ray imaging systems for volume imaging, and the replacement of dose-intensive CT-based diagnostic with harmonic ultrasound imaging. Regarding biomedical signal analysis (BSA), the four selected articles discuss the equivalence of different time-frequency approaches for signal analysis, an application to Cochlea implants, where time-frequency analysis is applied for controlling the replacement system, recent trends for fusion of different modalities, and the role of BSA as part of a brain machine interfaces. To cover the broad spectrum of publications in the field of biomedical image processing, six papers are focused. Important topics are content-based image retrieval in medical applications, automatic classification of tongue photographs from traditional Chinese medicine, brain perfusion analysis in single photon emission computed tomography (SPECT), model-based visualization of vascular trees, and virtual surgery, where enhanced visualization and haptic feedback techniques are combined with a sphere-filled model of the organ. The selected papers emphasize the five fields forming the chain of biomedical data processing: (1) data acquisition, (2) data reconstruction and pre-processing, (3) data handling, (4) data analysis, and (5) data visualization. Fields 1 and 2 form the sensor informatics, while fields 2 to 5 form signal or image informatics with respect to the nature of the data considered. Biomedical data acquisition and pre-processing, as well as data handling, analysis and visualization aims at providing reliable tools for decision support that improve the quality of health care. Comprehensive evaluation of the processing methods and their reliable integration in routine applications are future challenges in the field of sensor, signal and image informatics.

A generic concept for the implementation of medical image retrieval systems

This paper presents a technical framework to support the development and installation of system for content-based image retrieval in medical applications (IRMA). A strict separation of feature extraction, feature storage, feature comparison, and the user interfaces is suggested. This allows to reuse implemented components in different retrieval algorithms, which improves software quality, shortens the development cycle for applications, and allows to introduce standardized end-user interfaces. Based on the proposed framework, the IRMA engine has been established, which is currently used to evaluate content-based retrieval methods on a collection of 20,000 medical and 135,000 non-medical images.

Content-based Image Retrieval in Medical Applications

To develop a general structure for semantic image analysis that is suitable for content-based image retrieval in medical applications and an architecture for its efficient implementation. Stepwise content analysis of medical images results in six layers of information modeling incorporating medical expert knowledge (raw data layer, registered data layer, feature layer, scheme layer, object layer, knowledge layer). A reference database with 10,000 images categorized according to the image modality, orientation, body region, and biological system is used. By means of prototypes in each category, identification of objects and their geometrical or temporal relationships are handled in the object and the knowledge layer, respectively. A distributed system designed with only three core elements is implemented: (i) the central database holds program sources, processing scheme descriptions, images, features, and administrative information about the workstation cluster; (ii) the scheduler balances distributed computing; and (iii) the web server provides graphical user interfaces for data entry and retrieval, which can be easily adapted to a variety of applications for content-based image retrieval in medicine. Leaving-one-out experiments were distributed by the scheduler and controlled via corresponding job lists offering transparency regarding the viewpoints of a distributed system and the user. The proposed architecture is suitable for content-based image retrieval in medical applications. It improves current picture archiving and communication systems that still rely on alphanumerical descriptions, which are insufficient for image retrieval of high recall and precision.

Statistical framework for model-based image retrieval in medical applications

Recently, research in the field of content-based image retrieval has attracted a lot of attention. Nevertheless, most existing methods cannot be easily applied to medical image databases, as global image descriptions based on color, texture, or shape do not supply sufficient semantics for medical applications. The concept for content-based image retrieval in medical applications (IRMA) is therefore based on the separation of the following processing steps: categorization of the entire image; registration with respect to proto- types; extraction and query-dependent selection of local features; hierarchical blob representation including object identification; and finally, image retrieval. Within the first step of processing, images are classified according to image modality, body orientation, ana- tomic region, and biological system. The statistical classifier for the anatomic region is based on Gaussian kernel densities within a probabilistic framework for multiobject recognition. Special empha- sis is placed on invariance, employing a probabilistic model of vari- ability based on tangent distance and an image distortion model. The performance of the classifier is evaluated using a set of 1617 radiographs from daily routine, where the error rate of 8.0% in this six-class problem is an excellent result, taking into account the dif- ficulty of the task. The computed posterior probabilities are further- more used in the subsequent steps of the retrieval process. © 2003