Medical Image Analysis Methods Research Articles

Machine Learning Empowering Personalized Medicine: A Comprehensive Review of Medical Image Analysis Methods

Artificial intelligence (AI) advancements, especially deep learning, have significantly improved medical image processing and analysis in various tasks such as disease detection, classification, and anatomical structure segmentation. This work overviews fundamental concepts, state-of-the-art models, and publicly available datasets in the field of medical imaging. First, we introduce the types of learning problems commonly employed in medical image processing and then proceed to present an overview of commonly used deep learning methods, including convolutional neural networks (CNNs), recurrent neural networks (RNNs), and generative adversarial networks (GANs), with a focus on the image analysis task they are solving, including image classification, object detection/localization, segmentation, generation, and registration. Further, we highlight studies conducted in various application areas, encompassing neurology, brain imaging, retinal analysis, pulmonary imaging, digital pathology, breast imaging, cardiac imaging, bone analysis, abdominal imaging, and musculoskeletal imaging. The strengths and limitations of each method are carefully examined, and the paper identifies pertinent challenges that still require attention, such as the limited availability of annotated data, variability in medical images, and the interpretability issues. Finally, we discuss future research directions with a particular focus on developing explainable deep learning methods and integrating multi-modal data.

A U-Net with Statistical Shape Restrictions Applied to the Segmentation of the Left Ventricle in Echocardiographic Images

This paper aims to introduce an innovative approach to semantic segmentation by leveraging a convolutional neural network (CNN) for predicting the shape and pose parameters of the left ventricle (LV). Our approach involves a modified U-Net architecture with a regression layer as the final stage, as opposed to the traditional classification layer. This modification allows us to predict all the shape and pose parameters of a statistical shape model, including rotation, translation, scale, and deformation. The adapted U-Net is trained using data from a point distribution model (PDM) of the LV. The experimental results demonstrate a mean Dice coefficient of 0.82 on good quality images, and 0.66 including mean and low-quality images. Our approach successfully overcomes a common issue encountered in CNN-based semantic segmentation. Unlike the inaccurate pixel classification that often leads to unwanted blobs, our CNN generates statistically valid shapes. These shapes hold significant potential in initializing other methods, such as active shape models (ASMs). Our novel CNN-based approach provides a novel solution for semantic segmentation, offering shapes and pose parameters that can enhance the accuracy and reliability of subsequent medical image analysis methods.

18F-FDG PET/CT Image Deep Learning Predicts Colon Cancer Survival.

Colon cancer is a type of cancer that begins in the large intestine. In the process of efficacy evaluation, postoperative recurrence prediction and metastasis monitoring of colon cancer, traditional medical image analysis methods are highly dependent on the personal ability of the doctors. In the process of patient treatment, it not only increases the workload and work pressure for doctors, but also has some problems with traditional medical image analysis methods. Moreover, the traditional medical image analysis methods have problems such as insufficient prediction accuracy, slow prediction speed, and the risk of errors in prediction. When analyzing 18F-FDG PET/CT images by traditional medical image analysis methods, it is easy to cause problems such as untimely treatment plans and errors in diagnosis, which will adversely affect the survival of colon cancer patients. Although 18F-FDG PET/CT images have certain advantages in image clarity and accuracy compared with traditional medical imaging methods, the analysis method based on 18F-FDG PET/CT images also has certain effects in predicting the survival of colon cancer patients, but there are still many shortcomings: the 18F-FDG PET/CT image analysis method overly relies on the technical advantages of 8F-FDG PET/CT images; in the analysis and prediction of image data, it has not gotten rid of the dependence on the personal medical quality of the doctors; traditional medical image analysis methods are still used when analyzing and predicting images; there is no breakthrough in image analysis effects. In order to solve these problems, this paper combined deep learning theory, using three algorithms of the improved RBM algorithm, image feature extraction method based on deep learning, and regression neural network to analyze and predict 18F-FDG PET/CT images, and applied some algorithms to analyze and predict 18F-FDG PET/CT images, and also established a deep learning-based 18F-FDG PET/CT image survival analysis prediction model. Four aspects survival prediction accuracy, survival prediction speed, survival prediction precision, and physician satisfaction were studied through this model. The research results have shown that compared with traditional medical image analysis methods, the prediction accuracy of 18F-FDG PET/CT image survival analysis prediction model based on deep learning is improved by 0.83%, and the prediction speed is improved by 3.42%, as well as the prediction precision increased by 6.13%. The research results show that the deep learning-based 18F-FDG PET/CT image survival analysis prediction model established in this paper is of great significance to improve the survival rate of colon cancer patients, and also promotes the development of the medical industry.

A Framework for Simulating Cardiac MR Images with Varying Anatomy and Contrast.

One of the limiting factors for the development and adoption of novel deep-learning (DL) based medical image analysis methods is the scarcity of labeled medical images. Medical image simulation and synthesis can provide solutions by generating ample training data with corresponding ground truth labels. Despite recent advances, generated images demonstrate limited realism and diversity. In this work, we develop a flexible framework for simulating cardiac magnetic resonance (MR) images with variable anatomical and imaging characteristics for the purpose of creating a diversified virtual population. We advance previous works on both cardiac MR image simulation and anatomical modeling to increase the realism in terms of both image appearance and underlying anatomy. To diversify the generated images, we define parameters: 1) to alter the anatomy, 2) to assign MR tissue properties to various tissue types, and 3) to manipulate the image contrast via acquisition parameters. The proposed framework is optimized to generate a substantial number of cardiac MR images with ground truth labels suitable for downstream supervised tasks. A database of virtual subjects is simulated and its usefulness for aiding a DL segmentation method is evaluated. Our experiments show that training completely with simulated images can perform comparable with a model trained with real images for heart cavity segmentation in mid-ventricular slices. Moreover, such data can be used in addition to classical augmentation for boosting the performance when training data is limited, particularly by increasing the contrast and anatomical variation, leading to better regularization and generalization. The database is publicly available at https://osf.io/ bkzhm/ and the simulation code will be available at https: //github.com/sinaamirrajab/CMRI_Simulation.

A Tumor MRI Image Segmentation Framework Based on Class-Correlation Pattern Aggregation in Medical Decision-Making System

Medical image analysis methods have been applied to clinical scenarios of tumor diagnosis and treatment. Many studies have attempted to optimize the effectiveness of tumor MRI image segmentation by deep learning, but they do not consider the optimization of local details and the interaction of global semantic information. Second, although medical image pattern recognition can learn representative semantic features, it is challenging to ignore useless features in order to learn generalizable embeddings. Thus, a tumor-assisted segmentation method is proposed to detect tumor lesion regions and boundaries with complex shapes. Specifically, we introduce a denoising convolutional autoencoder (DCAE) for MRI image noise reduction. Furthermore, we design a novel tumor MRI image segmentation framework (NFSR-U-Net) based on class-correlation pattern aggregation, which first aggregates class-correlation patterns in MRI images to form a class-correlational representation. Then the relationship of similar class features is identified to closely correlate the dense representations of local features for classification, which is conducive to identifying image data with high heterogeneity. Meanwhile, the model uses a spatial attention mechanism and residual structure to extract effective information of the spatial dimension and enhance statistical information in MRI images, which bridges the semantic gap in skip connections. In the study, over 4000 MRI images from the Monash University Research Center for Artificial Intelligence are analyzed. The results show that the method achieves segmentation accuracy of up to 96% for tumor MRI images with low resource consumption.

Multiple Brain Tumor Classification with Dense CNN Architecture Using Brain MRI Images

Brain MR images are the most suitable method for detecting chronic nerve diseases such as brain tumors, strokes, dementia, and multiple sclerosis. They are also used as the most sensitive method in evaluating diseases of the pituitary gland, brain vessels, eye, and inner ear organs. Many medical image analysis methods based on deep learning techniques have been proposed for health monitoring and diagnosis from brain MRI images. CNNs (Convolutional Neural Networks) are a sub-branch of deep learning and are often used to analyze visual information. Common uses include image and video recognition, suggestive systems, image classification, medical image analysis, and natural language processing. In this study, a new modular deep learning model was created to retain the existing advantages of known transfer learning methods (DenseNet, VGG16, and basic CNN architectures) in the classification process of MR images and eliminate their disadvantages. Open-source brain tumor images taken from the Kaggle database were used. For the training of the model, two types of splitting were utilized. First, 80% of the MRI image dataset was used in the training phase and 20% in the testing phase. Secondly, 10-fold cross-validation was used. When the proposed deep learning model and other known transfer learning methods were tested on the same MRI dataset, an improvement in classification performance was obtained, but an increase in processing time was observed.

Analysis of cranial ultrasound images for newborn.

Neonatal cranial ultrasound plays an important role in the evaluation of neonatal brain development and related diseases. This paper preliminarily explored the analysis and interpretation methods of neonatal brain ultrasound images, and applied the relevant medical image analysis methods to analyze the relevant neonatal brain ultrasound images in more detail. Compared with other types of imaging methods, ultrasound has its unique advantages and characteristics in such applications as neonatal head imaging. The analysis steps and schemes adopted in this paper have certain reference significance for the analysis of other neonatal brain pictures.

Efficient Breast Cancer Classification Network with Dual Squeeze and Excitation in Histopathological Images.

Medical image analysis methods for mammograms, ultrasound, and magnetic resonance imaging (MRI) cannot provide the underline features on the cellular level to understand the cancer microenvironment which makes them unsuitable for breast cancer subtype classification study. In this paper, we propose a convolutional neural network (CNN)-based breast cancer classification method for hematoxylin and eosin (H&E) whole slide images (WSIs). The proposed method incorporates fused mobile inverted bottleneck convolutions (FMB-Conv) and mobile inverted bottleneck convolutions (MBConv) with a dual squeeze and excitation (DSE) network to accurately classify breast cancer tissue into binary (benign and malignant) and eight subtypes using histopathology images. For that, a pre-trained EfficientNetV2 network is used as a backbone with a modified DSE block that combines the spatial and channel-wise squeeze and excitation layers to highlight important low-level and high-level abstract features. Our method outperformed ResNet101, InceptionResNetV2, and EfficientNetV2 networks on the publicly available BreakHis dataset for the binary and multi-class breast cancer classification in terms of precision, recall, and F1-score on multiple magnification levels.

Explainable artificial intelligence (XAI) in deep learning-based medical image analysis.

With an increase in deep learning-based methods, the call for explainability of such methods grows, especially in high-stakes decision making areas such as medical image analysis. This survey presents an overview of explainable artificial intelligence (XAI) used in deep learning-based medical image analysis. A framework of XAI criteria is introduced to classify deep learning-based medical image analysis methods. Papers on XAI techniques in medical image analysis are then surveyed and categorized according to the framework and according to anatomical location. The paper concludes with an outlook of future opportunities for XAI in medical image analysis.

Deep Learning and Medical Image Analysis for COVID-19 Diagnosis and Prediction.

The coronavirus disease 2019 (COVID-19) pandemic has imposed dramatic challenges to health-care organizations worldwide. To combat the global crisis, the use of thoracic imaging has played a major role in the diagnosis, prediction, and management of COVID-19 patients with moderate to severe symptoms or with evidence of worsening respiratory status. In response, the medical image analysis community acted quickly to develop and disseminate deep learning models and tools to meet the urgent need of managing and interpreting large amounts of COVID-19 imaging data. This review aims to not only summarize existing deep learning and medical image analysis methods but also offer in-depth discussions and recommendations for future investigations. We believe that the wide availability of high-quality, curated, and benchmarked COVID-19 imaging data sets offers the great promise of a transformative test bed to develop, validate, and disseminate novel deep learning methods in the frontiers of data science and artificial intelligence.

Automatic lung nodule detection in thoracic CT scans using dilated slice-wise convolutions.

Most state-of-the-art automated medical image analysis methods for volumetric data rely on adaptations of two-dimensional (2D) and three-dimensional (3D) convolutional neural networks (CNNs). In this paper, we develop a novel unified CNN-based model that combines the benefits of 2D and 3D networks for analyzing volumetric medical images. In our proposed framework, multiscale contextual information is first extracted from 2D slices inside a volume of interest (VOI). This is followed by dilated 1D convolutions across slices to aggregate in-plane features in a slice-wise manner and encode the information in the entire volume. Moreover, we formalize a curriculum learning strategy for a two-stage system (i.e., a system that consists of screening and false positive reduction), where the training samples are presented to the network in a meaningful order to further improve the performance. We evaluated the proposed approach by developing a computer-aided detection (CADe) system for lung nodules. Our results on 888 CT exams demonstrate that the proposed approach can effectively analyze volumetric data by achieving a sensitivity of > 0.99 in the screening stage and a sensitivity of > 0.96 at eight false positives per case in the false positive reduction stage. Our experimental results show that the proposed method provides competitive results compared to state-of-the-art 3D frameworks. In addition, we illustrate the benefits of curriculum learning strategies in two-stage systems that are of common use in medical imaging applications.

Some Problematic Aspects of Coliform Bacteria Clustering on Medical Images in the Task of Identifying Possible Diseases

Medical image analysis methods are one of the sources for obtaining additional information about the investigated phenomena. We are looking at images of coliform bacteria. Analysis of these images allows you to determine the possibility of developing certain diseases. To do this, it is necessary to cluster the set of bacteria and count the bacteria. The paper highlights the features of clustering for coliform bacteria. Clustering results for real data are presented.

Medical Image Analysis Using AM-FM Models and Methods.

Medical image analysis methods require the use of effective representations for differentiating between lesions, diseased regions, and normal structure. Amplitude Modulation-Frequency Modulation (AM-FM) models provide effective representations through physically meaningful descriptors of complex non-stationary structures that can differentiate between the different lesions and normal structure. Based on AM-FM models, medical images are decomposed into AM-FM components where the instantaneous frequency provides a descriptor of local texture, the instantaneous amplitude captures slowly-varying brightness variations, while the instantaneous phase provides for a powerful descriptor of location, generalizing the traditionally important role of phase in the Fourier Analysis of images. Over the years, AM-FM representations have been used in a wide variety of medical image analysis applications based on a vastly reduced number of features that can be easily learned by simple classifiers. The paper provides an overview of AM-FM models and methods, followed by applications in medical image analysis. We also provide a summary of emerging trends and future directions.

Residual LSTM layered CNN for classification of gastrointestinal tract diseases

nowadays, considering the number of patients per specialist doctor, the size of the need for automatic medical image analysis methods can be understood. These systems, which are very advantageous compared to manual systems both in terms of cost and time, benefit from artificial intelligence (AI). AI mechanisms that mimic the decision-making process of a specialist increase their diagnosis performance day by day, depending on technological developments. In this study, an AI method is proposed to effectively classify Gastrointestinal (GI) Tract Image datasets containing a small number of labeled data. The proposed AI method uses the convolutional neural network (CNN) architecture, which is accepted as the most successful automatic classification method of today, as a backbone. According to our approach, a shallowly trained CNN architecture needs to be supported by a strong classifier to classify unbalanced datasets robustly. For this purpose, the features in each pooling layer in the CNN architecture are transmitted to an LSTM layer. A classification is made by combining all LSTM layers. All experiments are carried out using AlexNet, GoogLeNet, and ResNet to evaluate the contribution of the proposed residual LSTM structure fairly. Besides, three different experiments are carried out with 2000, 4000, and 6000 samples to determine the effect of sample number change on the proposed method. The performance of the proposed method is higher than other state-of-the-art methods.

A Survey of Dictionary Learning in Medical Image Analysis and Its Application for Glaucoma Diagnosis

Dictionary learning has shown its effectiveness in computer vision with the concise expression form but the powerful representation. Dictionary learning represents images with a bag of visual words (BoVW), which is a collection of atoms expressively representative for images. Recently, several task-specific dictionary learning methods have been proposed and successfully applied in medical image analysis, such as de-noising, classification, segmentation, and so on, which promotes the development of computer-aided diagnosis. In this paper, first we give a survey for dictionary learning-based medical image analysis methods including: (1) three discriminative dictionary learning frameworks, (2) CT image de-noising based on dictionary learning, and (3) histopathological image classification using sparse representation. Then, a novel method named Low-rank Shared Dictionary Learning (LRSDL), is presented to achieve accurate glaucoma diagnosis on fundus images. The LRSDL generates a shared codebook for image reconstruction and a particular one to handle the difference between the healthy and glaucomatous images. Benefit from this strategy, LRSDL not only possess distinct glaucoma-related features, but also share common patterns among all the fundus images. Experimental results show that the method effectively delivers glaucoma diagnosis with the accuracy of 92.90%. This endows dictionary learning method a great potential for glaucoma diagnosis and proves the feasibility of its application to medical image analysis.

Efficient Ultrasound Image Analysis Models with Sonographer Gaze Assisted Distillation.

Recent automated medical image analysis methods have attained state-of-the-art performance but have relied on memory and compute-intensive deep learning models. Reducing model size without significant loss in performance metrics is crucial for time and memory-efficient automated image-based decision-making. Traditional deep learning based image analysis only uses expert knowledge in the form of manual annotations. Recently, there has been interest in introducing other forms of expert knowledge into deep learning architecture design. This is the approach considered in the paper where we propose to combine ultrasound video with point-of-gaze tracked for expert sonographers as they scan to train memory-efficient ultrasound image analysis models. Specifically we develop teacher-student knowledge transfer models for the exemplar task of frame classification for the fetal abdomen, head, and femur. The best performing memory-efficient models attain performance within 5% of conventional models that are 1000× larger in size.

Investigation of Image Processing Techniques in MRI Based Medical Image Analysis Methods and Validation Metrics for Brain Tumor

Background: This paper reviews the recent techniques employed to process brain tumor images from Magnetic Resonance (MR) images. Automation in tumor and its sub-compartment segmentation from MR images is rising and necessary in rapid diagnosis and planning of human brain treatment. Discussion: The success of fine tumor segmentation depends on an entire pipeline of the processes such as preprocessing, segmentation, feature extraction and classification tools. Conclusion: Hence, the review aims to present a collective overview of existing techniques and explore the metrics which are used for various performance analysis for image processing techniques. Keywords: Brain tumor, magnetic resonance image, classification, segmentation, treatment, processing techniques.

Corrigendum to “Medical image analysis methods in MR/CT-imaged acute–subacute ischemic stroke lesion: Segmentation, prediction and insights into dynamic evolution simulation models. A critical appraisal” [NeuroImage: Clinical 1 (2012) 164–178

[This corrects the article DOI: 10.1016/j.nicl.2012.10.003.].

Constructing Benchmark Databases and Protocols for Medical Image Analysis: Diabetic Retinopathy

We address the performance evaluation practices for developing medical image analysis methods, in particular, how to establish and share databases of medical images with verified ground truth and solid evaluation protocols. Such databases support the development of better algorithms, execution of profound method comparisons, and, consequently, technology transfer from research laboratories to clinical practice. For this purpose, we propose a framework consisting of reusable methods and tools for the laborious task of constructing a benchmark database. We provide a software tool for medical image annotation helping to collect class label, spatial span, and expert's confidence on lesions and a method to appropriately combine the manual segmentations from multiple experts. The tool and all necessary functionality for method evaluation are provided as public software packages. As a case study, we utilized the framework and tools to establish the DiaRetDB1 V2.1 database for benchmarking diabetic retinopathy detection algorithms. The database contains a set of retinal images, ground truth based on information from multiple experts, and a baseline algorithm for the detection of retinopathy lesions.

Medical image analysis methods in MR/CT-imaged acute-subacute ischemic stroke lesion: Segmentation, prediction and insights into dynamic evolution simulation models. A critical appraisal.

Over the last 15 years, basic thresholding techniques in combination with standard statistical correlation-based data analysis tools have been widely used to investigate different aspects of evolution of acute or subacute to late stage ischemic stroke in both human and animal data. Yet, a wave of biology-dependent and imaging-dependent issues is still untackled pointing towards the key question: “how does an ischemic stroke evolve?” Paving the way for potential answers to this question, both magnetic resonance (MRI) and CT (computed tomography) images have been used to visualize the lesion extent, either with or without spatial distinction between dead and salvageable tissue. Combining diffusion and perfusion imaging modalities may provide the possibility of predicting further tissue recovery or eventual necrosis. Going beyond these basic thresholding techniques, in this critical appraisal, we explore different semi-automatic or fully automatic 2D/3D medical image analysis methods and mathematical models applied to human, animal (rats/rodents) and/or synthetic ischemic stroke to tackle one of the following three problems: (1) segmentation of infarcted and/or salvageable (also called penumbral) tissue, (2) prediction of final ischemic tissue fate (death or recovery) and (3) dynamic simulation of the lesion core and/or penumbra evolution. To highlight the key features in the reviewed segmentation and prediction methods, we propose a common categorization pattern. We also emphasize some key aspects of the methods such as the imaging modalities required to build and test the presented approach, the number of patients/animals or synthetic samples, the use of external user interaction and the methods of assessment (clinical or imaging-based). Furthermore, we investigate how any key difficulties, posed by the evolution of stroke such as swelling or reperfusion, were detected (or not) by each method. In the absence of any imaging-based macroscopic dynamic model applied to ischemic stroke, we have insights into relevant microscopic dynamic models simulating the evolution of brain ischemia in the hope to further promising and challenging 4D imaging-based dynamic models. By depicting the major pitfalls and the advanced aspects of the different reviewed methods, we present an overall critique of their performances and concluded our discussion by suggesting some recommendations for future research work focusing on one or more of the three addressed problems.