Medical Imaging Tasks Research Articles

Contrastive Registration for Unsupervised Medical Image Segmentation.

Medical image segmentation is an important task in medical imaging, as it serves as the first step for clinical diagnosis and treatment planning. While major success has been reported using deep learning supervised techniques, they assume a large and well-representative labeled set. This is a strong assumption in the medical domain where annotations are expensive, time-consuming, and inherent to human bias. To address this problem, unsupervised segmentation techniques have been proposed in the literature. Yet, none of the existing unsupervised segmentation techniques reach accuracies that come even near to the state-of-the-art of supervised segmentation methods. In this work, we present a novel optimization model framed in a new convolutional neural network (CNN)-based contrastive registration architecture for unsupervised medical image segmentation called CLMorph. The core idea of our approach is to exploit image-level registration and feature-level contrastive learning, to perform registration-based segmentation. First, we propose an architecture to capture the image-to-image transformation mapping via registration for unsupervised medical image segmentation. Second, we embed a contrastive learning mechanism in the registration architecture to enhance the discriminative capacity of the network at the feature level. We show that our proposed CLMorph technique mitigates the major drawbacks of existing unsupervised techniques. We demonstrate, through numerical and visual experiments, that our technique substantially outperforms the current state-of-the-art unsupervised segmentation methods on two major medical image datasets.

The Latent Doctor Model for Modeling Inter-Observer Variability.

Many inherently ambiguous tasks in medical imaging suffer from inter-observer variability, resulting in a reference standard defined by a distribution of labels with high variance. Training only on a consensus or majority vote label, as is common in medical imaging, discards valuable information on uncertainty amongst a panel of experts. In this work, we propose to train on the full label distribution to predict the uncertainty within a panel of experts and the most likely ground-truth label. To do so, we propose a new stochastic classification framework based on the conditional variational auto-encoder, which we refer to as the Latent Doctor Model (LDM). In an extensive comparative analysis, we compare the LDM with a model trained on the majority vote label and other methods capable of learning a distribution of labels. We show that the LDM is able to reproduce the reference-standard distribution significantly better than the majority vote baseline. Compared to the other baseline methods, we demonstrate that the LDM performs best at modeling the label distribution and its corresponding uncertainty in two prostate tumor grading tasks. Furthermore, we show competitive performance of the LDM with the more computationally demanding deep ensembles on a tumor budding classification task.

Framework for Deep Learning Based Multi-Modality Image Registration of Snapshot and Pathology Images.

Multi-modality image registration is an important task in medical imaging because it allows for information from different domains to be correlated. Histopathology plays a crucial role in oncologic surgery as it is the gold standard for investigating tissue composition from surgically excised specimens. Research studies are increasingly focused on registering medical imaging modalities such as white light cameras, magnetic resonance imaging, computed tomography, and ultrasound to pathology images. The main challenge in registration tasks involving pathology images comes from addressing the considerable amount of deformation present. This work provides a framework for deep learning-based multi-modality registration of microscopic pathology images to another imaging modality. The proposed framework is validated on the registration of prostate ex-vivo white light camera snapshot images to pathology hematoxylin-eosin images of the same specimen. A pipeline is presented detailing data acquisition, protocol considerations, image dissimilarity, training experiments, and validation. A comprehensive analysis is done on the impact of pre-processing, data augmentation, loss functions, and regularization. This analysis is supplemented by clinically motivated evaluation metrics to avoid the pitfalls of only using ubiquitous image comparison metrics. Consequently, a robust training configuration capable of performing the desired registration task is found. Utilizing the proposed approach, we achieved a dice similarity coefficient of 0.96, a mutual information score of 0.54, a target registration error of 2.4 mm, and a regional dice similarity coefficient of 0.70.

BFNet: a full-encoder skip connect way for medical image segmentation.

In recent years, semantic segmentation in deep learning has been widely applied in medical image segmentation, leading to the development of numerous models. Convolutional Neural Network (CNNs) have achieved milestone achievements in medical image analysis. Particularly, deep neural networks based on U-shaped architectures and skip connections have been extensively employed in various medical image tasks. U-Net is characterized by its encoder-decoder architecture and pioneering skip connections, along with multi-scale features, has served as a fundamental network architecture for many modifications. But U-Net cannot fully utilize all the information from the encoder layer in the decoder layer. U-Net++ connects mid parameters of different dimensions through nested and dense skip connections. However, it can only alleviate the disadvantage of not being able to fully utilize the encoder information and will greatly increase the model parameters. In this paper, a novel BFNet is proposed to utilize all feature maps from the encoder at every layer of the decoder and reconnects with the current layer of the encoder. This allows the decoder to better learn the positional information of segmentation targets and improves learning of boundary information and abstract semantics in the current layer of the encoder. Our proposed method has a significant improvement in accuracy with 1.4 percent. Besides enhancing accuracy, our proposed BFNet also reduces network parameters. All the advantages we proposed are demonstrated on our dataset. We also discuss how different loss functions influence this model and some possible improvements.

A LLM-Based Hybrid-Transformer Diagnosis System in Healthcare.

The application of computer vision-powered large language models (LLMs) for medical image diagnosis has significantly advanced healthcare systems. Recent progress in developing symmetrical architectures has greatly impacted various medical imaging tasks. While CNNs or RNNs have demonstrated excellent performance, these architectures often face notable limitations of substantial losses in detailed information, such as requiring to capture global semantic information effectively and relying heavily on deep encoders and aggressive downsampling. This paper introduces a novel LLM-based Hybrid-Transformer Network (HybridTransNet) designed to encode tokenized Big Data patches with the transformer mechanism, which elegantly embeds multimodal data of varying sizes as token sequence inputs of LLMS. Subsequently, the network performs both inter-scale and intra-scale self-attention, processing data features through a transformer-based symmetric architecture with a refining module, which facilitates accurately recovering both local and global context information. Additionally, the output is refined using a novel fuzzy selector. Compared to other existing methods on two distinct datasets, the experimental findings and formal assessment demonstrate that our LLM-based HybridTransNet provides superior performance for brain tumor diagnosis in healthcare informatics.

Pattern-Aware Transformer: Hierarchical Pattern Propagation in Sequential Medical Images.

This paper investigates how to effectively mine contextual information among sequential images and jointly model them in medical imaging tasks. Different from state-of-the-art methods that model sequential correlations via point-wise token encoding, this paper develops a novel hierarchical pattern-aware tokenization strategy. It handles distinct visual patterns independently and hierarchically, which not only ensures the full flexibility of attention aggregation under different pattern representations but also preserves both local and global information simultaneously. Based on this strategy, we propose a Pattern-Aware Transformer (PATrans) featuring a global-local dual-path pattern-aware cross-attention mechanism to achieve hierarchical pattern matching and propagation among sequential images. Furthermore, PATrans is plug-and-play and can be seamlessly integrated into various backbone networks for diverse downstream sequence modeling tasks. We demonstrate its general application paradigm across four domains and five benchmarks in video object detection and 3D volumetric semantic segmentation tasks, respectively. Impressively, PATrans sets new state-of-the-art across all these benchmarks, i.e., CVC-Video (92.3% detection F1), ASU-Mayo (99.1% localization F1), Lung Tumor (78.59% DSC), Nasopharynx Tumor (75.50% DSC), and Kidney Tumor (87.53% DSC). Codes and models are available at https://github.com/GGaoxiang/PATrans.

Feature-enhanced multi-sequence MRI-based fusion mechanism for breast tumor segmentation

Multi-sequence MRI plays a crucial role in the effective segmentation of breast tumors, contributing to accurate clinical diagnosis and treatment. However, the problem of missing certain sequence images may occur in clinical practice, leading to a potential impact on network performance. In medical imaging tasks, researchers are prone to ignore the correlation between different sequences, resulting in lack of expression of the extracted features. To address these problems, we propose the Feature-enhanced Multi-sequence Feature Fusion Network (FMFF-Net), which fuses information from dynamic contrast-enhanced (DCE) and diffusion-weighted imaging (DWI) sequences to improve the accuracy of breast tumor segmentation. Utilizing an attention mechanism and adversarial learning, FMFF-Net identifies potential mappings and relationships between sequences, generating features for missing sequences and enhancing feature representation. The network incorporates a High-Frequency Edge Attention Block (HFAB) to accentuate tumor edge details, leading to more precise segmentation. Tested on a dataset of 98 high-risk breast cancer MRI images, FMFF-Net demonstrated superior performance, with a Dice Similarity Coefficient (DSC) of 83.6%, Intersection over Union (IoU) of 73.4%, F1 score of 88.1%, and Sensitivity of 74.1%. Comparative analysis against mainstream segmentation methods revealed that our proposed FMFF-Net exhibits a competitive edge in the field of breast tumor segmentation.

Dynamic Corrected Split Federated Learning With Homomorphic Encryption for U-Shaped Medical Image Networks.

U-shaped networks have become prevalent in various medical image tasks such as segmentation, and restoration. However, most existing U-shaped networks rely on centralized learning which raises privacy concerns. To address these issues, federated learning (FL) and split learning (SL) have been proposed. However, achieving a balance between the local computational cost, model privacy, and parallel training remains a challenge. In this articler, we propose a novel hybrid learning paradigm called Dynamic Corrected Split Federated Learning (DC-SFL) for U-shaped medical image networks. To preserve data privacy, including the input, model parameters, label and output simultaneously, we propose to split the network into three parts hosted by different parties. We propose a Dynamic Weight Correction Strategy (DWCS) to stabilize the training process and avoid the model drift problem due to data heterogeneity. To further enhance privacy protection and establish a trustworthy distributed learning paradigm, we propose to introduce additively homomorphic encryption into the aggregation process of client-side model, which helps prevent potential collusion between parties and provides a better privacy guarantee for our proposed method. The proposed DC-SFL is evaluated on various medical image tasks, and the experimental results demonstrate its effectiveness. In comparison with state-of-the-art distributed learning methods, our method achieves competitive performance.

Tensor radiomics: paradigm for systematic incorporation of multi-flavoured radiomics features.

Radiomics features hold significant value as quantitative imaging biomarkers for diagnosis, prognosis, and treatment response assessment. To generate radiomics features and ultimately develop signatures, various factors can be manipulated, including image discretization parameters (e.g., bin number or size), convolutional filters, segmentation perturbation, or multi-modality fusion levels. Typically, only one set of parameters is employed, resulting in a single value or "flavour" for each radiomics feature. In contrast, we propose "tensor radiomics" (TR) where tensors of features calculated using multiple parameter combinations (i.e., flavours) are utilized to optimize the creation of radiomics signatures. We provide illustrative instances of TR implementation in positron emission tomography-computed tomography (PET-CT), magnetic resonance imaging (MRI), and CT by leveraging machine learning (ML) and deep learning (DL) methodologies, as well as reproducibility analyses: (I) to predict overall survival (OS) in lung cancer (CT) and head and neck cancer (PET-CT), TR was employed by varying bin sizes. This approach involved use of a hybrid deep neural network called 'TR-Net' and two ML-based techniques for combining different flavours. (II) TR was constructed by incorporating different segmentation perturbations and various bin sizes to classify the response of late-stage lung cancer to first-line immunotherapy using CT images. (III) In MRI of glioblastoma (GBM), TR was implemented to generate multi-flavour radiomics features, enabling enhanced analysis and interpretation. (IV) TR was employed via multiple PET-CT fusions in head and neck cancer. Flavours based on different fusions were created using Laplacian pyramids and wavelet transforms. Our findings demonstrated that TR outperformed conventional radiomics features in lung cancer CT and head and neck cancer PET-CT images, significantly enhancing OS prediction accuracy. TR also improved classification of lung cancer response to therapy and exhibited notable advantages in reproducibility compared to single-flavour features in MR imaging of GBM. Moreover, in head and neck cancer, TR through multiple PET-CT fusions exhibited improved performance in predicting OS. We conclude that the proposed TR paradigm has significant potential to improve performance in different medical imaging tasks. By incorporating multiple flavours of radiomics features, TR overcomes limitations associated with individual features and shows promise in enhancing prognostic capabilities in clinical settings.

Residual Aligner-based Network (RAN): Motion-separable structure for coarse-to-fine discontinuous deformable registration

Deformable image registration, the estimation of the spatial transformation between different images, is an important task in medical imaging. Deep learning techniques have been shown to perform 3D image registration efficiently. However, current registration strategies often only focus on the deformation smoothness, which leads to the ignorance of complicated motion patterns (e.g., separate or sliding motions), especially for the intersection of organs. Thus, the performance when dealing with the discontinuous motions of multiple nearby objects is limited, causing undesired predictive outcomes in clinical usage, such as misidentification and mislocalization of lesions or other abnormalities. Consequently, we proposed a novel registration method to address this issue: a new Motion Separable backbone is exploited to capture the separate motion, with a theoretical analysis of the upper bound of the motions’ discontinuity provided. In addition, a novel Residual Aligner module was used to disentangle and refine the predicted motions across the multiple neighboring objects/organs. We evaluate our method, Residual Aligner-based Network (RAN), on abdominal Computed Tomography (CT) scans and it has shown to achieve one of the most accurate unsupervised inter-subject registration for the 9 organs, with the highest-ranked registration of the veins (Dice Similarity Coefficient (%)/Average surface distance (mm): 62%/4.9mm for the vena cava and 34%/7.9mm for the portal and splenic vein), with a smaller model structure and less computation compared to state-of-the-art methods. Furthermore, when applied to lung CT, the RAN achieves comparable results to the best-ranked networks (94%/3.0mm), also with fewer parameters and less computation.

Self-supervised pre-training with contrastive and masked autoencoder methods for dealing with small datasets in deep learning for medical imaging

Deep learning in medical imaging has the potential to minimize the risk of diagnostic errors, reduce radiologist workload, and accelerate diagnosis. Training such deep learning models requires large and accurate datasets, with annotations for all training samples. However, in the medical imaging domain, annotated datasets for specific tasks are often small due to the high complexity of annotations, limited access, or the rarity of diseases. To address this challenge, deep learning models can be pre-trained on large image datasets without annotations using methods from the field of self-supervised learning. After pre-training, small annotated datasets are sufficient to fine-tune the models for a specific task. The most popular self-supervised pre-training approaches in medical imaging are based on contrastive learning. However, recent studies in natural image processing indicate a strong potential for masked autoencoder approaches. Our work compares state-of-the-art contrastive learning methods with the recently introduced masked autoencoder approach “SparK” for convolutional neural networks (CNNs) on medical images. Therefore, we pre-train on a large unannotated CT image dataset and fine-tune on several CT classification tasks. Due to the challenge of obtaining sufficient annotated training data in medical imaging, it is of particular interest to evaluate how the self-supervised pre-training methods perform when fine-tuning on small datasets. By experimenting with gradually reducing the training dataset size for fine-tuning, we find that the reduction has different effects depending on the type of pre-training chosen. The SparK pre-training method is more robust to the training dataset size than the contrastive methods. Based on our results, we propose the SparK pre-training for medical imaging tasks with only small annotated datasets.

Whale Optimization Algorithm with Fuzzy Wavelet Neural Network for Pneumonia Detection and Classification

Pneumonia detection and classification are a vital medical imaging task that purposes to automatically recognize and classify pneumonia-related abnormalities in chest radiographs or Chest X-Ray (CXR) imaging. Accurate and prior identification of pneumonia is important for suitable treatment and recovering patient results. The main drive is to recognize whether an X-ray image indicates the presence of pneumonia or not. A binary classification techniuqe is trained to distinguish normal and pneumonia X-rays. Deep Learning (DL) approaches are revealed major success in automating this process, assisting healthcare specialists in analyzing pneumonia more effectively. This article presents a Whale Optimization Algorithm with Fuzzy Wavelet Neural Network for Pneumonia Detection and Classification (WOAFWNN-PDC) technique on CXRs. The purpose of the WOAFWNN-PDC technique is to apply optimal DL approaches for the recognition and classification of pneumonia. In the presented WOAFWNN-PDC technique, Gaussian Filtering (GF) approach is used for the noise removal process. In addition, the MobileNetv3 model is utilized for the feature extraction method. Moreover, the FWNN technique was applied to the classification of pneumonia. Finally, the WOA can be executed for an optimum selection of the parameters related to the FWNN approach. The simulation value of the WOAFWNN-PDC algorithm was assessed on a benchmark medical database. The comparative analysis exhibits better results than the WOAFWNN-PDC technique.

Cardiac Magnetic Resonance Imaging Focus Generative Adversarial Network Segmentation

Cardiovascular function analysis is crucial for illness diagnosis, risk assessment, and therapy selection in clinical cardiology. Doctors may identify cardiac disorders such as right ventricular failure, hypertrophic cardiomyopathy, and dilated cardiomyopathy using a variety of imaging modalities that allow them to spot pathological alterations. The optimum course of therapy may be chosen more quickly thanks to accurate automation of the relevant duties. Artificial neural networks and deep learning are the foundation of generative adversarial networks (GANs), which are methods for creating synthetic pictures. The potential capacity of the GANs to solve problems has attracted interest in addition to their inherent flexibility and the adaptability of deep learning, on which they are founded. This survey aims to examine the significance of medical imaging in the study and diagnosis of cardiac disease. Demonstrate the widespread adoption of GAN Network approaches in the field of magnetic resonance imaging (MRI) medical image analysis; Explain the recent segmentation application of generative adversarial networks. GANs in cardiovascular imaging additionally Identify the hurdles to the effective application of the GAN Network to medical imaging tasks and highlight particular contributions that address or get around these problems

PaCL: Patient-aware contrastive learning through metadata refinement for generalized early disease diagnosis

Early diagnosis plays a pivotal role in effectively treating numerous diseases, especially in healthcare scenarios where prompt and accurate diagnoses are essential. Contrastive learning (CL) has emerged as a promising approach for medical tasks, offering advantages over traditional supervised learning methods. However, in healthcare, patient metadata contains valuable clinical information that can enhance representations, yet existing CL methods often overlook this data. In this study, we propose an novel approach that leverages both clinical information and imaging data in contrastive learning to enhance model generalization and interpretability. Furthermore, existing contrastive methods may be prone to sampling bias, which can lead to the model capturing spurious relationships and exhibiting unequal performance across protected subgroups frequently encountered in medical settings. To address these limitations, we introduce Patient-aware Contrastive Learning (PaCL), featuring an inter-class separability objective (IeSO) and an intra-class diversity objective (IaDO). IeSO harnesses rich clinical information to refine samples, while IaDO ensures the necessary diversity among samples to prevent class collapse. We demonstrate the effectiveness of PaCL both theoretically through causal refinements and empirically across six real-world medical imaging tasks spanning three imaging modalities: ophthalmology, radiology, and dermatology. Notably, PaCL outperforms previous techniques across all six tasks.

Dive into the details of self-supervised learning for medical image analysis.

Self-supervised learning (SSL) has achieved remarkable performance in various medical imaging tasks by dint of priors from massive unlabeled data. However, regarding a specific downstream task, there is still a lack of an instruction book on how to select suitable pretext tasks and implementation details throughout the standard "pretrain-then-finetune" workflow. In this work, we focus on exploiting the capacity of SSL in terms of four realistic and significant issues: (1) the impact of SSL on imbalanced datasets, (2) the network architecture, (3) the applicability of upstream tasks to downstream tasks and (4) the stacking effect of SSL and common policies for deep learning. We provide a large-scale, in-depth and fine-grained study through extensive experiments on predictive, contrastive, generative and multi-SSL algorithms. Based on the results, we have uncovered several insights. Positively, SSL advances class-imbalanced learning mainly by boosting the performance of the rare class, which is of interest to clinical diagnosis. Unfortunately, SSL offers marginal or even negative returns in some cases, including severely imbalanced and relatively balanced data regimes, as well as combinations with common training policies. Our intriguing findings provide practical guidelines for the usage of SSL in the medical context and highlight the need for developing universal pretext tasks to accommodate diverse application scenarios. The code of this paper can be found at https://github.com/EndoluminalSurgicalVision-IMR/Medical-SSL.

Understanding calibration of deep neural networks for medical image classification

Background and Objective – In the field of medical image analysis, achieving high accuracy is not enough; ensuring well-calibrated predictions is also crucial. Confidence scores of a deep neural network play a pivotal role in explainability by providing insights into the model's certainty, identifying cases that require attention, and establishing trust in its predictions. Consequently, the significance of a well-calibrated model becomes paramount in the medical imaging domain, where accurate and reliable predictions are of utmost importance. While there has been a significant effort towards training modern deep neural networks to achieve high accuracy on medical imaging tasks, model calibration and factors that affect it remain under-explored.Methods – To address this, we conducted a comprehensive empirical study that explores model performance and calibration under different training regimes. We considered fully supervised training, which is the prevailing approach in the community, as well as rotation-based self-supervised method with and without transfer learning, across various datasets and architecture sizes. Multiple calibration metrics were employed to gain a holistic understanding of model calibration.Results – Our study reveals that factors such as weight distributions and the similarity of learned representations correlate with the calibration trends observed in the models. Notably, models trained using rotation-based self-supervised pretrained regime exhibit significantly better calibration while achieving comparable or even superior performance compared to fully supervised models across different medical imaging datasets.Conclusion – These findings shed light on the importance of model calibration in medical image analysis and highlight the benefits of incorporating self-supervised learning approach to improve both performance and calibration.

DilatedFormer: dilated granularity transformer network for placental maturity grading in ultrasound

Placental maturity grading (PMG) is often utilized for evaluating fetal growth and maternal health. Currently, PMG often relied on the subjective judgment of the clinician, which is time-consuming and tends to incur a wrong estimation due to redundancy and repeatability of the process. The existing methods often focus on designing diverse hand-crafted features or combining deep features and hand-crafted features to learn a hybrid feature with an SVM for grading the placental maturity of ultrasound images. Motivated by the dominated performance of end-to-end convolutional neural networks (CNNs) at diverse medical imaging tasks, we devise a dilated granularity transformer network for learning multi-scale global transformer features for boosting PMG. Our network first devises dilated transformer blocks to learn multi-scale transformer features at each convolutional layer and then integrates these obtained multi-scale transformer features for predicting the final result of PMG. We collect 500 ultrasound images to verify our network, and experimental results show that our network clearly outperforms state-of-the-art methods on PMG. In the future, we will strive to improve the computational complexity and generalization ability of deep neural networks for PMG.

Attention U-net for automated pulmonary fissure integrity analysis in lung computed tomography images

Computed Tomography (CT) imaging is routinely used for imaging of the lungs. Deep learning can effectively automate complex and laborious tasks in medical imaging. In this work, a deep learning technique is utilized to assess lobar fissure completeness (also known as fissure integrity) from pulmonary CT images. The human lungs are divided into five separate lobes, divided by the lobar fissures. Fissure integrity assessment is important to endobronchial valve treatment screening. Fissure integrity is known to be a biomarker of collateral ventilation between lobes impacting the efficacy of valves designed to block airflow to diseased lung regions. Fissure integrity is also likely to impact lobar sliding which has recently been shown to affect lung biomechanics. Further widescale study of fissure integrity’s impact on disease susceptibility and progression requires rapid, reproducible, and noninvasive fissure integrity assessment. In this paper we describe IntegrityNet, an attention U-Net based automatic fissure integrity analysis tool. IntegrityNet is able to predict fissure integrity with an accuracy of 95.8%, 96.1%, and 89.8% for left oblique, right oblique, and right horizontal fissures, compared to manual analysis on a dataset of 82 subjects. We also show that our method is robust to COPD severity and reproducible across subject scans acquired at different time points.

Modality preserving U-Net for segmentation of multimodal medical images.

Recent advances in artificial intelligence and digital image processing have inspired the use of deep neural networks for segmentation tasks in multimodal medical imaging. Unlike natural images, multimodal medical images contain much richer information regarding different modal properties and therefore present more challenges for semantic segmentation. However, there is no report on systematic research that integrates multi-scaled and structured analysis of single-modal and multimodal medical images. We propose a deep neural network, named as Modality Preserving U-Net (MPU-Net), for modality-preserving analysis and segmentation of medical targets from multimodal medical images. The proposed MPU-Net consists of a modality preservation encoder (MPE) module that preserves the feature independency among the modalities and a modality fusion decoder (MFD) module that performs a multiscale feature fusion analysis for each modality in order to provide a rich feature representation for the final task. The effectiveness of such a single-modal preservation and multimodal fusion feature extraction approach is verified by multimodal segmentation experiments and an ablation study using brain tumor and prostate datasets from Medical Segmentation Decathlon (MSD). The segmentation experiments demonstrated the superiority of MPU-Net over other methods in the segmentation tasks for multimodal medical images. In the brain tumor segmentation tasks, the Dice scores (DSCs) for the whole tumor (WT), the tumor core (TC) and the enhancing tumor (ET) regions were 89.42%, 86.92%, and 84.59%, respectively. In the meanwhile, the 95% Hausdorff distance (HD95) results were 3.530, 4.899 and 2.555, respectively. In the prostate segmentation tasks, the DSCs for the peripheral zone (PZ) and the transitional zone (TZ) of the prostate were 71.20% and 90.38%, respectively. In the meanwhile, the 95% HD95 results were 6.367 and 4.766, respectively. The ablation study showed that the combination of single-modal preservation and multimodal fusion methods improved the performance of multimodal medical image feature analysis. In the segmentation tasks using brain tumor and prostate datasets, the MPU-Net method has achieved the improved performance in comparison with the conventional methods, indicating its potential application for other segmentation tasks in multimodal medical images.

An Investigation of Transfer Learning Approaches to Overcome Limited Labeled Data in Medical Image Analysis

A significant amount of research has investigated automating medical diagnosis using deep learning. However, because medical data are collected through diagnostic tests, deep learning methods used in existing studies have had a disadvantage in that the number of training samples is insufficient and the labeling cost is high. Training approaches considering the common characteristics of medical images are needed. Therefore, in this study, we investigated approaches to overcome the lack of data for representative medical imaging tasks using transfer learning technologies. The tasks were divided into image classification, object detection, and segmentation, commonly needed functions in medical image analyses. We proposed transfer learning approaches suitable for each task that can be applied when there are little medical image data available. These approaches were experimentally validated in the following applications that share similar issues of lacking data: cervical cancer classification (image classification), skin lesion detection and classification (object detection and classification), and pressure ulcer segmentation (segmentation). We also proposed multi-task learning and ensemble learning that can be applied to these applications. Finally, the approaches were compared with state-of-the-art results. In cervical cancer analysis, the performance was improved by 5.4% in sensitivity. Skin lesion classification showed improvement in accuracy of 8.7%, precision of 28.3%, and sensitivity of 39.7%. Finally, pressure ulcer segmentation improved in accuracy by 1.2%, intersection over union by 16.9%, and Dice similarity coefficient by 3.5%.