Semi-Supervised Medical Image Research Articles

Enhancing semi-supervised medical image segmentation with bidirectional copy-paste and masked image reconstruction

Enhancing semi-supervised medical image segmentation with bidirectional copy-paste and masked image reconstruction

Dual structure-aware image filterings for semi-supervised medical image segmentation

Semi-supervised image segmentation has attracted great attention recently. The key is how to leverage unlabeled images in the training process. Most methods maintain consistent predictions of the unlabeled images under variations (e.g., adding noise/perturbations, or creating alternative versions) in the image and/or model level. In most image-level variation, medical images often have prior structure information, which has not been well explored. In this paper, we propose novel dual structure-aware image filterings (DSAIF) as the image-level variations for semi-supervised medical image segmentation. Motivated by connected filtering that simplifies image via filtering in structure-aware tree-based image representation, we resort to the dual contrast invariant Max-tree and Min-tree representation. Specifically, we propose a novel connected filtering that removes topologically equivalent nodes (i.e. connected components) having no siblings in the Max/Min-tree. This results in two filtered images preserving topologically critical structure. Applying the proposed DSAIF to mutually supervised networks decreases the consensus of their erroneous predictions on unlabeled images. This helps to alleviate the confirmation bias issue of overfitting to noisy pseudo labels of unlabeled images, and thus effectively improves the segmentation performance. Extensive experimental results on three benchmark datasets demonstrate that the proposed method significantly/consistently outperforms some state-of-the-art methods. The source codes will be publicly available.

Beyond low-dimensional features: Enhancing semi-supervised medical image semantic segmentation with advanced consistency learning techniques

In medical imaging, semantic segmentation is crucial for accurate diagnosis. However, it is constrained by the scarcity of labeled data. To reduce the dependency on extensive annotations and improve the model’s learning efficiency and generalization capability, semi-supervised learning merges a small amount of labeled data with a large pool of unlabeled data. Unfortunately, current mainstream methods have not adapted to the subtle variations in low-dimensional features that are characteristic of medical images. Our method proposes a new consistency learning approach, specifically tailored to capture subtle nuances in low-dimensional features, such as the spatial arrangement of structures typical in medical images. It utilises a two-strong and two-weak perturbation strategy to compute multiple sets of losses. The introduction of a Multi-Head Feature Decoder module compels the model to learn beyond just colour and texture changes, enabling accurate predictions in areas with subtle variations. In the scenario where the proportion of labeled data is at its lowest, our method outperforms the current best methods on three datasets in terms of DICE scores by 2.49%, 2.57%, and 3.77%, respectively, demonstrating its significant superiority.

A Novel Perturbation Consistency Framework in Semi-Supervised Medical Image Segmentation

Semi-supervised medical image segmentation models often face challenges such as empirical mismatch and data imbalance. Traditional methods, like the two-stream perturbation model, tend to over-rely on strong perturbation, leaving weak perturbation and labeled images underutilized. To overcome these challenges, we propose an innovative hybrid copy-paste (HCP) method within the strong perturbation branch, encouraging unlabeled images to learn more comprehensive semantic information from labeled images and narrowing the empirical distribution gap. Additionally, we integrate contrastive learning into the weak perturbation branch, where contrastive learning samples are selected through semantic grouping contrastive sampling (SGCS) to address memory and variance issues. This sampling strategy ensures more effective use of weak perturbation data. This approach is particularly advantageous for pixel segmentation tasks with severely limited labels. Finally, our approach is validated on the public ACDC (Automated Cardiac Diagnosis Challenge) dataset, achieving a 90.6% DICE score, with just 7% labeled data. These results demonstrate the effectiveness of our method in improving segmentation performance with limited labeled data.

Structural tensor and frequency guided semi-supervised segmentation for medical images.

The method of semi-supervised semantic segmentation entails training with a limited number of labeled samples alongside many unlabeled samples, aiming to reduce dependence on pixel-level annotations. Most semi-supervised semantic segmentation methods primarily focus on sample augmentation in spatial dimensions to reduce the shortage of labeled samples. These methods tend to ignore the structural information of objects. In addition, frequency-domain information also supplies another perspective to evaluate information from images, which includes different properties compared to the spatialdomain. In this study, we attempt to answer these two questions: (1) is it helpful to provide structural information of objects in semi-supervised semantic segmentation tasks for medical images? (2) is it more effective to evaluate the segmentation performance in the frequency domain compared to the spatial domain for semi-supervised medical image segmentation? Therefore, we seek to introduce structural and frequency information to improve the performance of semi-supervised semantic segmentation for medicalimages. We present a novel structural tensor loss (STL) to guide feature learning on the spatial domain for semi-supervised semantic segmentation. Specifically, STL utilizes the structural information encoded in the tensors to enforce the consistency of objects across spatial regions, thereby promoting more robust and accurate feature extraction. Additionally, we proposed a frequency-domain alignment loss (FAL) to enable the neural networks to learn frequency-domain information across different augmented samples. It leverages the inherent patterns present in frequency-domain representations to guide the network in capturing and aligning features across diverse augmentation variations, thereby enhancing the model's robustness for the inputting variations. We conduct our experiments on three benchmark datasets, which include MRI (ACDC) for cardiac, CT (Synapse) for abdomen organs, and ultrasound image (BUSI) for breast lesion segmentation. The experimental results demonstrate that our method outperforms state-of-the-art semi-supervised approaches regarding the Dice similaritycoefficient. We find the proposed approach could improve the final performance of the semi-supervised medical image segmentation task. It will help reduce the need for medical image labels. Our code will are available at https://github.com/apple1986/STLFAL.

Dual-consistency guidance semi-supervised medical image segmentation with low-level detail feature augmentation

In deep-learning-based medical image segmentation tasks, semi-supervised learning can greatly reduce the dependence of the model on labeled data. However, existing semi-supervised medical image segmentation methods face the challenges of object boundary ambiguity and a small amount of available data, which limit the application of segmentation models in clinical practice. To solve these problems, we propose a novel semi-supervised medical image segmentation network based on dual-consistency guidance, which can extract reliable semantic information from unlabeled data over a large spatial and dimensional range in a simple and effective manner. This serves to improve the contribution of unlabeled data to the model accuracy. Specifically, we construct a split weak and strong consistency constraint strategy to capture data-level and feature-level consistencies from unlabeled data to improve the learning efficiency of the model. Furthermore, we design a simple multi-scale low-level detail feature enhancement module to improve the extraction of low-level detail contextual information, which is crucial to accurately locate object contours and avoid omitting small objects in semi-supervised medical image dense prediction tasks. Quantitative and qualitative evaluations on six challenging datasets demonstrate that our model outperforms other semi-supervised segmentation models in terms of segmentation accuracy and presents advantages in terms of generalizability. Code is available at https://github.com/0Jmyy0/SSMIS-DC.

Dual-branch Transformer for semi-supervised medical image segmentation.

In recent years, the use of deep learning for medical image segmentation has become a popular trend, but its development also faces some challenges. Firstly, due to the specialized nature of medical data, precise annotation is time-consuming and labor-intensive. Training neural networks effectively with limited labeled data is a significant challenge in medical image analysis. Secondly, convolutional neural networks commonly used for medical image segmentation research often focus on local features in images. However, the recognition of complex anatomical structures or irregular lesions often requires the assistance of both local and global information, which has led to a bottleneck in its development. Addressing these two issues, in this paper, we propose a novel network architecture. We integrate a shift window mechanism to learn more comprehensive semantic information and employ a semi-supervised learning strategy by incorporating a flexible amount of unlabeled data. Specifically, a typical U-shaped encoder-decoder structure is applied to obtain rich feature maps. Each encoder is designed as a dual-branch structure, containing Swin modules equipped with windows of different size to capture features of multiple scales. To effectively utilize unlabeled data, a level set function is introduced to establish consistency between the function regression and pixel classification. We conducted experiments on the COVID-19 CT dataset and DRIVE dataset and compared our approach with various semi-supervised and fully supervised learning models. On the COVID-19 CT dataset, we achieved a segmentation accuracy of up to 74.56%. Our segmentation accuracy on the DRIVE dataset was 79.79%. The results demonstrate the outstanding performance of our method on several commonly used evaluation metrics. The high segmentation accuracy of our model demonstrates that utilizing Swin modules with different window sizes can enhance the feature extraction capability of the model, and the level set function can enable semi-supervised models to more effectively utilize unlabeled data. This provides meaningful insights for the application of deep learning in medical image segmentation. Our code will be released once the manuscript is accepted for publication.

Diversity matters: Cross-head mutual mean-teaching for semi-supervised medical image segmentation

Semi-supervised medical image segmentation (SSMIS) has witnessed substantial advancements by leveraging limited labeled data and abundant unlabeled data. Nevertheless, existing state-of-the-art (SOTA) methods encounter challenges in accurately predicting labels for the unlabeled data, giving rise to disruptive noise during training and susceptibility to erroneous information overfitting. Moreover, applying perturbations to inaccurate predictions further impedes consistent learning. To address these concerns, we propose a novel cross-head mutual mean-teaching network (CMMT-Net) incorporated weak-strong data augmentations, thereby benefiting both co-training and consistency learning. More concretely, our CMMT-Net extends the cross-head co-training paradigm by introducing two auxiliary mean teacher models, which yield more accurate predictions and provide supplementary supervision. The predictions derived from weakly augmented samples generated by one mean teacher are leveraged to guide the training of another student with strongly augmented samples. Furthermore, two distinct yet synergistic data perturbations at the pixel and region levels are introduced. We propose mutual virtual adversarial training (MVAT) to smooth the decision boundary and enhance feature representations, and a cross-set CutMix strategy to generate more diverse training samples for capturing inherent structural data information. Notably, CMMT-Net simultaneously implements data, feature, and network perturbations, amplifying model diversity and generalization performance. Experimental results on three publicly available datasets indicate that our approach yields remarkable improvements over previous SOTA methods across various semi-supervised scenarios. The code is available at https://github.com/Leesoon1984/CMMT-Net.

Balanced feature fusion collaborative training for semi-supervised medical image segmentation

Collaborative learning is a fundamental component of consistency learning. It has been extensively utilized in semi-supervised medical image segmentation, primarily based on the learning of multiple models from each other. However, existing semi-supervised collaborative image segmentation methods face two primary issues. Firstly, these methods fail to fully leverage the hidden knowledge within the models during a knowledge exchange, resulting in inefficient knowledge sharing and limited generalization capabilities. To address this, we propose a novel approach, termed ‘fusion teacher’, which merges the knowledge of two models at the feature-level. This enhances the efficiency of knowledge exchange between models and generates more accurate pseudo-labels for consistency learning. Secondly, the initial and intermediate stages of collaborative learning are hindered by a significant performance gap between the fusion teacher and student models, impairs effective knowledge transfer. Our approach advocates a gradual increase in the dropout rate. This strategy enhances the transfer efficiency of knowledge from a fusion teacher to a student model. To demonstrate the efficacy of our method, we conduct experiments on the ISIC, ACDC, and AbdomenCT-1K datasets. Our approach achieves Dice scores of 87.4%, 84.8%, and 84.5%, respectively, with 10% labelled data. Compared with the current state-of-the-art (SOTA) methods, our method demonstrates strong competitiveness.

Semi-supervised medical image segmentation via cross teaching between MobileNet and MobileViT

Recently, deep learning methods that use a combination of convolutional neural networks and Transformers have shown excellent results in both completely supervised and semi-supervised medical image segmentation tasks. This study provides a simple and effective framework for semi-supervised medical image segmentation by introducing cross teaching between MobileNet and MobileViT. Specifically, we built two different two-path parallel semantic segmentation networks with MobileNet and MobileViT as the main modules. Cross teaching between MobileNet and MobileViT was performed, in which the prediction of each network was used as a pseudo-label to supervise the training of other networks in a direct end-to-end manner. Finally, experiments on two common benchmark tests showed that the proposed framework was superior to six existing semi-supervised learning methods. This shows that our framework can effectively use unlabeled data to improve the performance and is superior to the latest semi-supervised segmentation method. Codes are available at: github.com/yywbkn/MV2-cross-teaching-MobileViT.

Multi-scale spatial consistency for deep semi-supervised skin lesion segmentation

This paper introduces a novel semi-supervised framework, the Multiscale Spatial Consistency Network (MSCNet), for robust semi-supervised skin lesion segmentation. MSCNet uses local and global spatial consistency to leverage a minimal set of labeled data, supplemented by a large number of unlabeled data, to improve segmentation. The model is is based on a single Encoder–Decoder (ED) network, augmented with a Spatially-Constrained Mixture Model (SCMM) to enforce spatial coherence in predictions. To encode the local spatial consistency, a hierarchical superpixel structure is used capture local region context (LRC), bolstering the model capacity to discern fine-grained lesion details. Global consistency is enforced through the SCMM module, which uses a larger context for lesion/background discrimination. In addition, it enables efficient leveraging of the unlabeled data through pseudo-label generation. Experiments demonstrate that the MSCNet outperforms existing state-of-the-art methods in segmenting complex lesions. The MSCNet has an excellent generalization capability, offering a promising direction for semi-supervised medical image segmentation, particularly in scenarios with limited annotated data. The code is available at https://github.com/AdamaTG/MSCNet.

URCA: Uncertainty-based region clipping algorithm for semi-supervised medical image segmentation

Background and objectiveTraining convolutional neural networks based on large amount of labeled data has made great progress in the field of image segmentation. However, in medical image segmentation tasks, annotating the data is expensive and time-consuming because pixel-level annotation requires experts in the relevant field. Currently, the combination of consistent regularization and pseudo labeling-based semi-supervised methods has shown good performance in image segmentation. However, in the training process, a portion of low-confidence pseudo labels are generated by the model. And the semi-supervised segmentation method still has the problem of distribution bias between labeled and unlabeled data. The objective of this study is to address the challenges of semi-supervised learning and improve the segmentation accuracy of semi-supervised models on medical images. MethodsTo address these issues, we propose an Uncertainty-based Region Clipping Algorithm for semi-supervised medical image segmentation, which consists of two main modules. A module is introduced to compute the uncertainty of two sub-networks predictions with diversity using Monte Carlo Dropout, allowing the model to gradually learn from more reliable targets. To retain model diversity, we use different loss functions for different branches and use Non-Maximum Suppression in one of the branches. The other module is proposed to generate new samples by masking the low-confidence pixels in the original image based on uncertainty information. New samples are fed into the model to facilitate the model to generate pseudo labels with high confidence and enlarge the training data distribution. ResultsComprehensive experiments on the combination of two benchmarks ACDC and BraTS2019 show that our proposed model outperforms state-of-the-art methods in terms of Dice, HD95 and ASD. The results reach an average Dice score of 87.86 % and a HD95 score of 4.214 mm on ACDC dataset. For the brain tumor segmentation, the results reach an average Dice score of 84.79 % and a HD score of 10.13 mm. ConclusionsOur proposed method improves the accuracy of semi-supervised medical image segmentation. Extensive experiments on two public medical image datasets including 2D and 3D modalities demonstrate the superiority of our model. The code is available at: https://github.com/QuintinDong/URCA.

MVPCL: multi-view prototype consistency learning for semi-supervised medical image segmentation

MVPCL: multi-view prototype consistency learning for semi-supervised medical image segmentation

Semi-supervised CT image segmentation via contrastive learning based on entropy constraints.

Deep learning-based methods for fast target segmentation of computed tomography (CT) imaging have become increasingly popular. The success of current deep learning methods usually depends on a large amount of labeled data. Labeling medical data is a time-consuming and laborious task. Therefore, this paper aims to enhance the segmentation of CT images by using a semi-supervised learning method. In order to utilize the valid information in unlabeled data, we design a semi-supervised network model for contrastive learning based on entropy constraints. We use CNN and Transformer to capture the image's local and global feature information, respectively. In addition, the pseudo-labels generated by the teacher networks are unreliable and will lead to degradation of the model performance if they are directly added to the training. Therefore, unreliable samples with high entropy values are discarded to avoid the model extracting the wrong features. In the student network, we also introduce the residual squeeze and excitation module to learn the connection between different channels of each layer feature to obtain better segmentation performance. We demonstrate the effectiveness of the proposed method on the COVID-19 CT public dataset. We mainly considered three evaluation metrics: DSC, HD95, and JC. Compared with several existing state-of-the-art semi-supervised methods, our method improves DSC by 2.3%, JC by 2.5%, and reduces HD95 by 1.9mm.In this paper, a semi-supervised medical image segmentation method is designed by fusing CNN and Transformer and utilizing entropy-constrained contrastive learning loss, which improves the utilization of unlabeled medical images.

Review of Semi-Supervised Medical Image Segmentation based on the U-Net

Medical image segmentation is a core task in the field of medical image processing. It aims to separate areas of interest such as organs, tissues, and lesions from the background in medical images to facilitate further analysis, diagnosis, and treatment planning. Accurate image segmentation is crucial for improving the accuracy of disease diagnosis, assessing disease progression, and developing personalized treatment plans. However, fully supervised segmentation methods face the challenge of high annotation costs. With the emergence of the U-Net architecture, semi-supervised medical image segmentation based on U-Net is receiving increasing attention. This article reviews semi-supervised segmentation methods, the design concept and structure of the U-Net network, how it has been extended, and its application in semi-supervised medical image segmentation. The article also identifies the challenges faced by semi-supervised medical image segmentation techniques based on U-Net and speculates on possible future research directions. In conclusion, the article summarizes the potential of semi-supervised medical image segmentation technology based on U-Net as an accurate and efficient tool for medical diagnosis and treatment.

Semi-Supervised Medical Image Classification with Pseudo Labels Using Coalition Similarity Training

The development of medical image classification models necessitates a substantial number of labeled images for model training. In real-world scenarios, sample sizes are typically limited and labeled samples often constitute only a small portion of the dataset. This paper aims to investigate a collaborative similarity learning strategy that optimizes pseudo-labels to enhance model accuracy and expedite its convergence, known as the joint similarity learning framework. By integrating semantic similarity and instance similarity, the pseudo-labels are mutually refined to ensure their quality during initial training. Furthermore, the similarity score is utilized as a weight to guide samples away from misclassification predictions during the classification process. To enhance the model’s generalization ability, an adaptive consistency constraint is introduced into the loss function to improve performance on untrained datasets. The model achieved a satisfactory accuracy of 93.65% at 80% labeling ratio, comparable to supervised learning methods’ performance. Even with very low labeling ratio (e.g., 5%), the model still attained an accuracy of 74.28%. Comparison with other techniques such as Mean Teacher and FixMatch revealed that our approach significantly outperforms them in medical image classification tasks through improving accuracy by approximately 2%, demonstrating this framework’s leadership in medical image classification.

Semi-Supervised Medical Image Segmentation Based on Deep Consistent Collaborative Learning.

In the realm of medical image analysis, the cost associated with acquiring accurately labeled data is prohibitively high. To address the issue of label scarcity, semi-supervised learning methods are employed, utilizing unlabeled data alongside a limited set of labeled data. This paper presents a novel semi-supervised medical segmentation framework, DCCLNet (deep consistency collaborative learning UNet), grounded in deep consistent co-learning. The framework synergistically integrates consistency learning from feature and input perturbations, coupled with collaborative training between CNN (convolutional neural networks) and ViT (vision transformer), to capitalize on the learning advantages offered by these two distinct paradigms. Feature perturbation involves the application of auxiliary decoders with varied feature disturbances to the main CNN backbone, enhancing the robustness of the CNN backbone through consistency constraints generated by the auxiliary and main decoders. Input perturbation employs an MT (mean teacher) architecture wherein the main network serves as the student model guided by a teacher model subjected to input perturbations. Collaborative training aims to improve the accuracy of the main networks by encouraging mutual learning between the CNN and ViT. Experiments conducted on publicly available datasets for ACDC (automated cardiac diagnosis challenge) and Prostate datasets yielded Dice coefficients of 0.890 and 0.812, respectively. Additionally, comprehensive ablation studies were performed to demonstrate the effectiveness of each methodological contribution in this study.

Bilateral Supervision Network for Semi-Supervised Medical Image Segmentation.

Massive high-quality annotated data is required by fully-supervised learning, which is difficult to obtain for image segmentation since the pixel-level annotation is expensive, especially for medical image segmentation tasks that need domain knowledge. As an alternative solution, semi-supervised learning (SSL) can effectively alleviate the dependence on the annotated samples by leveraging abundant unlabeled samples. Among the SSL methods, mean-teacher (MT) is the most popular one. However, in MT, teacher model's weights are completely determined by student model's weights, which will lead to the training bottleneck at the late training stages. Besides, only pixel-wise consistency is applied for unlabeled data, which ignores the category information and is susceptible to noise. In this paper, we propose a bilateral supervision network with bilateral exponential moving average (bilateral-EMA), named BSNet to overcome these issues. On the one hand, both the student and teacher models are trained on labeled data, and then their weights are updated with the bilateral-EMA, and thus the two models can learn from each other. On the other hand, pseudo labels are used to perform bilateral supervision for unlabeled data. Moreover, for enhancing the supervision, we adopt adversarial learning to enforce the network generate more reliable pseudo labels for unlabeled data. We conduct extensive experiments on three datasets to evaluate the proposed BSNet, and results show that BSNet can improve the semi-supervised segmentation performance by a large margin and surpass other state-of-the-art SSL methods.

The student-teacher framework guided by self-training and consistency regularization for semi-supervised medical image segmentation.

Due to the high suitability of semi-supervised learning for medical image segmentation, a plethora of valuable research has been conducted and has achieved noteworthy success in this field. However, many approaches tend to confine their focus to a singular semi-supervised framework, thereby overlooking the potential enhancements in segmentation performance offered by integrating several frameworks. In this paper, we propose a novel semi-supervised framework named Pesudo-Label Mean Teacher (PLMT), which synergizes the self-training pipeline with pseudo-labeling and consistency regularization techniques. In particular, we integrate the student-teacher structure with consistency loss into the self-training pipeline to facilitate a mutually beneficial enhancement between the two methods. This structure not only generates remarkably accurate pseudo-labels for the self-training pipeline but also furnishes additional pseudo-label supervision for the student-teacher framework. Moreover, to explore the impact of different semi-supervised losses on the segmentation performance of the PLMT framework, we introduce adaptive loss weights. The PLMT could dynamically adjust the weights of different semi-supervised losses during the training process. Extension experiments on three public datasets demonstrate that our framework achieves the best performance and outperforms the other five semi-supervised methods. The PLMT is an initial exploration of the framework that melds the self-training pipeline with consistency regularization and offers a comparatively innovative perspective in semi-supervised image segmentation.

SC-SSL: Self-Correcting Collaborative and Contrastive Co-Training Model for Semi-Supervised Medical Image Segmentation.

Image segmentation achieves significant improvements with deep neural networks at the premise of a large scale of labeled training data, which is laborious to assure in medical image tasks. Recently, semi-supervised learning (SSL) has shown great potential in medical image segmentation. However, the influence of the learning target quality for unlabeled data is usually neglected in these SSL methods. Therefore, this study proposes a novel self-correcting co-training scheme to learn a better target that is more similar to ground-truth labels from collaborative network outputs. Our work has three-fold highlights. First, we advance the learning target generation as a learning task, improving the learning confidence for unannotated data with a self-correcting module. Second, we impose a structure constraint to encourage the shape similarity further between the improved learning target and the collaborative network outputs. Finally, we propose an innovative pixel-wise contrastive learning loss to boost the representation capacity under the guidance of an improved learning target, thus exploring unlabeled data more efficiently with the awareness of semantic context. We have extensively evaluated our method with the state-of-the-art semi-supervised approaches on four public-available datasets, including the ACDC dataset, M&Ms dataset, Pancreas-CT dataset, and Task_07 CT dataset. The experimental results with different labeled-data ratios show our proposed method's superiority over other existing methods, demonstrating its effectiveness in semi-supervised medical image segmentation.