Semi-Supervised Semantic Segmentation Research Articles

View-coherent correlation consistency for semi-supervised semantic segmentation

Semi-supervised semantic segmentation needs rich and robust supervision for unlabeled data. However, promoting or punishing feature similarities with vanilla contrastive learning can be unreliable for semi-supervised semantic segmentation: pixel pairs are assigned as either positive or negative based on noisy pseudo labels, and both reliable and wrongly-assigned pairs receive uniform penalties. To address this issue, we propose correlation consistency learning, which leverages rich pairwise relationships in self-correlation matrices and matches them to the similarities between soft pseudo labels to provide robust supervision. Unlike vanilla contrastive learning, our approach prioritizes pairs with highly confident pseudo labels and applies weaker penalties for pairs that are less confident. We also introduce a strong semi-supervised learning pipeline that applies data augmentation in a view-coherent manner: even under complex augmentation strategies, for each pixel, a match can be found in different augmentation views. The novelties of the proposed method are the correlation consistency loss and the view-coherent data augmentation, and their combination gives us the view-coherent correlation consistency (VC3) system, which achieves state-of-the-art results in several semi-supervised settings on two datasets.

Semi-supervised medical image segmentation via feature similarity and reliable-region enhancement

Semantic segmentation is a crucial task in the field of computer vision, and medical image segmentation, as its downstream task, has made significant breakthroughs in recent years. However, the issue of requiring a large number of annotations in medical image segmentation has remained a major challenge. Semi-supervised semantic segmentation has provided a powerful approach to address the annotation problem. Nevertheless, existing semi-supervised semantic segmentation methods in medical images have drawbacks, such as insufficient exploitation of unlabeled data information and inefficient utilization of all pseudo-label information. We introduces a novel segmentation model, the Feature Similarity and Reliable-region Enhancement Network (FSRENet), to overcome these limitations. Firstly, this paper proposes a Feature Similarity Module (FSM), which combines the dense feature prediction ability of true labels for unlabeled images with segmentation features as additional constraints, utilizing the similarity relationship between dense features to constrain segmentation features, and thus fully exploiting the dense feature information of unlabeled data. Additionally, the Reliable-region Enhancement Module (REM) designs a high-confidence network structure by fusing two networks that can learn from each other, forming a triple-network structure. The high-confidence network generates reliable pseudo-labels that further constrain the predictions of the two networks, achieving the goal of enhancing the weight of reliable regions, reducing the noise interference of pseudo-labels, and efficiently utilizing all pseudo-label information. Experimental results on the ACDC and LA datasets demonstrate that the FSRENet model proposed in this paper excels in the task of semi-supervised semantic segmentation of medical images and outperforms the majority of existing methods. Our code is available at: https://github.com/gdghds0/FSRENet-master.

An overview of research results and applications in the field of semi-supervised semantic segmentation

Traditional segmentation approaches, supervised deep learning methods, and semi-supervised deep learning methods have all found widespread use as the field of semi-supervised semantic segmentation has advanced. These methods have developed and progressed over time, opening up novel avenues of research in the field of image segmentation and giving potent resources for tackling difficult practical issues. These developments have deepened our understanding of image segmentation and provided flexible and efficient solutions to challenges in practical applications, ranging from classical traditional approaches to supervised methods based on deep learning, and beyond to semi-supervised methods that leverage both labeled and unlabeled data. Focusing on their specialized applications in medical and remote sensing image processing, this paper presents a complete overview of the development status of these methods. This study's image segmentation solutions can help tackle actual-world issues where annotated data is rare or expensive to some extent.

Semi-supervised semantic segmentation using cross-consistency training for pavement crack detection

Crack detection is crucial to providing information for assessing pavement condition and maintaining the safety of infrastructure. Traditional manual crack detection methods are relatively time-consuming and thus have gradually been replaced by automatic detection in recent years. However, most semantic segmentation algorithms require the manual labelling of a large amount of data, which consumes a great deal of time. A semi-supervised semantic word segmentation method based on cross-consistency training for road crack detection is established in this paper. To take advantage of unlabelled crack samples, this study enforced consistency between the primary and secondary decoder predictions, using different disturbed versions of the encoder output as inputs to improve the encoder representation. When only 60% of the annotated data were used, the applied method achieved a better performance than other mainstream semantic segmentation algorithms. Therefore, the applied method can be used in automated and efficient pavement detection for pavement inspection and maintenance.

Consistency regularisation in varying contexts and feature perturbations for semi-supervised semantic segmentation of histology images

Semantic segmentation of various tissue and nuclei types in histology images is fundamental to many downstream tasks in the area of computational pathology (CPath). In recent years, Deep Learning (DL) methods have been shown to perform well on segmentation tasks but DL methods generally require a large amount of pixel-wise annotated data. Pixel-wise annotation sometimes requires expert’s knowledge and time which is laborious and costly to obtain. In this paper, we present a consistency based semi-supervised learning (SSL) approach that can help mitigate this challenge by exploiting a large amount of unlabelled data for model training thus alleviating the need for a large annotated dataset. However, SSL models might also be susceptible to changing context and features perturbations exhibiting poor generalisation due to the limited training data. We propose an SSL method that learns robust features from both labelled and unlabelled images by enforcing consistency against varying contexts and feature perturbations. The proposed method incorporates context-aware consistency by contrasting pairs of overlapping images in a pixel-wise manner from changing contexts resulting in robust and context invariant features. We show that cross-consistency training makes the encoder features invariant to different perturbations and improves the prediction confidence. Finally, entropy minimisation is employed to further boost the confidence of the final prediction maps from unlabelled data. We conduct an extensive set of experiments on two publicly available large datasets (BCSS and MoNuSeg) and show superior performance compared to the state-of-the-art methods.

Semi-supervised semantic segmentation with multi-reliability and multi-level feature augmentation

Semi-supervised semantic segmentation aims to classify the pixels with both labeled and unlabeled images. How to utilize unlabeled images is a key part of semi-supervised learning. The existing methods tend to exploit the reliable pixels which can be classified to a specific category with high reliability, ignoring the unreliable pixels which cannot be identified into a special category with low reliability, resulting in information loss. Moreover, uneven number of pixels per category often leads to misclassification of small-scale category pixels. In this paper, we propose a multi-reliability and multi-level feature augmentation framework. Multi-reliability feature augmentation dedicates to extract unreliable/reliable pixel features to take full advantage of pixel information. Multi-level feature augmentation dedicates to extract category/image level semantic features to avoid the sparse semantic information provided by category with a small number of pixels, which can lead to misclassification. Furthermore, a variety of data perturbations are employed to enhance the robustness of our model. Experimental results on the PASCAL VOC 2012, Cityscapes datasets verify the effectiveness of our method. Our code is available at https://github.com/JianJianYin/MMFA.

Twin Pseudo-training for semi-supervised semantic segmentation

Recent advancements in semi-supervised semantic segmentation have demonstrated the effectiveness of utilizing pseudo labels supervision to mitigate the limitations of pixel-wise annotations. However, pseudo-labels generated using self-training techniques typically contain a significant amount of noise, which can impede the training process of the supervised model. In this study, we identify low- and high-level semantic errors as the two key factors that hinder the accuracy of pseudo labels. To fully exploit the potential of pseudo labels, we introduce a novel semi-supervised framework named Twin Pseudo-training (TPseudo), which employs a consistency and disagreement collaboration strategy. Specifically, we correct pseudo labels with a False-positive Filter (FPF) to reduce high-level semantic noise and refine low-level semantic biases using a Semantic Error Detector (SED). Lastly, we design a Self-Adaptive Weight (SAW) loss function based on a disagreement between two predictions to exploit each pixel of pseudo labels. Experimental results on the standard benchmarks PASCAL VOC2012 and Cityscapes demonstrate the efficacy of the proposed method.

Boosting Semi-Supervised Semantic Segmentation with Probabilistic Representations

Recent breakthroughs in semi-supervised semantic segmentation have been developed through contrastive learning. In prevalent pixel-wise contrastive learning solutions, the model maps pixels to deterministic representations and regularizes them in the latent space. However, there exist inaccurate pseudo-labels which map the ambiguous representations of pixels to the wrong classes due to the limited cognitive ability of the model. In this paper, we define pixel-wise representations from a new perspective of probability theory and propose a Probabilistic Representation Contrastive Learning (PRCL) framework that improves representation quality by taking its probability into consideration. Through modelling the mapping from pixels to representations as the probability via multivariate Gaussian distributions, we can tune the contribution of the ambiguous representations to tolerate the risk of inaccurate pseudo-labels. Furthermore, we define prototypes in the form of distributions, which indicates the confidence of a class, while the point prototype cannot. More- over, we propose to regularize the distribution variance to enhance the reliability of representations. Taking advantage of these benefits, high-quality feature representations can be derived in the latent space, thereby the performance of se- mantic segmentation can be further improved. We conduct sufficient experiment to evaluate PRCL on Pascal VOC and CityScapes to demonstrate its superiority. The code is available at https://github.com/Haoyu-Xie/PRCL.

Attention-aided semantic segmentation network for weed identification in pineapple field

Based on Unmanned aerial vehicle (UAV) platforms, targeted weed management has become the mainstream method to address the harm caused by weeds in farmland, and accurate weed area identification is an important prerequisite for this approach. In this paper, we conducted research on the existing issues in weed recognition based on PSPNet. To enhance the feature extraction ability of the model and improve its performance in complex field environments, we explored the effects of three types of attention modules when inserted into different positions in the network. The experimental results showed that the best performance was achieved when the ECA module was inserted into the SPP layer of the network. Additionally, to improve the utilization of limited data and enhance model performance, we designed a semi-supervised semantic segmentation method and conducted ablation experiments to determine the optimal network parameters and verify their effectiveness. Finally, we combined these two comprehensive optimization strategies and tested them on a weed dataset. The results show that the mean pixel accuracy (MPA) and F-score of the improved weed identification model are 88.66 % and 87.79 %, 3.82 % and 2.74 % higher than that of the original network while effectively increasing prediction accuracy. This study can provide a practical reference for precise weed distribution mapping and subsequent implementation of adaptive spraying on UAV platforms.

Semi-Supervised Semantic Segmentation for Identification of Irrelevant Objects in a Waste Recycling Plant

In waste recycling plants, measuring the waste volume and weight at the beginning of the treatment process is key for a better management of resources. This task can be conducted by using orthophoto images, but it is necessary to remove from those images the objects, such as containers or trucks, that are not involved in the measurement process. This work proposes the application of deep learning for the semantic segmentation of those irrelevant objects. Several deep architectures are trained and compared, while three semi-supervised learning methods (PseudoLabeling, Distillation and Model Distillation) are proposed to take advantage of non-annotated images. In these experiments, the U-net++ architecture with an EfficientNetB3 backbone, trained with the set of labelled images, achieves the best overall multi Dice score of 91.23%. The application of semi-supervised learning methods further boosts the segmentation accuracy in a range between 1.31% and 2.59%, on average.

Editorial

Dear Readers,  I am very happy to announce the fifth regular issue of 2023. In this issue, 5 articles by 22 authors from 5 countries cover a variety of topical research aspects in computer science. Allow me to express my appreciation to all authors for their sound research and to the editorial board for the highly valuable reviews and comments for improvement. This continuous stream of relevant and novel contributions, along with the generous support of the consortium members, sustains the quality of our journal.  In the ongoing effort to further strengthen our journal, I would like to expand the editorial board: If you are a tenured associate professor or above with a strong publication record, you are welcome to apply to join our editorial board. We are also interested in high-quality proposals for special issues on new topics and trends. Please consider yourself and encourage your colleagues to submit high-quality articles or special issues for our journal.  In the fifth regular issue, I am very pleased to introduce the following five accepted articles: César Domínguez Pérez, Jónathan Heras, Eloy Mata, Vico Pascual, Lucas Fernández-Cedrón, Marcos Martínez-Lanchares, Jon Pellejero-Espinosa, Antonio Rubio-Loscertales, and Carlos Tarragona-Perez from Spain report on their deep learning approach for semi-supervised semantic segmentation to identify irrelevant objects in a waste recycling plant. In a research collaboration between Australia and Iraq, Mitchell Jensen, Khamael Al-Dulaimi, Khairiyah Saeed Abduljabbar and Jasmine Banks are focusing on their work on autoimmune disease detection in humans, more specifically on automating the classification procedure of HEp-2 stained cells from microscopic images and improving the accuracy of computer-aided diagnosis. Kashif Mehboob Khan, Warda Haider, Najeed Ahmed Khan, and Darakhshan Saleem from Pakistan address their research on big data provenance using blockchain for qualitative analytics through machine learning. Abubakhari Sserwadda, Alper Ozcan, and Yusuf Yaslan from Türkiye present their research on a novel end-to-end unified topological similarity and centrality driven hybrid deep learning model for temporal link prediction. And last but not least, Tahseen A. Wotaifi, and Ban N. Dhannoon from Iraq aim in their research to use deep learning, pre-trained models, and machine learning based on Convolution Neural Networks to predict Arabic and English fake news based on three public and available datasets: the Fake-or-Real dataset, the AraNews dataset, and the Sentimental LIAR dataset. Enjoy Reading! Cordially,  Christian Gütl, Managing Editor

Improving Semi-Supervised and Domain-Adaptive Semantic Segmentation with Self-Supervised Depth Estimation

Training deep networks for semantic segmentation requires large amounts of labeled training data, which presents a major challenge in practice, as labeling segmentation masks is a highly labor-intensive process. To address this issue, we present a framework for semi-supervised and domain-adaptive semantic segmentation, which is enhanced by self-supervised monocular depth estimation (SDE) trained only on unlabeled image sequences. In particular, we utilize SDE as an auxiliary task comprehensively across the entire learning framework: First, we automatically select the most useful samples to be annotated for semantic segmentation based on the correlation of sample diversity and difficulty between SDE and semantic segmentation. Second, we implement a strong data augmentation by mixing images and labels using the geometry of the scene. Third, we transfer knowledge from features learned during SDE to semantic segmentation by means of transfer and multi-task learning. And fourth, we exploit additional labeled synthetic data with Cross-Domain DepthMix and Matching Geometry Sampling to align synthetic and real data. We validate the proposed model on the Cityscapes dataset, where all four contributions demonstrate significant performance gains, and achieve state-of-the-art results for semi-supervised semantic segmentation as well as for semi-supervised domain adaptation. In particular, with only 1/30 of the Cityscapes labels, our method achieves 92% of the fully-supervised baseline performance and even 97% when exploiting additional data from GTA. The source code is available at https://github.com/lhoyer/improving_segmentation_with_selfsupervised_depth.

Feature-enhanced adversarial semi-supervised semantic segmentation network for pulmonary embolism annotation

This study established a feature-enhanced adversarial semi-supervised semantic segmentation model to automatically annotate pulmonary embolism (PE) lesion areas in computed tomography pulmonary angiogram (CTPA) images. In the current study, all of the PE CTPA image segmentation methods were trained by supervised learning. However, when CTPA images come from different hospitals, the supervised learning models need to be retrained and the images need to be relabeled. Therefore, this study proposed a semi-supervised learning method to make the model applicable to different datasets by the addition of a small number of unlabeled images. By training the model with both labeled and unlabeled images, the accuracy of unlabeled images was improved and the labeling cost was reduced. Our proposed semi-supervised segmentation model included a segmentation network and a discriminator network. We added feature information generated from the encoder of the segmentation network to the discriminator so that it could learn the similarities between the prediction label and ground truth label. The HRNet-based architecture was modified and used as the segmentation network. This HRNet-based architecture could maintain a higher resolution for convolutional operations to improve the prediction of small PE lesion areas. We used a labeled open-source dataset and an unlabeled National Cheng Kung University Hospital (NCKUH) (IRB number: B-ER-108-380) dataset to train the semi-supervised learning model, and the resulting mean intersection over union (mIOU), dice score, and sensitivity reached 0.3510, 0.4854, and 0.4253, respectively, on the NCKUH dataset. Then we fine-tuned and tested the model with a small number of unlabeled PE CTPA images in a dataset from China Medical University Hospital (CMUH) (IRB number: CMUH110-REC3-173). Comparing the results of our semi-supervised model with those of the supervised model, the mIOU, dice score, and sensitivity improved from 0.2344, 0.3325, and 0.3151 to 0.3721, 0.5113, and 0.4967, respectively. In conclusion, our semi-supervised model can improve the accuracy on other datasets and reduce the labor cost of labeling with the use of only a small number of unlabeled images for fine-tuning.

Instance recognition of street trees from urban point clouds using a three-stage neural network

As one of the most important components of urban space, the geometric and semantic properties of road trees are crucial for the assessment and upgrade of urban environments. Accurate positioning of individual trees is a critical procedure for this task; however, traditional manual methods require considerable personnel and material resources. By contrast, individual tree segmentation based on a vehicle-mounted mobile laser scanner (MLS) operating along urban streets can provide a feasible and efficient solution. Recent studies (especially those based on deep learning) on point cloud semantic segmentation and forest tree detection have achieved significant progress; however, they rarely perform instance recognition of urban trees. In addition, extracting individual urban trees from point clouds to balance accuracy and efficiency remains challenging. To this end, we propose an effective pipeline to segment individual trees from urban MLS data. The proposed approach encompasses two main steps: tree-point extraction and individual-tree segmentation. First, a patch-guided semi-supervised semantic segmentation network was designed to separate non-tree points from entire urban scene. The extracted tree points are further divided into instance-level trees using a three-stage segmentation network for the individual tree segmentation. Specifically, a tree-centroid prediction module is proposed to ensure the accurate localization of street trees, even with irregular positions, in complex urban environments. Subsequently, a special confidence-aware attention mechanism is developed to resolve challenging ambiguities from complementary perspectives. Finally, a dynamic shifting-based clustering algorithm is introduced to speed up the segmentation of each tree guided by the predicted centroid and directly transfer the IDs back to the point clouds. The proposed method is evaluated and validated on MLS point clouds collected in two urban areas, robustly producing accurate instance-level roadside trees. The instance-level segmentation results in average precision and recall of (96.62%, 96.17%) and (95.62%, 94.58%), respectively, on these two MLS point cloud datasets. Point-level segmentation also exhibits mean precision, recall, and weighted coverage exceeding 90%. We also verify that the proposed method could achieve a promising result on an airborne laser scanning point cloud dataset without retraining, thereby demonstrating a good generalization ability. To further verify the quality of the segmented tree instances, a reconstruction method is adopted to employ these independent street trees in trunk point cloud modeling. Our contribution may assist in developing intelligent urban green space planning and management.

Semi-Supervised Semantic Segmentation of Remote Sensing Images Based on Dual Cross-Entropy Consistency

Semantic segmentation is a growing topic in high-resolution remote sensing image processing. The information in remote sensing images is complex, and the effectiveness of most remote sensing image semantic segmentation methods depends on the number of labels; however, labeling images requires significant time and labor costs. To solve these problems, we propose a semi-supervised semantic segmentation method based on dual cross-entropy consistency and a teacher-student structure. First, we add a channel attention mechanism to the encoding network of the teacher model to reduce the predictive entropy of the pseudo label. Secondly, the two student networks share a common coding network to ensure consistent input information entropy, and a sharpening function is used to reduce the information entropy of unsupervised predictions for both student networks. Finally, we complete the alternate training of the models via two entropy-consistent tasks: (1) semi-supervising student prediction results via pseudo-labels generated from the teacher model, (2) cross-supervision between student models. Experimental results on publicly available datasets indicate that the suggested model can fully understand the hidden information in unlabeled images and reduce the information entropy in prediction, as well as reduce the number of required labeled images with guaranteed accuracy. This allows the new method to outperform the related semi-supervised semantic segmentation algorithm at half the proportion of labeled images.

Lightweight Semantic Architecture Modeling by 3D Feature Line Detection

Existing architecture semantic modeling methods in 3D complex urban scenes continue facing difficulties, such as limited training data, lack of semantic information, and inflexible model processing. Focusing on extracting and adopting accurate semantic information into a modeling process, this work presents a framework for lightweight modeling of buildings that joints point clouds semantic segmentation and 3D feature line detection constrained by geometric and photometric consistency. The main steps are: (1) Extraction of single buildings from point clouds using 2D-3D semi-supervised semantic segmentation under photometric and geometric constraints. (2) Generation of lightweight building models by using 3D plane-constrained multi-view feature line extraction and optimization. (3) Introduction of detailed semantics of building elements into independent 3D building models by using fine-grained segmentation of multi-view images to achieve high-accuracy architecture lightweight modeling with fine-grained semantic information. Experimental results demonstrate that it can perform independent lightweight modeling of each building on point cloud at various scales and scenes, with accurate geometric appearance details and realistic textures. It also enables independent processing and analysis of each building in the scenario, making them more useful in practical applications.

Semi-Supervised Semantic Segmentation Network for Point Clouds Based on 3D Shape

The semantic segmentation of point clouds has significant applications in fields such as autonomous driving, robot vision, and smart cities. As LiDAR technology continues to develop, point clouds have gradually become the main type of 3D data. However, due to the disordered and scattered nature of point cloud data, it is challenging to effectively segment them semantically. A three-dimensional (3D) shape provides an important means of studying the spatial relationships between different objects and their structures in point clouds. Thus, this paper proposes a semi-supervised semantic segmentation network for point clouds based on 3D shape, which we call SBSNet. This network groups and encodes the geometric information of 3D objects to form shape features. It utilizes an attention mechanism and local information fusion to capture shape context information and calculate the data features. The experimental results showed that the proposed method achieved an overall intersection ratio of 85.3% in the ShapeNet dataset and 90.6% accuracy in the ModelNet40 dataset. Empirically, it showed strong performance on par or even better than state-of-the-art models.

SemiRoadExNet: A semi-supervised network for road extraction from remote sensing imagery via adversarial learning

Road extraction from remote sensing imagery is a popular and frontier research focus, since road information plays an essential role in application fields, such as urban management, map updating and traffic planning. Deep learning based methods have shown their dominance on road extraction from remote sensing imagery. However, the performance of the existing road extraction methods relies heavily on a large amount of high-quality annotated training data, which is usually hard to obtain in practice. Current semi-supervised road extraction models can effectively reduce the dependency on the labeled data, nevertheless they cannot fully utilize the latent information of low-confidence pixels in pseudo-labels effectively. These pixels are usually a border between road and non-road area and of importance for the prediction accuracy of road extraction models. In order to address these issues, we proposed a novel semi-supervised road extraction network, named SemiRoadExNet. SemiRoadExNet is based on a Generative Adversarial Network (GAN), containing one generator with two discriminators. Firstly, both labeled and unlabeled images are put into the generator network for road extraction, and the outputs of the generator not only include road segmentation results but also the corresponding entropy maps. The entropy maps represent the confidence of prediction (road or non-road) for each pixel. Then, the two discriminators enforce the feature distributions keeping the consistency of road prediction maps and entropy maps between the labeled and unlabeled data. During the adversarial training, the generator is continuously regularized by exploiting the potential information from unlabeled data, thus the generalization capacity of the proposed model can be improved effectively. Compared to several state-of-the-art semi-supervised semantic segmentation methods, the proposed SemiRoadExNet achieves 0.96–5.38% IoU improvements on DeepGlobe Road Extraction, Massachusetts Roads and CHN6-CUG datasets respectively. The source code of d SemiRoadExNet is freely available at https://github.com/hchen118/SemiRoadExNet.

Adversarial co-training for semantic segmentation over medical images

Background and objective:Abundant labeled data drives the model training for better performance, but collecting sufficient labels is still challenging. To alleviate the pressure of label collection, semi-supervised learning merges unlabeled data into training process. However, the joining of unlabeled data (e.g., data from different hospitals with different acquisition parameters) will change the original distribution. Such a distribution shift leads to a perturbation in the training process, potentially leading to a confirmation bias. In this paper, we investigate distribution shift and develop methods to increase the robustness of our models, with the goal of improving performance in semi-supervised semantic segmentation of medical images. We study distribution shift and increase model robustness to it, for improving practical performance in semi-supervised segmentation over medical images. Methods:To alleviate the issue of distribution shift, we introduce adversarial training into the co-training process. We simulate perturbations caused by the distribution shift via adversarial perturbations and introduce the adversarial perturbation to attack the supervised training to improve the robustness against the distribution shift. Benefiting from label guidance, supervised training does not collapse under adversarial attacks. For co-training, two sub-models are trained from two views (over two disjoint subsets of the dataset) to extract different kinds of knowledge independently. Co-training outperforms single-model by integrating both views of knowledge to avoid confirmation bias. Results:For practicality, we conduct extensive experiments on challenging medical datasets. Experimental results show desirable improvements to state-of-the-art counterparts (Yu and Wang, 2019; Peng et al., 2020; Perone et al., 2019). We achieve a DSC score of 87.37% with only 20% of labels on the ACDC dataset, almost same to using 100% of labels. On the SCGM dataset with more distribution shift, we achieve a DSC score of 78.65% with 6.5% of labels, surpassing 10.30% over Peng et al. (2020). Our evaluative results show superior robustness against distribution shifts in medical scenarios. Conclusion:Empirical results show the effectiveness of our work for handling distribution shift in medical scenarios.

Adversarial Dual-Student With Differentiable Spatial Warping for Semi-Supervised Semantic Segmentation

A common challenge posed to robust semantic segmentation is the expensive data annotation cost. Existing semi-supervised solutions show great potential for solving this problem. Their key idea is constructing consistency regularization with unsupervised data augmentation from unlabeled data for model training. The perturbations for unlabeled data enable the consistency training loss, which benefits semi-supervised semantic segmentation. However, these perturbations destroy image context and introduce unnatural boundaries, which is harmful for semantic segmentation. Besides, the widely adopted semi-supervised learning framework, i.e. mean-teacher, suffers performance limitation since the student model finally converges to the teacher model. In this paper, first of all, we propose a context friendly differentiable geometric warping to conduct unsupervised data augmentation; secondly, a novel adversarial dual-student framework is proposed to improve the Mean-Teacher from the following two aspects: (1) dual student models are learned independently except for a stabilization constraint to encourage exploiting model diversities; (2) adversarial training scheme is applied to both students and the discriminators are resorted to distinguish reliable pseudo-label of unlabeled data for self-training. Effectiveness is validated via extensive experiments on PASCAL VOC2012 and Cityscapes. Our solution significantly improves the performance and state-of-the-art results are achieved on both datasets. Remarkably, compared with fully supervision, our solution achieves comparable mIoU of 73.4% using only 12.5% annotated data on PASCAL VOC2012. Our codes and models are available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/cao-cong/ADS-SemiSeg</uri> .