Semi-Supervised Semantic Segmentation Research Articles

Improving Semi-Supervised and Domain-Adaptive Semantic Segmentation with Self-Supervised Depth Estimation

Training deep networks for semantic segmentation requires large amounts of labeled training data, which presents a major challenge in practice, as labeling segmentation masks is a highly labor-intensive process. To address this issue, we present a framework for semi-supervised and domain-adaptive semantic segmentation, which is enhanced by self-supervised monocular depth estimation (SDE) trained only on unlabeled image sequences. In particular, we utilize SDE as an auxiliary task comprehensively across the entire learning framework: First, we automatically select the most useful samples to be annotated for semantic segmentation based on the correlation of sample diversity and difficulty between SDE and semantic segmentation. Second, we implement a strong data augmentation by mixing images and labels using the geometry of the scene. Third, we transfer knowledge from features learned during SDE to semantic segmentation by means of transfer and multi-task learning. And fourth, we exploit additional labeled synthetic data with Cross-Domain DepthMix and Matching Geometry Sampling to align synthetic and real data. We validate the proposed model on the Cityscapes dataset, where all four contributions demonstrate significant performance gains, and achieve state-of-the-art results for semi-supervised semantic segmentation as well as for semi-supervised domain adaptation. In particular, with only 1/30 of the Cityscapes labels, our method achieves 92% of the fully-supervised baseline performance and even 97% when exploiting additional data from GTA. The source code is available at https://github.com/lhoyer/improving_segmentation_with_selfsupervised_depth.

Feature-enhanced adversarial semi-supervised semantic segmentation network for pulmonary embolism annotation

This study established a feature-enhanced adversarial semi-supervised semantic segmentation model to automatically annotate pulmonary embolism (PE) lesion areas in computed tomography pulmonary angiogram (CTPA) images. In the current study, all of the PE CTPA image segmentation methods were trained by supervised learning. However, when CTPA images come from different hospitals, the supervised learning models need to be retrained and the images need to be relabeled. Therefore, this study proposed a semi-supervised learning method to make the model applicable to different datasets by the addition of a small number of unlabeled images. By training the model with both labeled and unlabeled images, the accuracy of unlabeled images was improved and the labeling cost was reduced. Our proposed semi-supervised segmentation model included a segmentation network and a discriminator network. We added feature information generated from the encoder of the segmentation network to the discriminator so that it could learn the similarities between the prediction label and ground truth label. The HRNet-based architecture was modified and used as the segmentation network. This HRNet-based architecture could maintain a higher resolution for convolutional operations to improve the prediction of small PE lesion areas. We used a labeled open-source dataset and an unlabeled National Cheng Kung University Hospital (NCKUH) (IRB number: B-ER-108-380) dataset to train the semi-supervised learning model, and the resulting mean intersection over union (mIOU), dice score, and sensitivity reached 0.3510, 0.4854, and 0.4253, respectively, on the NCKUH dataset. Then we fine-tuned and tested the model with a small number of unlabeled PE CTPA images in a dataset from China Medical University Hospital (CMUH) (IRB number: CMUH110-REC3-173). Comparing the results of our semi-supervised model with those of the supervised model, the mIOU, dice score, and sensitivity improved from 0.2344, 0.3325, and 0.3151 to 0.3721, 0.5113, and 0.4967, respectively. In conclusion, our semi-supervised model can improve the accuracy on other datasets and reduce the labor cost of labeling with the use of only a small number of unlabeled images for fine-tuning.

Instance recognition of street trees from urban point clouds using a three-stage neural network

As one of the most important components of urban space, the geometric and semantic properties of road trees are crucial for the assessment and upgrade of urban environments. Accurate positioning of individual trees is a critical procedure for this task; however, traditional manual methods require considerable personnel and material resources. By contrast, individual tree segmentation based on a vehicle-mounted mobile laser scanner (MLS) operating along urban streets can provide a feasible and efficient solution. Recent studies (especially those based on deep learning) on point cloud semantic segmentation and forest tree detection have achieved significant progress; however, they rarely perform instance recognition of urban trees. In addition, extracting individual urban trees from point clouds to balance accuracy and efficiency remains challenging. To this end, we propose an effective pipeline to segment individual trees from urban MLS data. The proposed approach encompasses two main steps: tree-point extraction and individual-tree segmentation. First, a patch-guided semi-supervised semantic segmentation network was designed to separate non-tree points from entire urban scene. The extracted tree points are further divided into instance-level trees using a three-stage segmentation network for the individual tree segmentation. Specifically, a tree-centroid prediction module is proposed to ensure the accurate localization of street trees, even with irregular positions, in complex urban environments. Subsequently, a special confidence-aware attention mechanism is developed to resolve challenging ambiguities from complementary perspectives. Finally, a dynamic shifting-based clustering algorithm is introduced to speed up the segmentation of each tree guided by the predicted centroid and directly transfer the IDs back to the point clouds. The proposed method is evaluated and validated on MLS point clouds collected in two urban areas, robustly producing accurate instance-level roadside trees. The instance-level segmentation results in average precision and recall of (96.62%, 96.17%) and (95.62%, 94.58%), respectively, on these two MLS point cloud datasets. Point-level segmentation also exhibits mean precision, recall, and weighted coverage exceeding 90%. We also verify that the proposed method could achieve a promising result on an airborne laser scanning point cloud dataset without retraining, thereby demonstrating a good generalization ability. To further verify the quality of the segmented tree instances, a reconstruction method is adopted to employ these independent street trees in trunk point cloud modeling. Our contribution may assist in developing intelligent urban green space planning and management.

Semi-Supervised Semantic Segmentation of Remote Sensing Images Based on Dual Cross-Entropy Consistency

Semantic segmentation is a growing topic in high-resolution remote sensing image processing. The information in remote sensing images is complex, and the effectiveness of most remote sensing image semantic segmentation methods depends on the number of labels; however, labeling images requires significant time and labor costs. To solve these problems, we propose a semi-supervised semantic segmentation method based on dual cross-entropy consistency and a teacher-student structure. First, we add a channel attention mechanism to the encoding network of the teacher model to reduce the predictive entropy of the pseudo label. Secondly, the two student networks share a common coding network to ensure consistent input information entropy, and a sharpening function is used to reduce the information entropy of unsupervised predictions for both student networks. Finally, we complete the alternate training of the models via two entropy-consistent tasks: (1) semi-supervising student prediction results via pseudo-labels generated from the teacher model, (2) cross-supervision between student models. Experimental results on publicly available datasets indicate that the suggested model can fully understand the hidden information in unlabeled images and reduce the information entropy in prediction, as well as reduce the number of required labeled images with guaranteed accuracy. This allows the new method to outperform the related semi-supervised semantic segmentation algorithm at half the proportion of labeled images.

Lightweight Semantic Architecture Modeling by 3D Feature Line Detection

Existing architecture semantic modeling methods in 3D complex urban scenes continue facing difficulties, such as limited training data, lack of semantic information, and inflexible model processing. Focusing on extracting and adopting accurate semantic information into a modeling process, this work presents a framework for lightweight modeling of buildings that joints point clouds semantic segmentation and 3D feature line detection constrained by geometric and photometric consistency. The main steps are: (1) Extraction of single buildings from point clouds using 2D-3D semi-supervised semantic segmentation under photometric and geometric constraints. (2) Generation of lightweight building models by using 3D plane-constrained multi-view feature line extraction and optimization. (3) Introduction of detailed semantics of building elements into independent 3D building models by using fine-grained segmentation of multi-view images to achieve high-accuracy architecture lightweight modeling with fine-grained semantic information. Experimental results demonstrate that it can perform independent lightweight modeling of each building on point cloud at various scales and scenes, with accurate geometric appearance details and realistic textures. It also enables independent processing and analysis of each building in the scenario, making them more useful in practical applications.

Semi-Supervised Semantic Segmentation Network for Point Clouds Based on 3D Shape

The semantic segmentation of point clouds has significant applications in fields such as autonomous driving, robot vision, and smart cities. As LiDAR technology continues to develop, point clouds have gradually become the main type of 3D data. However, due to the disordered and scattered nature of point cloud data, it is challenging to effectively segment them semantically. A three-dimensional (3D) shape provides an important means of studying the spatial relationships between different objects and their structures in point clouds. Thus, this paper proposes a semi-supervised semantic segmentation network for point clouds based on 3D shape, which we call SBSNet. This network groups and encodes the geometric information of 3D objects to form shape features. It utilizes an attention mechanism and local information fusion to capture shape context information and calculate the data features. The experimental results showed that the proposed method achieved an overall intersection ratio of 85.3% in the ShapeNet dataset and 90.6% accuracy in the ModelNet40 dataset. Empirically, it showed strong performance on par or even better than state-of-the-art models.

SemiRoadExNet: A semi-supervised network for road extraction from remote sensing imagery via adversarial learning

Road extraction from remote sensing imagery is a popular and frontier research focus, since road information plays an essential role in application fields, such as urban management, map updating and traffic planning. Deep learning based methods have shown their dominance on road extraction from remote sensing imagery. However, the performance of the existing road extraction methods relies heavily on a large amount of high-quality annotated training data, which is usually hard to obtain in practice. Current semi-supervised road extraction models can effectively reduce the dependency on the labeled data, nevertheless they cannot fully utilize the latent information of low-confidence pixels in pseudo-labels effectively. These pixels are usually a border between road and non-road area and of importance for the prediction accuracy of road extraction models. In order to address these issues, we proposed a novel semi-supervised road extraction network, named SemiRoadExNet. SemiRoadExNet is based on a Generative Adversarial Network (GAN), containing one generator with two discriminators. Firstly, both labeled and unlabeled images are put into the generator network for road extraction, and the outputs of the generator not only include road segmentation results but also the corresponding entropy maps. The entropy maps represent the confidence of prediction (road or non-road) for each pixel. Then, the two discriminators enforce the feature distributions keeping the consistency of road prediction maps and entropy maps between the labeled and unlabeled data. During the adversarial training, the generator is continuously regularized by exploiting the potential information from unlabeled data, thus the generalization capacity of the proposed model can be improved effectively. Compared to several state-of-the-art semi-supervised semantic segmentation methods, the proposed SemiRoadExNet achieves 0.96–5.38% IoU improvements on DeepGlobe Road Extraction, Massachusetts Roads and CHN6-CUG datasets respectively. The source code of d SemiRoadExNet is freely available at https://github.com/hchen118/SemiRoadExNet.

Adversarial co-training for semantic segmentation over medical images

Background and objective:Abundant labeled data drives the model training for better performance, but collecting sufficient labels is still challenging. To alleviate the pressure of label collection, semi-supervised learning merges unlabeled data into training process. However, the joining of unlabeled data (e.g., data from different hospitals with different acquisition parameters) will change the original distribution. Such a distribution shift leads to a perturbation in the training process, potentially leading to a confirmation bias. In this paper, we investigate distribution shift and develop methods to increase the robustness of our models, with the goal of improving performance in semi-supervised semantic segmentation of medical images. We study distribution shift and increase model robustness to it, for improving practical performance in semi-supervised segmentation over medical images. Methods:To alleviate the issue of distribution shift, we introduce adversarial training into the co-training process. We simulate perturbations caused by the distribution shift via adversarial perturbations and introduce the adversarial perturbation to attack the supervised training to improve the robustness against the distribution shift. Benefiting from label guidance, supervised training does not collapse under adversarial attacks. For co-training, two sub-models are trained from two views (over two disjoint subsets of the dataset) to extract different kinds of knowledge independently. Co-training outperforms single-model by integrating both views of knowledge to avoid confirmation bias. Results:For practicality, we conduct extensive experiments on challenging medical datasets. Experimental results show desirable improvements to state-of-the-art counterparts (Yu and Wang, 2019; Peng et al., 2020; Perone et al., 2019). We achieve a DSC score of 87.37% with only 20% of labels on the ACDC dataset, almost same to using 100% of labels. On the SCGM dataset with more distribution shift, we achieve a DSC score of 78.65% with 6.5% of labels, surpassing 10.30% over Peng et al. (2020). Our evaluative results show superior robustness against distribution shifts in medical scenarios. Conclusion:Empirical results show the effectiveness of our work for handling distribution shift in medical scenarios.

Adversarial Dual-Student With Differentiable Spatial Warping for Semi-Supervised Semantic Segmentation

A common challenge posed to robust semantic segmentation is the expensive data annotation cost. Existing semi-supervised solutions show great potential for solving this problem. Their key idea is constructing consistency regularization with unsupervised data augmentation from unlabeled data for model training. The perturbations for unlabeled data enable the consistency training loss, which benefits semi-supervised semantic segmentation. However, these perturbations destroy image context and introduce unnatural boundaries, which is harmful for semantic segmentation. Besides, the widely adopted semi-supervised learning framework, i.e. mean-teacher, suffers performance limitation since the student model finally converges to the teacher model. In this paper, first of all, we propose a context friendly differentiable geometric warping to conduct unsupervised data augmentation; secondly, a novel adversarial dual-student framework is proposed to improve the Mean-Teacher from the following two aspects: (1) dual student models are learned independently except for a stabilization constraint to encourage exploiting model diversities; (2) adversarial training scheme is applied to both students and the discriminators are resorted to distinguish reliable pseudo-label of unlabeled data for self-training. Effectiveness is validated via extensive experiments on PASCAL VOC2012 and Cityscapes. Our solution significantly improves the performance and state-of-the-art results are achieved on both datasets. Remarkably, compared with fully supervision, our solution achieves comparable mIoU of 73.4% using only 12.5% annotated data on PASCAL VOC2012. Our codes and models are available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/cao-cong/ADS-SemiSeg</uri> .

Dense FixMatch: a simple semi-supervised learning method for pixel-wise prediction tasks

We propose Dense FixMatch, a simple method for online semi-supervised learning of dense and structured prediction tasks combining pseudo-labeling and consistency regularization via strong data augmentation. We enable the application of FixMatch in semi-supervised learning problems beyond image classification by adding a matching operation on the pseudo-labels. This allows us to still use the full strength of data augmentation pipelines, including geometric transformations. We evaluate it on semi-supervised semantic segmentation on Cityscapes and Pascal VOC with different percentages of labeled data and ablate design choices and hyper-parameters. Dense FixMatch significantly improves results compared to supervised learning using only labeled data, approaching its performance with 1/4 of the labeled samples.

Mechanical Assembly Monitoring Method Based on Semi-Supervised Semantic Segmentation

Semantic segmentation of assembly images is to recognize the assembled parts and find wrong assembly operations. However, the training of supervised semantic segmentation requires a large amount of labeled data, which is time-consuming and laborious. Moreover, the sizes of mechanical assemblies are not uniform, leading to low segmentation accuracy of small-target objects. This study proposes an adversarial learning network for semi-supervised semantic segmentation of mechanical assembly images (AdvSemiSeg-MA). A fusion method of ASFF multiscale output is proposed, which combines the outputs of different dimensions of ASFF into one output. This fusion method can make full use of the high-level semantic features and low-level fine-grained features, which helps to improve the segmentation accuracy of the model for small targets. Meanwhile, the multibranch structure RFASPP module is proposed, which enlarges the receptive field and ensures the target object is close to the center of the receptive field. The CoordConv module is introduced to allow the convolution to perceive spatial position information, thus enabling the semantic segmentation network to be position-sensitive. In the discriminator network, spectral normalization is introduced. The proposed method obtains state-of-art results on the synthesized assembly depth image dataset and performs well on actual assembly RGB image datasets.

MS4D-Net: Multitask-Based Semi-Supervised Semantic Segmentation Framework with Perturbed Dual Mean Teachers for Building Damage Assessment from High-Resolution Remote Sensing Imagery

In the aftermath of a natural hazard, rapid and accurate building damage assessment from remote sensing imagery is crucial for disaster response and rescue operations. Although recent deep learning-based studies have made considerable improvements in assessing building damage, most state-of-the-art works focus on pixel-based, multi-stage approaches, which are more complicated and suffer from partial damage recognition issues at the building-instance level. In the meantime, it is usually time-consuming to acquire sufficient labeled samples for deep learning applications, making a conventional supervised learning pipeline with vast annotation data unsuitable in time-critical disaster cases. In this study, we present an end-to-end building damage assessment framework integrating multitask semantic segmentation with semi-supervised learning to tackle these issues. Specifically, a multitask-based Siamese network followed by object-based post-processing is first constructed to solve the semantic inconsistency problem by refining damage classification results with building extraction results. Moreover, to alleviate labeled data scarcity, a consistency regularization-based semi-supervised semantic segmentation scheme with iteratively perturbed dual mean teachers is specially designed, which can significantly reinforce the network perturbations to improve model performance while maintaining high training efficiency. Furthermore, a confidence weighting strategy is embedded into the semi-supervised pipeline to focus on convincing samples and reduce the influence of noisy pseudo-labels. The comprehensive experiments on three benchmark datasets suggest that the proposed method is competitive and effective in building damage assessment under the circumstance of insufficient labels, which offers a potential artificial intelligence-based solution to respond to the urgent need for timeliness and accuracy in disaster events.

Revisiting Consistency for Semi-Supervised Semantic Segmentation.

Semi-supervised learning is an attractive technique in practical deployments of deep models since it relaxes the dependence on labeled data. It is especially important in the scope of dense prediction because pixel-level annotation requires substantial effort. This paper considers semi-supervised algorithms that enforce consistent predictions over perturbed unlabeled inputs. We study the advantages of perturbing only one of the two model instances and preventing the backward pass through the unperturbed instance. We also propose a competitive perturbation model as a composition of geometric warp and photometric jittering. We experiment with efficient models due to their importance for real-time and low-power applications. Our experiments show clear advantages of (1) one-way consistency, (2) perturbing only the student branch, and (3) strong photometric and geometric perturbations. Our perturbation model outperforms recent work and most of the contribution comes from the photometric component. Experiments with additional data from the large coarsely annotated subset of Cityscapes suggest that semi-supervised training can outperform supervised training with coarse labels. Our source code is available at https://github.com/Ivan1248/semisup-seg-efficient.

Semi-Supervised Remote Sensing Image Semantic Segmentation Method Based on Deep Learning

In this paper, we study the semi-supervised semantic segmentation problem via limited labeled samples and a large number of unlabeled samples. We propose a self-learning semi-supervised approach for the semantic segmentation of high-resolution remote sensing images. Our approach uses two networks (UNet and DeepLabV3) to predict the labels of the same unlabeled sample, and the pseudo labels samples with high prediction consistency are added to the training samples to improve the accuracy of semantic segmentation under the condition of limited labeled samples. Our method expands training data samples by using unlabeled data samples with pseudo labels. In order to verify the effectiveness of the proposed method, some experiments were conducted on the improved ISPRS Vaihingen 2D Semantic Labeling dataset using the method that we proposed. We focus on the extraction of forest and vegetation information and focus on the impact of a large number of unlabeled samples on the precision of semantic segmentation, we combine water, surface, buildings, cars, and background into one category and named others, and we call the changed dataset the improved ISPRS Vaihingen dataset. The experimental results show that the proposed method can effectively improve the semantic segmentation accuracy of high-scoring remote sensing images with limited samples than common deep semi-supervised learning.

Querying Labeled for Unlabeled: Cross-Image Semantic Consistency Guided Semi-Supervised Semantic Segmentation.

Semi-supervised semantic segmentation aims to learn a semantic segmentation model via limited labeled images and adequate unlabeled images. The key to this task is generating reliable pseudo labels for unlabeled images. Existing methods mainly focus on producing reliable pseudo labels based on the confidence scores of unlabeled images while largely ignoring the use of labeled images with accurate annotations. In this paper, we propose a Cross-Image Semantic Consistency guided Rectifying (CISC-R) approach for semi-supervised semantic segmentation, which explicitly leverages the labeled images to rectify the generated pseudo labels. Our CISC-R is inspired by the fact that images belonging to the same class have a high pixel-level correspondence. Specifically, given an unlabeled image and its initial pseudo labels, we first query a guiding labeled image that shares the same semantic information with the unlabeled image. Then, we estimate the pixel-level similarity between the unlabeled image and the queried labeled image to form a CISC map, which guides us to achieve a reliable pixel-level rectification for the pseudo labels. Extensive experiments on the PASCAL VOC 2012, Cityscapes, and COCO datasets demonstrate that the proposed CISC-R can significantly improve the quality of the pseudo labels and outperform the state-of-the-art methods. Code is available at https://github.com/Luffy03/CISC-R.

Label Propagation and Contrastive Regularization for Semisupervised Semantic Segmentation of Remote Sensing Images

Remarkable progress based on deep neural networks has been achieved on the semantic segmentation in remote sensing images. However, pixel-level labeling is expensive for remote sensing images. Semi-supervised semantic segmentation becomes an alternative approach to reduce the cost of annotation, and it is crucial to utilize efficiently a large number of unlabeled data. Nevertheless inevitably, there is the unbalanced class distribution between labeled and unlabeled data of remote sensing scene. Existing semi-supervised methods train unlabeled images in isolation from labeled images and only learn reliable pixel pseudo-labels, leading to underutilization of unlabeled images. This article proposes a novel semi-supervised semantic segmentation approach based on label propagation and contrastive regularization for remote sensing images. Specifically, the unlabeled images are augmented by randomly copy-pasting the class regions from labeled images. A prototype feature constraint module is used to enforce the constraint on the pixel features of unlabeled images relying on the prototype features from labeled images, achieving feature alignment on the entire dataset. Furthermore, we present the region contrastive learning module that guides the model to learn feature consistency under different perturbations and compact feature representations over class regions on unlabeled images. Extensive experimental results on multiple remote sensing datasets demonstrate that our proposed approach achieves superior performance compared with state-of-the-art semi-supervised semantic segmentation methods.

EI-HCR: An Efficient End-to-End Hybrid Consistency Regularization Algorithm for Semisupervised Remote Sensing Image Segmentation

Recently, remote sensing image (RSI) semantic segmentation technology has advanced greatly, with the fully supervised process achieving particularly strong performance. However, the technology depends heavily on dataset labels, leading to high annotation costs. To alleviate this problem, we propose a novel efficient end-to-end hybrid consistency regularization algorithm (EI-HCR) for the semi-supervised semantic segmentation of RSI, wherein only a few labeled images and a large number of unlabeled images are effectively used. First, we devise data perturbation (DP) consistency regularization (CR), which includes a data mix-up method to combine unlabeled and labeled images. Then, we employ teacher and student networks to conduct model perturbation (MP) CR. Both segmentation results are regarded as pseudo-labels for each other. In the end, the semi-supervised loss is composed of DP and MP consistency loss, and supervises network training along with the fully supervised loss. More importantly, we first combine the characteristics of knowledge distillation to make the student network more lightweight, efficiently reducing the model inference time. Experimental results demonstrate the effectiveness of EI-HCR on the ISPRS Vaihingen and Massachusetts Buildings datasets. With only 5% of the labeled images, EI-HCR can achieve the same accuracy as the fully supervised training with 50% of the labeled images, and the number of student model parameters is only 9.64 M, indicating the method’s great advantages over other algorithms.

Conservative-Progressive Collaborative Learning for Semi-supervised Semantic Segmentation.

Pseudo supervision is regarded as the core idea in semi-supervised learning for semantic segmentation, and there is always a tradeoff between utilizing only the high-quality pseudo labels and leveraging all the pseudo labels. Addressing that, we propose a novel learning approach, called Conservative-Progressive Collaborative Learning (CPCL), among which two predictive networks are trained in parallel, and the pseudo supervision is implemented based on both the agreement and disagreement of the two predictions. One network seeks common ground via intersection supervision and is supervised by the high-quality labels to ensure a more reliable supervision, while the other network reserves differences via union supervision and is supervised by all the pseudo labels to keep exploring with curiosity. Thus, the collaboration of conservative evolution and progressive exploration can be achieved. To reduce the influences of the suspicious pseudo labels, the loss is dynamic re-weighted according to the prediction confidence. Extensive experiments demonstrate that CPCL achieves state-of-the-art performance for semi-supervised semantic segmentation.

Semisupervised Semantic Segmentation With Certainty-Aware Consistency Training for Remote Sensing Imagery

Semi-supervised learning is a forcible method to lessen the cost of annotation for remote sensing semantic segmentation tasks. Recent related researches indicate that consistency training is one of the most effective strategies in semi-supervised learning. The core of consistency training is maintaining model outputs consistent under various perturbations. However, the current consistency training-based semi-supervised semantic segmentation frameworks lack the analysis of model uncertainty, which increases the generation of semantic ambiguity on remote sensing images. Therefore, we propose the certainty-aware consistency training (CACT) strategy to mitigate the influence of semantic ambiguity caused by model uncertainty. The certainty-aware consistency training strategy consists of two novel parts: certainty-aware consistency correction (CACC) and class-balanced adaptive threshold (CBAT). The CACC starts with generating a high-quality prediction target, then models the importance of the consistent output target and corrects the output predictions according to the certainty map, increasing the focus on reliable predictions. The CBAT uses a dynamic class-balanced adaptive threshold to filter out unreliable predictions, further reducing the impact of semantic ambiguity. Finally, considerable experimental results on the DLRSD, WHDLD, and Potsdam demonstrate that our framework has excellent performance on semi-supervised remote sensing semantic segmentation scenarios.

Adversarial Dense Contrastive Learning for Semi-supervised Semantic Segmentation.

Semi-supervised dense prediction tasks, such as semantic segmentation, can be greatly improved through the use of contrastive learning. However, this approach presents two key challenges: selecting informative negative samples from a highly redundant pool and implementing effective data augmentation. To address these challenges, we present an adversarial contrastive learning method specifically for semi-supervised semantic segmentation. Direct learning of adversarial negatives is adopted to retain discriminative information from the past, leading to higher learning efficiency. Our approach also leverages an advanced data augmentation strategy called AdverseMix, which combines information from under-performing classes to generate more diverse and challenging samples. Additionally, we use auxiliary labels and classifiers to prevent over-adversarial negatives from affecting the learning process. Our experiments on the Pascal VOC and Cityscapes datasets demonstrate that our method outperforms the state-of-the-art by a significant margin, even when using a small fraction of labeled data.