Unsupervised Domain Adaptation Methods Research Articles

A Domain-Adaptive Segmentation Method Based on Segment Anything Model for Mechanical Assembly

Semantic segmentation of assembly images can effectively identify and prevent non-standard behaviors such as wrong assembly and missing assembly in the assembly process. However, supervised methods require massive training samples and are time-consuming to label. Unsupervised domain adaptation methods can address this issue without labeling the target data. Recently, the large-scale Segment Anything Model (SAM) has achieved significant success in semantic segmentation owing to its powerful feature extraction and instant interaction capabilities. Based on this, this paper proposed a new framework called U-SAM for unsupervised domain-adaptive semantic segmentation. Compared to other frameworks, U-SAM can obtain more segmented features while maintaining shallower features. Meanwhile, this study proposed an iterative loop strategy for the prompt module in U-SAM. This strategy gradually improves the accuracy of target prompt labels and solves the problem of no prompt labels in the target data. Additionally, a novel method of using oblique boxes instead of regular boxes was developed to achieve higher box prompt quality. Finally, a line discriminator module and a trainable line-guided filtering module were designed to improve the quality of edge segmentation. The experimental results indicate that the U-SAM network framework achieves a Dice accuracy of 84.4% for semantic segmentation on the mechanical assembly image dataset. This confirmed that our proposed method can be applied to the assembly monitoring process of target data.

Unsupervised domain adaptation for the semantic segmentation of remote sensing images via a class-aware Fourier transform and a fine-grained discriminator

The semantic segmentation of remote sensing images is vital for Earth observation purposes. However, its performance can decline significantly due to differences in dataset distributions between training (source) and deployment (target) settings. Unsupervised domain adaptation can be used to counter this problem by leveraging the knowledge acquired from the labelled source domain and by adapting it to the unlabelled target domain. Existing methods focus on either input-level or feature-level alignments, which can be sub-optimal for addressing large domain gaps. To this end, this paper introduces a new unsupervised domain adaptation method that employs concurrently two levels of alignment: first, at the input level, an adaptive Fourier-based image-to-image translation approach is utilised to generate target-styled source images with class-based low-amplitude changes. Then, at the feature level, an adaptive fine-grained domain discriminator is introduced that incorporates class information into two parallel discriminators, for source vs. target and target-styled source image vs. target settings. Experimental results indicate that the proposed method improves significantly cross-domain semantic segmentation performance with respect to the state-of-the-art.

A review of optic disc and optic cup segmentation based on fundus images

AbstractOptic disc (OD) and optic cup (OC) segmentation is an important task in ophthalmic medicine and is crucial for aiding glaucoma screening. With the development of smart healthcare and the increase of large datasets, there is an increasing number of research efforts targeting OD and OC segmentation, making it particularly important to provide a systematic review of the latest advances in the field. This paper presents a systematic review of commonly used datasets, evaluation metrics, and related research results in the field of OD and OC segmentation. The advantages and disadvantages of segmentation techniques based on traditional and deep learning methods are comparatively analysed. In addition, this study emphasizes the importance of OD and OC segmentation efforts in smart healthcare. Despite the technological advances, the lack of generalization capability is still a major obstacle limiting its clinical application. To address this issue, this study explores unsupervised domain adaptation methods to enhance the generalization performance of segmentation techniques and provide new strategies for clinical diagnosis. Finally, this paper discusses the challenges and future research directions faced by OD and OC segmentation when applied in the medical field to help readers comprehensively grasp the research dynamics in this area.

Dual Classifier Adaptation: Source-Free UDA via Adaptive Pseudo-Labels Learning

Different from Unsupervised Domain Adaptation (UDA), Source-Free Unsupervised Domain Adaptation (SFUDA) transfers source knowledge to target domain without accessing the source data, using only the source model, has attracted much attention recently. One mainstream SFUDA method fine-tunes the source model by self-training to generate pseudo-labels of the target data. However, due to the significant differences between different domains, these target pseudo-labels often contain some noise, and it will inevitably degenerates the target performance. For this purpose, we propose an innovative SFUDA method with adaptive pseudo-labels learning named Dual Classifier Adaptation (DCA). In DCA, a dual classifier structure is introduced to adaptively learn target pseudo-labels by cooperation between source and target classifiers. Simultaneously, the minimax entropy is introduced for target learning, in order to adapt target data to source model, while capture the intrinsic cluster structure in target domain as well. After compared our proposed method DCA with a range of UDA and SFUDA methods, DCA achieves far ahead performance on several benchmark datasets.

A Co-Training Framework for Heterogeneous Heuristic Domain Adaptation.

The purpose of this article is to address unsupervised domain adaptation (UDA) where a labeled source domain and an unlabeled target domain are given. Recent advanced UDA methods attempt to remove domain-specific properties by separating domain-specific information from domain-invariant representations, which heavily rely on the designed neural network structures. Meanwhile, they do not consider class discriminate representations when learning domain-invariant representations. To this end, this article proposes a co-training framework for heterogeneous heuristic domain adaptation (CO-HHDA) to address the above issues. First, a heterogeneous heuristic network is introduced to model domain-specific characters. It allows structures of heuristic network to be different between domains to avoid underfitting or overfitting. Specially, we initialize a small structure that is shared between domains and increase a subnetwork for the domain which preserves rich specific information. Second, we propose a co-training scheme to train two classifiers, a source classifier and a target classifier, to enhance class discriminate representations. The two classifiers are designed based on domain-invariant representations, where the source classifier learns from the labeled source data, and the target classifier is trained from the generated target pseudolabeled data. The two classifiers teach each other in the training process with high-quality pseudolabeled data. Meanwhile, an adaptive threshold is presented to select reliable pseudolabels in each classifier. Empirical results on three commonly used benchmark datasets demonstrate that the proposed CO-HHDA outperforms the state-of-the-art domain adaptation methods.

Per-class curriculum for Unsupervised Domain Adaptation in semantic segmentation

AbstractAccurate training of deep neural networks for semantic segmentation requires a large number of pixel-level annotations of real images, which are expensive to generate or not even available. In this context, Unsupervised Domain Adaptation (UDA) can transfer knowledge from unlimited synthetic annotations to unlabeled real images of a given domain. UDA methods are composed of an initial training stage with labeled synthetic data followed by a second stage for feature alignment between labeled synthetic and unlabeled real data. In this paper, we propose a novel approach for UDA focusing the initial training stage, which leads to increased performance after adaptation. We introduce a curriculum strategy where each semantic class is learned progressively. Thereby, better features are obtained for the second stage. This curriculum is based on: (1) a class-scoring function to determine the difficulty of each semantic class, (2) a strategy for incremental learning based on scoring and pacing functions that limits the required training time unlike standard curriculum-based training and (3) a training loss to operate at class level. We extensively evaluate our approach as the first stage of several state-of-the-art UDA methods for semantic segmentation. Our results demonstrate significant performance enhancements across all methods: improvements of up to 10% for entropy-based techniques and 8% for adversarial methods. These findings underscore the dependency of UDA on the accuracy of the initial training. The implementation is available at https://github.com/vpulab/PCCL.

A Multi-Modal Egocentric Activity Recognition Approach towards Video Domain Generalization.

Egocentric activity recognition is a prominent computer vision task that is based on the use of wearable cameras. Since egocentric videos are captured through the perspective of the person wearing the camera, her/his body motions severely complicate the video content, imposing several challenges. In this work we propose a novel approach for domain-generalized egocentric human activity recognition. Typical approaches use a large amount of training data, aiming to cover all possible variants of each action. Moreover, several recent approaches have attempted to handle discrepancies between domains with a variety of costly and mostly unsupervised domain adaptation methods. In our approach we show that through simple manipulation of available source domain data and with minor involvement from the target domain, we are able to produce robust models, able to adequately predict human activity in egocentric video sequences. To this end, we introduce a novel three-stream deep neural network architecture combining elements of vision transformers and residual neural networks which are trained using multi-modal data. We evaluate the proposed approach using a challenging, egocentric video dataset and demonstrate its superiority over recent, state-of-the-art research works.

SFDA-CD: A Source-Free Unsupervised Domain Adaptation for VHR Image Change Detection

Deep models may have disappointing performance in real applications due to the domain shifts in data distributions between the source and target domain. Although a few unsupervised domain adaptation methods have been proposed to make the pre-train models effective on target domain datasets, constraints like data privacy, security, and transmission limits restrict access to VHR remote sensing images, making existing unsupervised domain adaptation methods almost ineffective in specific change detection areas. Therefore, we propose a source-free unsupervised domain adaptation change detection structure to complete specific change detection tasks, using only the pre-trained source model and unlabelled target data. The GAN-based source generation component is designed to generate synthetic source data, which, to some extent, reflects the distribution of the source domain. Moreover, these data can be utilised in model knowledge transfer. The model adaptation component facilitates knowledge transfer between models by minimising the differences between deep features, using AAM (Attention Adaptation Module) to extract the difference between high-level features, meanwhile we proposed ISM (Intra-domain Self-supervised Module) to train target model in a self-supervised strategy in order to improve the knowledge adaptation. Our SFDA-CD framework demonstrates superior accuracy over existing unsupervised domain adaptation change detection methods, which has 0.6% cIoU and 1.5% F1 score up in cross-regional tasks and 1.4% cIoU and 1.9% F1 score up in cross-scenario tasks, proving that it can effectively reduce the domain shift between the source and target domains even without access to source data. Additionally, it can facilitate knowledge transfer from the source model to the target model.

DDAGCN: an unsupervised cross-domain identification method for tie rod bolt loosening in a rod-fastening rotor system under different working conditions

The cross-domain diagnosis of tie rod bolt loosening is essential for guaranteeing the healthy operation of rod-fastening rotor (RFR) systems. The unsupervised domain adaptation (UDA) method effectively alleviates the impact of domain discrepancy and has been applied for cross-domain diagnosis. Traditional UDA methods mainly focus on the marginal and conditional distributions with fixed weights to adapt the domain distribution discrepancy. However, the fixed distribution combination cannot satisfy the requirement of feature domain alignment under different working conditions, and the relative importance of the two distributions cannot be evaluated quantitatively. This paper proposes an improved dynamic distribution adaptive graph convolutional network (DDAGCN) for the cross-domain diagnosis of tie rod bolt loosening under different working conditions. This method can quantitatively evaluate the relative significance of each distribution in representing the distribution discrepancy. First, it combines the convolutional neural network and the graph convolutional network to extract the features in the graph structure by using the connection relationship between nodes, and realizes the full extraction of neighbourhood information of nodes. Then, the dynamic distribution adaptive alignment strategy is introduced to construct the dynamic linear combination of marginal and conditional distributions, so as to measure the distribution discrepancy between domains. Meanwhile, the domain adversarial module is combined to further reduce the domain gap and finally realize feature alignment. The extracted domain invariant features can effectively enhance the generalization ability and fault identification ability of the model. The case of the public bearing dataset verifies that the effectiveness and generalization ability of the proposed method for cross-domain fault diagnosis under different working conditions is superior to other compared methods. In addition, the identification ability of the proposed method for the degree of tie rod bolt loosening is verified by the self-made bolt loosening dataset of the RFR system.

Domain Generalizable Person Search Using Unreal Dataset

Collecting and labeling real datasets to train the person search networks not only requires a lot of time and effort, but also accompanies privacy issues. The weakly-supervised and unsupervised domain adaptation methods have been proposed to alleviate the labeling burden for target datasets, however, their generalization capability is limited. We introduce a novel person search method based on the domain generalization framework, that uses an automatically labeled unreal dataset only for training but is applicable to arbitrary unseen real datasets. To alleviate the domain gaps when transferring the knowledge from the unreal source dataset to the real target datasets, we estimate the fidelity of person instances which is then used to train the end-to-end network adaptively. Moreover, we devise a domain-invariant feature learning scheme to encourage the network to suppress the domain-related features. Experimental results demonstrate that the proposed method provides the competitive performance to existing person search methods even though it is applicable to arbitrary unseen datasets without any prior knowledge and re-training burdens.

CMDA: Cross-Modal and Domain Adversarial Adaptation for LiDAR-Based 3D Object Detection

Recent LiDAR-based 3D Object Detection (3DOD) methods show promising results, but they often do not generalize well to target domains outside the source (or training) data distribution. To reduce such domain gaps and thus to make 3DOD models more generalizable, we introduce a novel unsupervised domain adaptation (UDA) method, called CMDA, which (i) leverages visual semantic cues from an image modality (i.e., camera images) as an effective semantic bridge to close the domain gap in the cross-modal Bird's Eye View (BEV) representations. Further, (ii) we also introduce a self-training-based learning strategy, wherein a model is adversarially trained to generate domain-invariant features, which disrupt the discrimination of whether a feature instance comes from a source or an unseen target domain. Overall, our CMDA framework guides the 3DOD model to generate highly informative and domain-adaptive features for novel data distributions. In our extensive experiments with large-scale benchmarks, such as nuScenes, Waymo, and KITTI, those mentioned above provide significant performance gains for UDA tasks, achieving state-of-the-art performance.

Gaze from Origin: Learning for Generalized Gaze Estimation by Embedding the Gaze Frontalization Process

Gaze estimation aims to accurately estimate the direction or position at which a person is looking. With the development of deep learning techniques, a number of gaze estimation methods have been proposed and achieved state-of-the-art performance. However, these methods are limited to within-dataset settings, whose performance drops when tested on unseen datasets. We argue that this is caused by infinite and continuous gaze labels. To alleviate this problem, we propose using gaze frontalization as an auxiliary task to constrain gaze estimation. Based on this, we propose a novel gaze domain generalization framework named Gaze Frontalization-based Auxiliary Learning (GFAL) Framework which embeds the gaze frontalization process, i.e., guiding the feature so that the eyeball can rotate and look at the front (camera), without any target domain information during training. Experimental results show that our proposed framework is able to achieve state-of-the-art performance on gaze domain generalization task, which is competitive with or even superior to the SOTA gaze unsupervised domain adaptation methods.

Unsupervised domain adaptation of dynamic extension networks based on class decision boundaries

In response to the problems of inaccurate feature alignment, loss of source domain information, imbalanced sample distribution, and biased class decision boundaries in traditional unsupervised domain adaptation methods, this paper proposes a class decision boundary-based dynamic expansion network unsupervised domain adaptation method called CDE-Net. Specifically, our method dynamically expands the autoencoder-based network structure, which can preserve source domain feature information while gradually adapting to the target domain data distribution and learning useful feature information from the target domain. Meanwhile, by minimizing clustering loss and conditional entropy loss, CDE-Net can explore the intrinsic structure of the data and push class decision boundaries away from dense data areas. We experimentally verify our method on three medical image datasets, chest X-rays, intracranial hemorrhage, and mammography, and achieve an average AUC improvement of 25.8% or more compared to non-transfer methods. In addition, we compare our method with previous unsupervised domain adaptation methods, and the experimental results show that our method achieves better classification accuracy and generalization performance.

Consecutive knowledge meta-adaptation learning for unsupervised medical diagnosis

Deep learning-based computer-aided diagnosis has garnered significant attention in both academic research and clinical applications. Due to the challenges in collecting well-labeled clinical data, unsupervised domain adaptation methods have become widely used in medical image analysis. However, existing unsupervised domain adaptation methods in the clinic field fail to adapt to scenarios where new, unlabeled target data continually arrive in an online manner. To tackle this real-world clinical challenge, a meta-adaptation framework is innovatively developed, termed consecutive lesion knowledge meta-adaptation, primarily comprising semantic adaptation and representation adaptation phases. Specifically, the semantic knowledge learned from the source lesion domain will be transferred to the consecutive target lesion in the semantic adaptation phase. To further align the transferable representation knowledge across the source and multiple target lesion domains, the feature-extractor is optimized in the representation adaptation phase. Furthermore, to mitigate catastrophic forgetting during continuous unsupervised domain adaptation, a domain-quantizer is designed to preserve utility knowledge from previously learned target lesion domains. Moreover, as semantic knowledge is continuously evolving and artificially designed kernels struggle to model such complex distributions, a self-adaptive kernel is designed to flexibly measure the shift between the source and consecutive target lesion domains during the semantic adaptation phase. Experimental results on medical image classification tasks demonstrate the effectiveness of the proposed method.

Unsupervised Multitarget Domain Adaptation With Dictionary-Bridged Knowledge Exploitation.

Unsupervised domain adaptation (UDA) is an emerging learning paradigm that models on unlabeled datasets by leveraging model knowledge built on other labeled datasets, in which the statistical distributions of these datasets are usually not identical. Formally, UDA is to leverage knowledge from a labeled source domain to promote an unlabeled target domain. Although there have been a variety of methods proposed to address the UDA problem, most of them are dedicated to single-source-to-single-target domain, while the works on single-source-to-multitarget domain are relatively rare. Compared to the single-source domain with single-target domain scenario, the UDA from single-source domain to multitarget domain is more challenging since it needs to consider not only the relationships between the source and the target domains but also those among the target domains. To this end, this article proposes a kind of dictionary learning-based unsupervised multitarget domain adaptation method (DL-UMTDA). In DL-UMTDA, a common dictionary is constructed to correlate the single-source and multitarget domains, while individual dictionaries are designed to exploit the private knowledge for the target domains. Through learning the corresponding dictionary representation coefficients in the UDA process, the correlations from the source to the target domains as well as these potential relationships between the target domains can be effectively exploited. In addition, we design an alternating algorithm to solve the DL-UMTDA model with theoretical convergence guarantee. Finally, extensive experiments on benchmark (Office + Caltech) and real datasets (AgeDB, Morph, and CACD) validate the superiority of the proposed method.

Feature Re-Representation and Reliable Pseudo Label Retraining for Cross-Domain Semantic Segmentation.

This paper presents a novel unsupervised domain adaptation method for semantic segmentation. We argue that a good representation of the target-domain data should keep both the knowledge from the source domain and the target-domain-specific information. To obtain the knowledge from the source domain, we first learn a set of bases to characterize the feature distribution of the source domain, then features from both the source and the target domain are re-represented as a weighted summation of the source bases. A discriminator is additionally introduced to make the re-representation responsibilities of both domain features under the same bases indistinguishable. In this way, the domain gap between the source re-representation and target re-representation is minimized, and the re-represented target domain features contain the source domain information. Then we combine the feature re-representation with the original domain-specific feature together for subsequent pixel-wise classification. To further make the re-represented target features semantically meaningful, a Reliable Pseudo Label Retraining (RPLR) strategy is proposed, which utilizes the consistency of the prediction by the networks trained with multi-view source images to select the clean pseudo labels on unlabeled target images for re-training. Extensive experiments demonstrate the competitive performance of our approach for unsupervised domain adaptation on the semantic segmentation benchmarks.

Learning to Generalize over Subpartitions for Heterogeneity-Aware Domain Adaptive Nuclei Segmentation

Abstract Annotation scarcity and cross-modality/stain data distribution shifts are two major obstacles hindering the application of deep learning models for nuclei analysis, which holds a broad spectrum of potential applications in digital pathology. Recently, unsupervised domain adaptation (UDA) methods have been proposed to mitigate the distributional gap between different imaging modalities for unsupervised nuclei segmentation in histopathology images. However, existing UDA methods are built upon the assumption that data distributions within each domain should be uniform. Based on the over-simplified supposition, they propose to align the histopathology target domain with the source domain integrally, neglecting severe intra-domain discrepancy over subpartitions incurred by mixed cancer types and sampling organs. In this paper, for the first time, we propose to explicitly consider the heterogeneity within the histopathology domain and introduce open compound domain adaptation (OCDA) to resolve the crux. In specific, a two-stage disentanglement framework is proposed to acquire domain-invariant feature representations at both image and instance levels. The holistic design addresses the limitations of existing OCDA approaches which struggle to capture instance-wise variations. Two regularization strategies are specifically devised herein to leverage the rich subpartition-specific characteristics in histopathology images and facilitate subdomain decomposition. Moreover, we propose a dual-branch nucleus shape and structure preserving module to prevent nucleus over-generation and deformation in the synthesized images. Experimental results on both cross-modality and cross-stain scenarios over a broad range of diverse datasets demonstrate the superiority of our method compared with state-of-the-art UDA and OCDA methods. Graphical abstract

Simplifying open-set video domain adaptation with contrastive learning

In an effort to reduce annotation costs in action recognition, unsupervised video domain adaptation methods have been proposed that aim to adapt a predictive model from a labelled dataset (i.e., source domain) to an unlabelled dataset (i.e., target domain). In this work we address a more realistic scenario, called open-set video domain adaptation (OUVDA), where the target dataset contains “unknown” semantic categories that are not shared with the source. The challenge lies in aligning the shared classes of the two domains while separating the shared classes from the unknown ones. In this work we propose to address OUVDA with an unified contrastive learning framework that learns discriminative and well-clustered features. We also propose a video-oriented temporal contrastive loss that enables our method to better cluster the feature space by exploiting the freely available temporal information in video data. We show that discriminative feature space facilitates better separation of the unknown classes, and thereby allows us to use a simple similarity based score to identify them. We conduct thorough experimental evaluation on multiple OUVDA benchmarks and show the effectiveness of our proposed method against the prior art.

ST-GAN: A Swin Transformer-Based Generative Adversarial Network for Unsupervised Domain Adaptation of Cross-Modality Cardiac Segmentation.

Unsupervised domain adaptation (UDA) methods have shown great potential in cross-modality medical image segmentation tasks, where target domain labels are unavailable. However, the domain shift among different image modalities remains challenging, because the conventional UDA methods are based on convolutional neural networks (CNNs), which tend to focus on the texture of images and cannot establish the global semantic relevance of features due to the locality of CNNs. This paper proposes a novel end-to-end Swin Transformer-based generative adversarial network (ST-GAN) for cross-modality cardiac segmentation. In the generator of ST-GAN, we utilize the local receptive fields of CNNs to capture spatial information and introduce the Swin Transformer to extract global semantic information, which enables the generator to better extract the domain-invariant features in UDA tasks. In addition, we design a multi-scale feature fuser to sufficiently fuse the features acquired at different stages and improve the robustness of the UDA network. We extensively evaluated our method with two cross-modality cardiac segmentation tasks on the MS-CMR 2019 dataset and the M&Ms dataset. The results of two different tasks show the validity of ST-GAN compared with the state-of-the-art cross-modality cardiac image segmentation methods.

Unsupervised Domain Adaptation for Medical Image Segmentation Using Transformer With Meta Attention.

Image segmentation is essential to medical image analysis as it provides the labeled regions of interest for the subsequent diagnosis and treatment. However, fully-supervised segmentation methods require high-quality annotations produced by experts, which is laborious and expensive. In addition, when performing segmentation on another unlabeled image modality, the segmentation performance will be adversely affected due to the domain shift. Unsupervised domain adaptation (UDA) is an effective way to tackle these problems, but the performance of the existing methods is still desired to improve. Also, despite the effectiveness of recent Transformer-based methods in medical image segmentation, the adaptability of Transformers is rarely investigated. In this paper, we present a novel UDA framework using a Transformer for building a cross-modality segmentation method with the advantages of learning long-range dependencies and transferring attentive information. To fully utilize the attention learned by the Transformer in UDA, we propose Meta Attention (MA) and use it to perform a fully attention-based alignment scheme, which can learn the hierarchical consistencies of attention and transfer more discriminative information between two modalities. We have conducted extensive experiments on cross-modality segmentation using three datasets, including a whole heart segmentation dataset (MMWHS), an abdominal organ segmentation dataset, and a brain tumor segmentation dataset. The promising results show that our method can significantly improve performance compared with the state-of-the-art UDA methods.