Unsupervised Domain Adaptation Approach Research Articles

Per-class curriculum for Unsupervised Domain Adaptation in semantic segmentation

AbstractAccurate training of deep neural networks for semantic segmentation requires a large number of pixel-level annotations of real images, which are expensive to generate or not even available. In this context, Unsupervised Domain Adaptation (UDA) can transfer knowledge from unlimited synthetic annotations to unlabeled real images of a given domain. UDA methods are composed of an initial training stage with labeled synthetic data followed by a second stage for feature alignment between labeled synthetic and unlabeled real data. In this paper, we propose a novel approach for UDA focusing the initial training stage, which leads to increased performance after adaptation. We introduce a curriculum strategy where each semantic class is learned progressively. Thereby, better features are obtained for the second stage. This curriculum is based on: (1) a class-scoring function to determine the difficulty of each semantic class, (2) a strategy for incremental learning based on scoring and pacing functions that limits the required training time unlike standard curriculum-based training and (3) a training loss to operate at class level. We extensively evaluate our approach as the first stage of several state-of-the-art UDA methods for semantic segmentation. Our results demonstrate significant performance enhancements across all methods: improvements of up to 10% for entropy-based techniques and 8% for adversarial methods. These findings underscore the dependency of UDA on the accuracy of the initial training. The implementation is available at https://github.com/vpulab/PCCL.

Feature Re-Representation and Reliable Pseudo Label Retraining for Cross-Domain Semantic Segmentation.

This paper presents a novel unsupervised domain adaptation method for semantic segmentation. We argue that a good representation of the target-domain data should keep both the knowledge from the source domain and the target-domain-specific information. To obtain the knowledge from the source domain, we first learn a set of bases to characterize the feature distribution of the source domain, then features from both the source and the target domain are re-represented as a weighted summation of the source bases. A discriminator is additionally introduced to make the re-representation responsibilities of both domain features under the same bases indistinguishable. In this way, the domain gap between the source re-representation and target re-representation is minimized, and the re-represented target domain features contain the source domain information. Then we combine the feature re-representation with the original domain-specific feature together for subsequent pixel-wise classification. To further make the re-represented target features semantically meaningful, a Reliable Pseudo Label Retraining (RPLR) strategy is proposed, which utilizes the consistency of the prediction by the networks trained with multi-view source images to select the clean pseudo labels on unlabeled target images for re-training. Extensive experiments demonstrate the competitive performance of our approach for unsupervised domain adaptation on the semantic segmentation benchmarks.

Domain adaptation for structural health monitoring via physics-informed and self-attention-enhanced generative adversarial learning

Health monitoring technologies, empowered by sensor-driven information and model updating, play an important role in assessing the status of civil structures and detecting anomalies. However, significant domain discrepancies exist in the distribution of physical parameters between numerical structural models and their real-world counterparts. As a result, many health monitoring theories struggle to be effective in practice, even if they perform perfectly in the simulated data. To bridge the domain discrepancies, this paper proposes an unsupervised domain adaptation approach based on an adapted cycle-consistent generative adversarial network (CycleGAN) that incorporates physical constraints and a self-attention mechanism. The approach focuses on the mutual transformation of the multi-channel time series obtained from numerical models and actual structures. Specifically, the physical constraints are derived from the governing equation of linear dynamic systems, while the self-attention mechanism is achieved by adding transformer structures to both the generator and discriminator. Through free-vibration experiments on a steel beam and a large-scale steel bridge model, the physical constraints and transformer structures have proven beneficial for improving the learning capability and training stability of the GAN model. Furthermore, the proposed approach is not only verified as effective in transforming acceleration responses between the test structures and their corresponding finite element models in both time and frequency domains but has also been shown to reproduce mode shapes accurately.

Enhancing medical image analysis with unsupervised domain adaptation approach across microscopes and magnifications

In the domain of medical image analysis, deep learning models are heralding a revolution, especially in detecting complex and nuanced features characteristic of diseases like tumors and cancers. However, the robustness and adaptability of these models across varied imaging conditions and magnifications remain a formidable challenge. This paper introduces the Fourier Adaptive Recognition System (FARS), a pioneering model primarily engineered to address adaptability in malarial parasite recognition. Yet, the foundational principles guiding FARS lend themselves seamlessly to broader applications, including tumor and cancer diagnostics. FARS capitalizes on the untapped potential of transitioning from bounding box labels to richer semantic segmentation labels, enabling a more refined examination of microscopy slides. With the integration of adversarial training and the Color Domain Aware Fourier Domain Adaptation (F2DA), the model ensures consistent feature extraction across diverse microscopy configurations. The further inclusion of category-dependent context attention amplifies FARS’s cross-domain versatility. Evidenced by a substantial elevation in cross-magnification performance from 31.3% mAP to 55.19% mAP and a 15.68% boost in cross-domain adaptability, FARS positions itself as a significant advancement in malarial parasite recognition. Furthermore, the core methodologies of FARS can serve as a blueprint for enhancing precision in other realms of medical image analysis, especially in the complex terrains of tumor and cancer imaging. The code is available at; https://github.com/Mr-TalhaIlyas/FARS.

Neural network approach for shape-based euhedral pyrite identification in X-ray CT data with adversarial unsupervised domain adaptation

We explore an attenuation and shape-based identification of euhedral pyrites in high-resolution X-ray Computed Tomography (XCT) data using deep neural networks. To deal with the scarcity of annotated data we generate a complementary training set of synthetic images. To investigate and address the domain gap between the synthetic and XCT data, several deep learning models, with and without domain adaption, are trained and compared. We find that a model trained on a small set of human annotations, while displaying over-fitting, can rival the human annotators. The unsupervised domain adaptation approaches are successful in bridging the domain gap, which significantly improves their performance. A domain-adapted model, trained on a dataset that fuses synthetic and real data, is the overall best-performing model. This highlights the possibility of using synthetic datasets for the application of deep learning in mineralogy.

An Optimal Unsupervised Domain Adaptation Approach With Applications to Pipeline Fault Diagnosis: Balancing Invariance and Variance

An Optimal Unsupervised Domain Adaptation Approach With Applications to Pipeline Fault Diagnosis: Balancing Invariance and Variance

Unsupervised Domain Adaptation With Class-Aware Memory Alignment.

Unsupervised domain adaptation (UDA) is to make predictions on unlabeled target domain by learning the knowledge from a label-rich source domain. In practice, existing UDA approaches mainly focus on minimizing the discrepancy between different domains by mini-batch training, where only a few instances are accessible at each iteration. Due to the randomness of sampling, such a batch-level alignment pattern is unstable and may lead to misalignment. To alleviate this risk, we propose class-aware memory alignment (CMA) that models the distributions of the two domains by two auxiliary class-aware memories and performs domain adaptation on these predefined memories. CMA is designed with two distinct characteristics: class-aware memories that create two symmetrical class-aware distributions for different domains and two reliability-based filtering strategies that enhance the reliability of the constructed memory. We further design a unified memory-based loss to jointly improve the transferability and discriminability of features in the memories. State-of-the-art (SOTA) comparisons and careful ablation studies show the effectiveness of our proposed CMA.

Weakly supervised end-to-end domain adaptation for person re-identification

Unsupervised domain adaptation (UDA) approaches for person re-identification (Re-ID) heavily rely on accurate pedestrian location, which is often unavailable in real-world scenarios. With respect to this situation, a two-step approach is commonly employed. However, this approach involves a complex process of disentangling UDA person Re-ID and pedestrian location, which consumes significant time and resources. This paper introduces a novel approach called Weakly Supervised End-to-End Domain Adaptation for Person Re-ID (WS-PreID) to address this problem. WS-PreID is designed to perform weakly supervised end-to-end person Re-ID by integrating pedestrian location and UDA person Re-ID in a unified manner. Our framework is a weakly supervised end-to-end network (WS-net) that operates solely on pedestrian bounding boxes. This approach mitigates the impact of background noise on the overall performance. Additionally, WS-PreID introduces a pseudo-label generation strategy with feature invariance (PLFI) to calculate pseudo-labels for person Re-ID training. PLFI utilizes both current features and comprehensive features to improve the precision of the assigned pseudo-labels. To evaluate the effectiveness of WS-PreID, we conducted comparative evaluations using the PRW and CUHK-SYSU datasets. The experimental results validate that WS-PreID achieves an end-to-end solution and significantly improves the performance of person Re-ID.

Optimal domain adaptive object detection with self-training and adversarial-based approach for construction site monitoring

In practice, object detection models used for construction site monitoring exhibit performance degradation owing to different monitoring settings and dynamic construction environments. Collecting and labeling images from new construction sites is necessary to mitigate these challenges; however, this requires significant resources. This paper presents an optimal domain-adaptive object detection model that employs an unsupervised domain adaptation approach to address the performance degradation challenges. Experiments were conducted at two construction sites to validate and identify the most effective domain-adaptive object detection model for construction sites. Several domain-adaptive object detection models have been developed, and our experimental results demonstrate that a hybrid approach, combining self-training and adversarial-based approaches improves the mean average precision by 12.8 and 7.1 over that of the baseline model at each construction site. These findings underscore the potential of domain adaptation-based methods to train object detection models in the construction domain, offering improved performance and reduced labeling efforts at specific target construction sites.

Zeroth- and first-order difference discrimination for unsupervised domain adaptation

Unsupervised domain adaptation transfers empirical knowledge from a label-rich source domain to a fully unlabeled target domain with a different distribution. A core idea of many existing approaches is to reduce the distribution divergence between domains. However, they focused only on part of the discrimination, which can be categorized into optimizing the following four objectives: reducing the intraclass distance between domains, enlarging the interclass distances between domains, reducing the intraclass distances within domains, and enlarging the interclass distances within domains. Moreover, because few methods consider multiple types of objectives, the consistency of data representations produced by different types of objectives has not yet been studied. In this paper, to address the above issues, we propose a zeroth- and first-order difference discrimination (ZFOD) approach for unsupervised domain adaptation. It first optimizes the above four objectives simultaneously. To improve the discrimination consistency of the data across the two domains, we propose a first-order difference constraint to align the interclass differences across domains. Because the proposed method needs pseudolabels for the target domain, we adopt a recent pseudolabel generation method to alleviate the negative impact of imprecise pseudolabels. We conducted an extensive comparison with nine representative conventional methods and seven remarkable deep learning-based methods on four benchmark datasets. Experimental results demonstrate that the proposed method, as a conventional approach, not only significantly outperforms the nine conventional comparison methods but is also competitive with the seven deep learning-based comparison methods. In particular, our method achieves an accuracy of 93.4% on the Office+Caltech10 dataset, which outperforms the other comparison methods. An ablation study further demonstrates the effectiveness of the proposed constraint in aligning the objectives.

Learning with limited target data to detect cells in cross-modality images.

Deep neural networks have achieved excellent cell or nucleus quantification performance in microscopy images, but they often suffer from performance degradation when applied to cross-modality imaging data. Unsupervised domain adaptation (UDA) based on generative adversarial networks (GANs) has recently improved the performance of cross-modality medical image quantification. However, current GAN-based UDA methods typically require abundant target data for model training, which is often very expensive or even impossible to obtain for real applications. In this paper, we study a more realistic yet challenging UDA situation, where (unlabeled) target training data is limited and previous work seldom delves into cell identification. We first enhance a dual GAN with task-specific modeling, which provides additional supervision signals to assist with generator learning. We explore both single-directional and bidirectional task-augmented GANs for domain adaptation. Then, we further improve the GAN by introducing a differentiable, stochastic data augmentation module to explicitly reduce discriminator overfitting. We examine source-, target-, and dual-domain data augmentation for GAN enhancement, as well as joint task and data augmentation in a unified GAN-based UDA framework. We evaluate the framework for cell detection on multiple public and in-house microscopy image datasets, which are acquired with different imaging modalities, staining protocols and/or tissue preparations. The experiments demonstrate that our method significantly boosts performance when compared with the reference baseline, and it is superior to or on par with fully supervised models that are trained with real target annotations. In addition, our method outperforms recent state-of-the-art UDA approaches by a large margin on different datasets.

Domain Adaptive Sampling for Cross-Domain Point Cloud Recognition

Point cloud recognition has recently gained increasing research interest due to the huge potential in real-world applications such as autonomous driving, robotics, etc. However, the point clouds of similar objects often exhibit notable geometric variations due to the difference in capturing devices or environmental changes. This leads to significant performance degradation when the learned point cloud recognition model is applied to a new scenario, which is also known as the domain adaptation issue. In this work, we propose a new unsupervised domain adaptation approach for point cloud recognition via domain adaptive sampling (DAS). In particular, we propose a two-level sampling strategy of point level and instance level to improve the cross-domain recognition ability of the model. First, we propose a domain adaptive point sampling (DAPS) strategy to enhance the domain-invariant representation of point clouds by progressively focusing on representative points in each point cloud based on geometric consistency. Then, we further propose an instance-level domain adaptive cloud sampling (DACS) strategy to learn target-specific information based on a self-paced learning paradigm, where we select a set of pseudo-labeled target point clouds to train our designed light-weighted adapters without modifying the learned domain-invariant representation. We validate our domain adaptive sampling approach on the benchmark datasets PointDA-10 and GraspNetPC-10, where our method achieves new state-of-the-art performance.

Leveraging topology for domain adaptive road segmentation in satellite and aerial imagery

Getting precise aspects of road through segmentation from remote sensing imagery is useful for many real-world applications such as autonomous vehicles, urban development and planning, and achieving sustainable development goals (SDGs).11https://sdgs.un.org/goals. Roads are only a small part of the image, and their appearance, type, width, elevation, directions, etc. exhibit large variations across geographical areas. Furthermore, due to differences in urbanization styles, planning, and the natural environments; regions along the roads vary significantly. Due to these variations among the train and test domains (domain shift), the road segmentation algorithms fail to generalize to new geographical locations. Unlike the generic domain alignment scenarios, road segmentation has no scene structure and generic domain adaptive segmentation methods are unable to enforce topological properties like continuity, connectivity, smoothness, etc., thus resulting in degraded domain alignment. In this work, we propose a topology-aware unsupervised domain adaptation approach for road segmentation in remote sensing imagery. During domain adaptation for road segmentation, we predict road skeleton, an auxiliary task to enforce the topological constraints. To enforce consistent predictions of road and skeleton, especially in the unlabeled target domain, the conformity loss is defined across the skeleton prediction head and the road-segmentation head. Furthermore, for self-training, we filter out the noisy pseudo-labels by using a connectivity-based pseudo-labels refinement strategy, on both road and skeleton segmentation heads, thus avoiding holes and discontinuities. Extensive experiments on the benchmark datasets show the effectiveness of the proposed approach compared to existing state-of-the-art methods. Specifically, for SpaceNet to DeepGlobe adaptation, the proposed approach outperforms the competing methods by a minimum margin of 6.6%, 6.7%, and 9.8% in IoU, F1-score, and APLS, respectively. (The source code is available on Github).

S2AC: Self-Supervised Attention Correlation Alignment Based on Mahalanobis Distance for Image Recognition

Susceptibility to domain changes for image classification hinders the application and development of deep neural networks. Domain adaptation (DA) makes use of domain-invariant characteristics to improve the performance of a model trained on labeled data from one domain (source domain) on an unlabeled domain (target) with a different data distribution. But existing DA methods simply use pretrained models (e.g., AlexNet, ResNet) for feature extraction, which are convolutional models that are trapped in localized features and fail to acquire long-distance dependencies. Furthermore, many approaches depend too much on pseudo-labels, which can impair adaptation efficiency and lead to unstable and inconsistent results. In this research, we present S2AC, a novel approach for unsupervised deep domain adaptation, that makes use of a stacked attention architecture as a feature map extractor. Our method can fuse domain discrepancy with minimizing a linear transformation of the second statistics (covariances) extended by the p-norm, while simultaneously designing pretext tasks on heuristics to improve the generality of the learning representation. In addition, we have developed a new trainable relative position embedding that not only reduces the model parameters but also enhances model accuracy and expedites the training process. To illustrate our method’s efficacy and controllability, we designed extensive experiments based on the Office31, Office_Caltech_10, and OfficeHome datasets. To the best of our knowledge, the proposed method is the first attempt at incorporating attention-based networks and self-supervised learning for image domain adaptation, and has shown promising results.

Unsupervised Domain Adaptation by Causal Learning for Biometric Signal-based HCI

Biometric signal based human-computer interface (HCI) has attracted increasing attention due to its wide application in healthcare, entertainment, neurocomputing, and so on. In recent years, deep learning-based approaches have made great progress on biometric signal processing. However, the state-of-the-art (SOTA) approaches still suffer from model degradation across subjects or sessions. In this work, we propose a novel unsupervised domain adaptation approach for biometric signal-based HCI via causal representation learning. Specifically, three kinds of interventions on biometric signals (i.e., subjects, sessions, and trials) can be selected to generalize deep models across the selected intervention. In the proposed approach, a generative model is trained for producing intervened features that are subsequently used for learning transferable and causal relations with three modes. Experiments on the EEG-based emotion recognition task and sEMG-based gesture recognition task are conducted to confirm the superiority of our approach. An improvement of +0.21% on the task of inter-subject EEG-based emotion recognition is achieved using our approach. Besides, on the task of inter-session sEMG-based gesture recognition, our approach achieves improvements of +1.47%, +3.36%, +1.71%, and +1.01% on sEMG datasets including CSL-HDEMG, CapgMyo DB-b, 3DC, and Ninapro DB6, respectively. The proposed approach also works on the task of inter-trial sEMG-based gesture recognition and an average improvement of +0.66% on Ninapro databases is achieved. These experimental results show the superiority of the proposed approach compared with the SOTA unsupervised domain adaptation methods on HCIs based on biometric signal.

An unsupervised domain adaptation approach with enhanced transferability and discriminability for bearing fault diagnosis under few-shot samples

An unsupervised domain adaptation approach with enhanced transferability and discriminability for bearing fault diagnosis under few-shot samples

Attentive continuous generative self-training for unsupervised domain adaptive medical image translation.

Self-training is an important class of unsupervised domain adaptation (UDA) approaches that are used to mitigate the problem of domain shift, when applying knowledge learned from a labeled source domain to unlabeled and heterogeneous target domains. While self-training-based UDA has shown considerable promise on discriminative tasks, including classification and segmentation, through reliable pseudo-label filtering based on the maximum softmax probability, there is a paucity of prior work on self-training-based UDA for generative tasks, including image modality translation. To fill this gap, in this work, we seek to develop a generative self-training (GST) framework for domain adaptive image translation with continuous value prediction and regression objectives. Specifically, we quantify both aleatoric and epistemic uncertainties within our GST using variational Bayes learning to measure the reliability of synthesized data. We also introduce a self-attention scheme that de-emphasizes the background region to prevent it from dominating the training process. The adaptation is then carried out by an alternating optimization scheme with target domain supervision that focuses attention on the regions with reliable pseudo-labels. We evaluated our framework on two cross-scanner/center, inter-subject translation tasks, including tagged-to-cine magnetic resonance (MR) image translation and T1-weighted MR-to-fractional anisotropy translation. Extensive validations with unpaired target domain data showed that our GST yielded superior synthesis performance in comparison to adversarial training UDA methods.

Deep discriminative clustering and structural constraint for cross-domain fault diagnosis of rotating machinery

With the rapid development of intelligent manufacturing, fault diagnostic methods based on deep learning have achieved impressive results. However, most methods require plentiful annotated samples and are based on the assumption that data from the source and target domains has the same distribution. These two conditions are difficult to satisfy in practical engineering. In light of these problems, an unsupervised domain adaptation approach named Deep Discriminative Clustering network with Structural Constraint (DDCSC) is proposed in this article. In our method, a Convolutional Neural Network (CNN) module is exploited for learning feature representations of raw data. Then a softmax module is employed to simultaneously predict class probabilities and cluster assignments of the source and target data, respectively. The learnable cluster centroids are introduced into the latent feature space to alleviate the data distribution discrepancy while better capturing the discriminative structure of the target data. In addition, geometric properties of the source data in a feature space are constrained to expand the scope of each category, which facilitates to improve prediction accuracy. An information-theoretic metric is considered as the objective function of discriminative clustering. Diagnostic experiments on a rolling bearing dataset demonstrate that our approach outperforms other popular intelligent approaches and confirms the effectiveness of discriminative clustering.

A multi-grained unsupervised domain adaptation approach for semantic segmentation

When transferring knowledge between different datasets, domain mismatch greatly hinders model’s performance. So domain adaption has been brought up to tackle the problem. Traditional methods focusing either on global or local alignment play a limited role in improving model’s performance. In this paper, we propose a multi-grained unsupervised domain adaptation approach (Muda) for semantic segmentation. Muda aims to enforce multi-grained semantic consistency between domains by aligning domains at both global and category level. Specifically, coarse-grained adaptation uses global adversarial learning on an image translation model and a main segmentation model, which respectively attempts to eliminate appearance differences and to get similar segmentation maps from two domains. While fine-grained adaptation employs an auxiliary model to adapt category information to refine pseudo labels of target data. Experiments and ablation studies are conducted on two synthetic-to-real benchmarks: GTA5 → Cityscapes and SYNTHIA → Cityscapes, which show that our model outperforms the state-of-the-art methods.

UC-SFDA: Source-free domain adaptation via uncertainty prediction and evidence-based contrastive learning

Most unsupervised domain adaptation approaches learn domain-invariant features assuming that source and target domain data are available simultaneously. In practice, the availability of source samples is only sometimes possible. This paper establishes a novel source-free domain adaptation (SFDA) framework based on uncertainty prediction and a neighborhood-guided evidence-based contrastive learning scheme. First, we develop an evidence analyzer based on the uncertainty prediction principle of Dempster–Shafer (D–S) evidence theory, which improves the network capability for discriminating different types of samples. The transformer layer with a self-attention module is adopted to capture long-distance feature dependencies such that the proposed network has better generalization ability on multiple domains. Then, we offer a high-confidence target domain sample (HCS) acquisition strategy through evidence theory, entropy criterion, and distance information. A joint confidence enhancement scheme obtains the final HCS that generates pseudo-labels. Finally, we propose an optimization method based on evidence theory, evidence-based comparative learning, and internal neighborhood structure to ensure the separability between classes and compactness within categories. Experimental results show that the proposed framework performs superiorly on two standard datasets on multiple adaptation tasks. The code for this project is available at github.com/oolown/UC-SFDA.