Unsupervised Domain Adaptation Framework Research Articles

Shape-Consistent One-Shot Unsupervised Domain Adaptation for Rail Surface Defect Segmentation

Deep neural networks have greatly improved the performance of rail surface defect segmentation when the test samples have the same distribution as the training samples. However, in practical inspection scenarios, the rail surface exhibits variations in appearance due to different service time and natural conditions. Conventional deep learning models show limited generalization in scenes with distribution differences. To address this problem, we propose a novel one-shot unsupervised domain adaptation framework. Specifically, we introduce a shape consistent style transfer module that performs pixel-level distribution alignment between the training and test images. Based on the one-shot test image, the training image is reconstructed to have the same appearance as the test image. Meanwhile, we employ a multi-task learning strategy to prevent content distortion of the reconstructed images. To improve the robustness of the model to distribution differences, we design an edge-aware defect segmentation model and train the model using the reconstructed training images. The experimental results show that our method effectively improves the robustness of the model to distribution differences and achieves satisfying results in the task of rail surface defect segmentation.

A novel one-to-multiple unsupervised domain adaptation framework for abdominal organ segmentation.

Abdominal multi-organ segmentation in multi-sequence magnetic resonance images (MRI) is of great significance in many clinical scenarios, e.g., MRI-oriented pre-operative treatment planning. Labeling multiple organs on a single MR sequence is a time-consuming and labor-intensive task, let alone manual labeling on multiple MR sequences. Training a model by one sequence and generalizing it to other domains is one way to reduce the burden of manual annotation, but the existence of domain gap often leads to poor generalization performance of such methods. Image translation-based unsupervised domain adaptation (UDA) is a common way to address this domain gap issue. However, existing methods focus less on keeping anatomical consistency and are limited by one-to-one domain adaptation, leading to low efficiency for adapting a model to multiple target domains. This work proposes a unified framework called OMUDA for one-to-multiple unsupervised domain-adaptive segmentation, where disentanglement between content and style is used to efficiently translate a source domain image into multiple target domains. Moreover, generator refactoring and style constraint are conducted in OMUDA for better maintaining cross-modality structural consistency and reducing domain aliasing. The average Dice Similarity Coefficients (DSCs) of OMUDA for multiple sequences and organs on the in-house test set, the AMOS22 dataset and the CHAOS dataset are 85.51%, 82.66% and 91.38%, respectively, which are slightly lower than those of CycleGAN(85.66% and 83.40%) in the first two data sets and slightly higher than CycleGAN(91.36%) in the last dataset. But compared with CycleGAN, OMUDA reduces floating-point calculations by about 87 percent in the training phase and about 30 percent in the inference stage respectively. The quantitative results in both segmentation performance and training efficiency demonstrate the usability of OMUDA in some practical scenes, such as the initial phase of product development.

Semantically preserving adversarial unsupervised domain adaptation network for improving disease recognition from chest x-rays.

Supervised deep learning methods have been successfully applied in medical imaging. However, training deep learning systems often requires ample annotated data. Due to cost and time restrictions, not all collected medical images, e.g., chest x-rays (CXRs), can be labeled in practice. To classify these unlabeled images, a solution may involve adopting a model trained with sufficient labeled data in relevant domains (with both source and target being CXRs). However, domain shift may cause the trained model not able to generalize well on unlabeled target datasets. This work aims to develop a novel unsupervised domain adaptation (UDA) framework to improve recognition performance on unlabeled target data. We present a semantically preserving adversarial UDA network, i.e., SPA-UDA net, with the potential to bridge the domain gap, by reconstructing the images in the target domain via an adversarial encode-and-reconstruct translation architecture. To preserve the class-specific semantic information (i.e., with or without disease) of the original images when translating, a semantically consistent framework is embedded. This framework is designed to guarantee that fine-grained disease-related information on the original images can be safely transferred. Furthermore, the proposed SPA-UDA net does not require paired images from source and target domains when training, which reduces the cost of arranging data significantly and is ideal for UDA. We evaluate the proposed SPA-UDA net on two public CXR datasets for lung disease recognition. The experimental results show that the proposed framework achieves significant performance improvements compared to other state-of-the-art UDA methods.

Unsupervised Domain Adaptation for Medical Image Segmentation by Selective Entropy Constraints and Adaptive Semantic Alignment

Generalizing a deep learning model to new domains is crucial for computer-aided medical diagnosis systems. Most existing unsupervised domain adaptation methods have made significant progress in reducing the domain distribution gap through adversarial training. However, these methods may still produce overconfident but erroneous results on unseen target images. This paper proposes a new unsupervised domain adaptation framework for cross-modality medical image segmentation. Specifically, We first introduce two data augmentation approaches to generate two sets of semantics-preserving augmented images. Based on the model's predictive consistency on these two sets of augmented images, we identify reliable and unreliable pixels. We then perform a selective entropy constraint: we minimize the entropy of reliable pixels to increase their confidence while maximizing the entropy of unreliable pixels to reduce their confidence. Based on the identified reliable and unreliable pixels, we further propose an adaptive semantic alignment module which performs class-level distribution adaptation by minimizing the distance between same class prototypes between domains, where unreliable pixels are removed to derive more accurate prototypes. We have conducted extensive experiments on the cross-modality cardiac structure segmentation task. The experimental results show that the proposed method significantly outperforms the state-of-the-art comparison algorithms. Our code and data are available at https://github.com/fengweie/SE_ASA.

Unsupervised domain adaptation for crack detection

The reliable and fast detection of cracks is crucial for assessing the condition and maintaining civil infrastructure. However, due to diverse construction materials, imaging conditions, and environmental interference, there exists a domain shift between crack images collected from civil infrastructure. This shift results in significant performance drops of crack detection models trained on one dataset when applied to another, limiting their cross-dataset applicability. To address this issue, this paper proposes DACrack, an unsupervised domain adaptation framework for crack detection of civil infrastructure. The proposed method performs domain adaptation at the input, feature, and output levels using contrastive mechanisms, adversarial learning, and variational autoencoders. Extensive experiments demonstrate the effectiveness and robustness of the proposed method for cross-dataset crack detection. By mitigating the impact of domain shift, DACrack offers a more reliable and accurate solution for assessing the condition of civil infrastructure.

Dual adversarial models with cross-coordination consistency constraint for domain adaption in brain tumor segmentation.

The brain tumor segmentation task with different domains remains a major challenge because tumors of different grades and severities may show different distributions, limiting the ability of a single segmentation model to label such tumors. Semi-supervised models (e.g., mean teacher) are strong unsupervised domain-adaptation learners. However, one of the main drawbacks of using a mean teacher is that given a large number of iterations, the teacher model weights converge to those of the student model, and any biased and unstable predictions are carried over to the student. In this article, we proposed a novel unsupervised domain-adaptation framework for the brain tumor segmentation task, which uses dual student and adversarial training techniques to effectively tackle domain shift with MR images. In this study, the adversarial strategy and consistency constraint for each student can align the feature representation on the source and target domains. Furthermore, we introduced the cross-coordination constraint for the target domain data to constrain the models to produce more confident predictions. We validated our framework on the cross-subtype and cross-modality tasks in brain tumor segmentation and achieved better performance than the current unsupervised domain-adaptation and semi-supervised frameworks.

Unsupervised Adversarial Domain Adaptation Regression for Rate of Penetration Prediction

Summary The rate of penetration (ROP) refers to the speed at which a drill bit breaks through rock and deepens the drill hole. ROP is of great significance for drilling optimization and drilling cost savings. In real-world settings, the ROP data available for learning and training in a new oil field are scarce or even completely missing. In this paper, we propose a novel unsupervised multisource domain adaptation (MSDA) regression method for ROP that considers transferring the knowledge learned from a well-labelled source domain to the target domain with few labeled ROP data. First, we build a multisource unsupervised domain adaptation framework based on adversarial learning (WD-MUDA) which uses a weighted combination of multiple source domains to realize the fine-grained alignment of different data distributions. Specifically, we define a new similarity metric for different domains based on the Wasserstein distance. Furthermore, considering the uneven distribution of real drilling data samples, a novel regression loss is introduced to minimize the gradient discrepancy between multisource and target samples and improve the prediction accuracy of target samples. Extensive experiments on real drilling data sets reveal that the proposed method is effective and outperforms the state-of-the-art domain adaptation methods for ROP prediction.

A medical unsupervised domain adaptation framework based on Fourier transform image translation and multi-model ensemble self-training strategy.

Well-established segmentation models will suffer performance degradation when deployed on data with heterogeneous features, especially in the field of medical image analysis. Although researchers have proposed many approaches to address this problem in recent years, most of them are feature-adaptation-based adversarial networks, the problems such as training instability often arise in adversarial training. To ameliorate this challenge and improve the robustness of processing data with different distributions, we propose a novel unsupervised domain adaptation framework for cross-domain medical image segmentation. In our proposed approach, Fourier transform guided images translation and multi-model ensemble self-training are integrated into a unified framework. First, after Fourier transform, the amplitude spectrum of source image is replaced with that of target image, and reconstructed by the inverse Fourier transform. Second, we augment target dataset with the synthetic cross-domain images, performing supervised learning using the original source set labels while implementing regularization by entropy minimization on predictions of unlabeled target data. We employ several segmentation networks with different hyperparameters simultaneously, pseudo-labels are generated by averaging their outputs and comparing to confidence threshold, and gradually optimize the quality of pseudo-labels through multiple rounds self-training. We employed our framework to two liver CT datasets for bidirectional adaptation experiments. In both experiments, compared to the segmentation network without domain alignment, dice similarity coefficient (DSC) increased by nearly 34% and average symmetric surface distance (ASSD) decreased by about 10. The DSC values were also improved by 10.8% and 6.7%, respectively, compared to the existing model. We propose a Fourier transform-based UDA framework, the experimental results and comparisons demonstrate that the proposed method can effectively diminish the performance degradation caused by domain shift and performs best on the cross-domain segmentation tasks. Our proposed multi-model ensemble training strategy can also improve the robustness of the segmentation system.

Hypergraph and cross-attention-based unsupervised domain adaptation framework for cross-domain myocardial infarction localization

Solving individual differences between subjects is critical for the promotion of electrocardiogram (ECG) classification algorithms in the intelligent health monitoring industry. Popular inter-subject-based solutions usually require the manual labeling of heartbeats and frequent updating of the model for new subjects. To track these problems, we propose a hypergraph and cross-attention-based unsupervised domain adaptation (HGCA-UDA) framework for the myocardial infarction localization. Specifically, we first build a hypergraph-based dual-channel network, that can simultaneously learn specific feature representations from an ECG lead and disease category levels for samples from different domains. We then design a cross-attention module to align cross-domain locally similar samples. Subsequently, a domain alignment strategy based on the Wasserstein distance is proposed to align the global edge feature distribution. Finally, a pseudo-label generation scheme is proposed to further align fine-grained category information. We conduct extensive experiments on two public benchmark datasets (the Physikalisch-Technische Bundesanstalt (PTB) and PTB_XL database), and the results show that the proposed HGCR-UDA (with unlabeled patients) achieves comparable results compared with state-of-the-art inter-patient-based methods (with labeled patients) and has excellent applications prospects in the field of intelligent health monitoring.

A novel unsupervised domain adaptation framework based on graph convolutional network and multi-level feature alignment for inter-subject ECG classification

Electrocardiogram (ECG) is an effective non-invasive tool that can detect arrhythmias. Recently, deep learning (DL) has been widely used in ECG classification algorithms. However, differences between subjects lead to data shifts, hindering the further extension of DL algorithms. To solve this problem, we propose a novel multi-level unsupervised domain adaptation framework (MLUDAF) to diagnose arrhythmias. During feature extraction, we use the atrous spatial pyramid pooling residual (ASPP-R) module to extract spatio-temporal features of the samples. Then the graph convolutional network (GCN) module is used to extract the data structure features. During domain adaptation, we design three alignment mechanisms: domain alignment, semantic alignment, and structure alignment. The three alignment strategies are integrated into a unified deep network to guide the feature extractor to extract domain sharing and distinguishable semantic representations, which can reduce the differences between the source and target domains. Experimental results based on the MIT-BIH database show that the proposed method achieves an overall accuracy of 96.8% for arrhythmia detection. Compared to other methods, the proposed method achieves competitive performance. Cross-domain experiments between databases also demonstrate its strong generalizability. Therefore, the proposed method is promising for application in medical diagnosis systems.

Source free unsupervised domain adaptation for electro-mechanical actuator fault diagnosis

A common necessity for prior unsupervised domain adaptation methods that can improve the domain adaptation in unlabeled target domain dataset is access to source domain dataset and target domain dataset simultaneously. However, data privacy makes it not always possible to access source domain dataset and target domain dataset in actual industrial equipment simultaneously, especially for aviation component like Electro-Mechanical Actuator (EMA) whose dataset are often not shareable due to the data copyright and confidentiality. To address this problem, this paper proposes a source free unsupervised domain adaptation framework for EMA fault diagnosis. The proposed framework is a combination of feature network and classifier. Firstly, source domain datasets are only applied to train a source model. Secondly, the well-trained source model is transferred to target domain and classifier is frozen based on source domain hypothesis. Thirdly, nearest centroid filtering is introduced to filter the reliable pseudo labels for unlabeled target domain dataset, and finally, supervised learning and pseudo label clustering are applied to fine-tune the transferred model. In comparison with several traditional unsupervised domain adaptation methods, case studies based on low- and high-frequency monitoring signals on EMA indicate the effectiveness of the proposed method.

Contrastive Domain Adaptation-Based Sparse SAR Target Classification under Few-Shot Cases

Due to the imaging mechanism of synthetic aperture radar (SAR), it is difficult and costly to acquire abundant labeled SAR images. Moreover, a typical matched filtering (MF) based image faces the problems of serious noise, sidelobes, and clutters, which will bring down the accuracy of SAR target classification. Different from the MF-based result, a sparse image shows better quality with less noise and higher image signal-to-noise ratio (SNR). Therefore, theoretically using it for target classification will achieve better performance. In this paper, a novel contrastive domain adaptation (CDA) based sparse SAR target classification method is proposed to solve the problem of insufficient samples. In the proposed method, we firstly construct a sparse SAR image dataset by using the complex image based iterative soft thresholding (BiIST) algorithm. Then, the simulated and real SAR datasets are simultaneously sent into an unsupervised domain adaptation framework to reduce the distribution difference and obtain the reconstructed simulated SAR images for subsequent target classification. Finally, the reconstructed simulated images are manually labeled and fed into a shallow convolutional neural network (CNN) for target classification along with a small number of real sparse SAR images. Since the current definition of the number of small samples is still vague and inconsistent, this paper defines few-shot as less than 20 per class. Experimental results based on MSTAR under standard operating conditions (SOC) and extended operating conditions (EOC) show that the reconstructed simulated SAR dataset makes up for the insufficient information from limited real data. Compared with other typical deep learning methods based on limited samples, our method is able to achieve higher accuracy especially under the conditions of few shots.

A Fine-Grained Unsupervised Domain Adaptation Framework for Semantic Segmentation of Remote Sensing Images

A Fine-Grained Unsupervised Domain Adaptation Framework for Semantic Segmentation of Remote Sensing Images

A Novel 3D Unsupervised Domain Adaptation Framework for Cross-Modality Medical Image Segmentation.

We consider the problem of volumetric (3D) unsupervised domain adaptation (UDA) in cross-modality medical image segmentation, aiming to perform segmentation on the unannotated target domain (e.g. MRI) with the help of labeled source domain (e.g. CT). Previous UDA methods in medical image analysis usually suffer from two challenges: 1) they focus on processing and analyzing data at 2D level only, thus missing semantic information from the depth level; 2) one-to-one mapping is adopted during the style-transfer process, leading to insufficient alignment in the target domain. Different from the existing methods, in our work, we conduct a first of its kind investigation on multi-style image translation for complete image alignment to alleviate the domain shift problem, and also introduce 3D segmentation in domain adaptation tasks to maintain semantic consistency at the depth level. In particular, we develop an unsupervised domain adaptation framework incorporating a novel quartet self-attention module to efficiently enhance relationships between widely separated features in spatial regions on a higher dimension, leading to a substantial improvement in segmentation accuracy in the unlabeled target domain. In two challenging cross-modality tasks, specifically brain structures and multi-organ abdominal segmentation, our model is shown to outperform current state-of-the-art methods by a significant margin, demonstrating its potential as a benchmark resource for the biomedical and health informatics research community.

Unsupervised Domain Adaptation with Adversarial Self-Training for Crop Classification Using Remote Sensing Images

Crop type mapping is regarded as an essential part of effective agricultural management. Automated crop type mapping using remote sensing images is preferred for the consistent monitoring of crop types. However, the main obstacle to generating annual crop type maps is the collection of sufficient training data for supervised classification. Classification based on unsupervised domain adaptation, which uses prior information from the source domain for target domain classification, can solve the impractical problem of collecting sufficient training data. This study presents self-training with domain adversarial network (STDAN), a novel unsupervised domain adaptation framework for crop type classification. The core purpose of STDAN is to combine adversarial training to alleviate spectral discrepancy problems with self-training to automatically generate new training data in the target domain using an existing thematic map or ground truth data. STDAN consists of three analysis stages: (1) initial classification using domain adversarial neural networks; (2) the self-training-based updating of training candidates using constraints specific to crop classification; and (3) the refinement of training candidates using iterative classification and final classification. The potential of STDAN was evaluated by conducting six experiments reflecting various domain discrepancy conditions in unmanned aerial vehicle images acquired at different regions and times. In most cases, the classification performance of STDAN was found to be compatible with the classification using training data collected from the target domain. In particular, the superiority of STDAN was shown to be prominent when the domain discrepancy was substantial. Based on these results, STDAN can be effectively applied to automated cross-domain crop type mapping without analyst intervention when prior information is available in the target domain.

HierMUD: Hierarchical multi-task unsupervised domain adaptation between bridges for drive-by damage diagnosis

Monitoring bridges through vibration responses of drive-by vehicles enables efficient and low-cost bridge maintenance by allowing each vehicle to inspect multiple bridges and eliminating the needs for installing and maintaining sensors on every bridge. However, many existing drive-by monitoring approaches are based on supervised learning models that require massive labeled data from every bridge. It is expensive and time-consuming, if not impossible, to obtain these labeled data. Furthermore, directly applying a supervised learning model trained on one bridge to new bridges would result in low accuracy due to the shift between different bridges’ data distributions. Moreover, when we have multiple tasks (e.g., damage detection, localization, and quantification), the distribution shifts become more challenging than having only one task because different tasks have distinct distribution shifts and varying task difficulties. To this end, we introduce HierMUD, the first Hierarchical Multi-task Unsupervised Domain adaptation framework that transfers the damage diagnosis model learned from one bridge to a new bridge without requiring any labels from the new bridge. Specifically, our framework learns a hierarchical neural network model in an adversarial way to extract features that are informative to multiple tasks and invariant across multiple bridges. To match distributions over multiple tasks, we design a new loss function based on a newly derived generalization risk bound to adaptively assign higher weights to tasks with more shifted distributions. To learn multiple tasks with varying task difficulties, we split them into easy-to-learn and hard-to-learn tasks based on their distributions. Then, we formulate a feature hierarchy to utilize more learning resources to improve the hard-to-learn tasks’ performance. We evaluate our framework with experimental data from 2 bridges and 3 vehicles. We achieve up to 2X better performance than baseline methods, including average accuracy of 95% for damage detection, 93% for localization, and 0.38 lbs mean absolute error for quantification.

ACDC: Online unsupervised cross-domain adaptation

We consider the problem of online unsupervised cross-domain adaptation, where two independent but related data streams with different feature spaces – a fully labeled source stream and an unlabeled target stream – are learned together. Unique characteristics and challenges such as covariate shift, asynchronous concept drifts, and contrasting data throughput arise. We propose ACDC, an adversarial unsupervised domain adaptation framework that handles multiple data streams with a complete self-evolving neural network structure that reacts to these defiances. ACDC encapsulates three modules into a single model: A denoising autoencoder that extracts features, an adversarial module that performs domain conversion, and an estimator that learns the source stream and predicts the target stream. ACDC is a flexible and expandable framework with little hyper-parameter tunability. Our experimental results under the prequential test-then-train protocol indicate an improvement in target accuracy over the baseline methods, achieving more than a 10% increase in some cases.

See Eye to Eye: A Lidar-Agnostic 3D Detection Framework for Unsupervised Multi-Target Domain Adaptation

Sampling discrepancies between different manufacturers and models of lidar sensors result in inconsistent representations of objects. This leads to performance degradation when 3D detectors trained for one lidar are tested on other types of lidars. Remarkable progress in lidar manufacturing has brought about advances in mechanical, solid-state, and recently, adjustable scan pattern lidars. For the latter, existing works often require fine-tuning the model each time scan patterns are adjusted, which is infeasible. We explicitly deal with the sampling discrepancy by proposing a novel unsupervised multi-target domain adaptation framework, SEE, for transferring the performance of state-of-the-art 3D detectors across both fixed and flexible scan pattern lidars without requiring fine-tuning of models by end-users. Our approach interpolates the underlying geometry and normalises the scan pattern of objects from different lidars before passing them to the detection network. We demonstrate the effectiveness of SEE on public datasets, achieving state-of-the-art results, and additionally provide quantitative results on a novel high-resolution lidar to prove the industry applications of our framework.

AdaSL: An Unsupervised Domain Adaptation framework for Arabic multi-dialectal Sequence Labeling

Dialectal Arabic (DA) refers to varieties of everyday spoken languages in the Arab world. These dialects differ according to the country and region of the speaker, and their textual content is constantly growing with the rise of social media networks and web blogs. Although research on Natural Language Processing (NLP) on standard Arabic, namely Modern Standard Arabic (MSA), has witnessed remarkable progress, research efforts on DA are rather limited. This is due to numerous challenges, such as the scarcity of labeled data as well as the nature and structure of DA. While some recent works have reached decent results on several DA sentence classification tasks, other complex tasks, such as sequence labeling, still suffer from weak performances when it comes to DA varieties with either a limited amount of labeled data or unlabeled data only. Besides, it has been shown that zero-shot transfer learning from models trained on MSA does not perform well on DA. In this paper, we introduce AdaSL, a new unsupervised domain adaptation framework for Arabic multi-dialectal sequence labeling, leveraging unlabeled DA data, labeled MSA data, and existing multilingual and Arabic Pre-trained Language Models (PLMs). The proposed framework relies on four key components: (1) domain adaptive fine-tuning of multilingual/MSA language models on unlabeled DA data, (2) sub-word embedding pooling, (3) iterative self-training on unlabeled DA data, and (4) iterative DA and MSA distribution alignment. We evaluate our framework on multi-dialectal Named Entity Recognition (NER) and Part-of-Speech (POS) tagging tasks.The overall results show that the zero-shot transfer learning, using our proposed framework, boosts the performance of the multilingual PLMs by 40.87% in macro-F1 score for the NER task, while it boosts the accuracy by 6.95% for the POS tagging task. For the Arabic PLMs, our proposed framework increases performance by 16.18% macro-F1 for the NER task and 2.22% accuracy for the POS tagging task, and thus, achieving new state-of-the-art zero-shot transfer learning performance for Arabic multi-dialectal sequence labeling.

Cycle and Self-Supervised Consistency Training for Adapting Semantic Segmentation of Aerial Images

Semantic segmentation is a critical problem for many remote sensing (RS) image applications. Benefiting from large-scale pixel-level labeled data and the continuous evolution of deep neural network architectures, the performance of semantic segmentation approaches has been constantly improved. However, deploying a well-trained model on unseen and diverse testing environments remains a major challenge: a large gap between data distributions in train and test domains results in severe performance loss, while manual dense labeling is costly and not scalable. To this end, we proposed an unsupervised domain adaptation framework for RS image semantic segmentation that is both practical and effective. The framework is supported by the consistency principle, including the cycle consistency in the input space and self-supervised consistency in the training stage. Specifically, we introduce cycle-consistent generative adversarial networks to reduce the discrepancy between source and target distributions by translating one into the other. The translated source data then drive a pipeline of supervised semantic segmentation model training. We enforce consistency of model predictions across target image transformations in order to provide self-supervision for the unlabeled target data. Experiments and extensive ablation studies demonstrate the effectiveness of the proposed approach on two challenging benchmarks, on which we achieve up to 9.95% and 7.53% improvements in accuracy over the state-of-the-art methods, respectively.