Unsupervised Domain Adaptation Framework Research Articles

Physics-informed unsupervised domain adaptation framework for cross-machine bearing fault diagnosis

Varying components and operating conditions in industrial machines lead to different distribution characteristics and fault states of monitoring data for critical rotating machinery. Adequate and accurate labelling of monitoring data is challenging to achieve. Existing data-driven unsupervised domain adaptation methods exhibit poor interpretability and low confidence in the target domain’s pseudo-labelling. Therefore, a physics-informed unsupervised domain adaptation framework is proposed. Data and physical information are combined to enhance the reliability and interpretability of the model while ensuring its generalization capability. First, physical information is extracted by merging the knowledge of the mechanical component fault characteristics with indicators like the signal spectrum’s energy value. Subsequently, the physical label and loss are created. The physical loss is then embedded in the loss function of the source domain model. Next, by aligning the physical labels and the clustering results of the spectral clustering, the target domain samples’ hard pseudo-labels are obtained. The probability values of the sample belonging to each category as determined by the Wasserstein criterion comprise the soft pseudo-label of the sample. Furthermore, the confidence of hard pseudo-labels is continually increased by iteratively updating the probability values of soft pseudo-labels using the multi-label as a guide and the inverse improved sigmoid function as a gradient. Finally, unsupervised cross-machine diagnostic models with high robustness can be obtained. Three datasets are employed in the experimental section, and five sub-datasets are created and crossed for cross-machine experiments to confirm the suggested approach’s efficacy and superiority.

LUIE: Learnable physical model-guided underwater image enhancement with bi-directional unsupervised domain adaptation

Recently, learning-based underwater enhancement (UIE) methods have made considerable progress, significantly benefiting downstream tasks such as underwater semantic segmentation and underwater depth estimation. Most existing unsupervised UIE methods utilize the atmospheric image formation model to decompose underwater images into background color, transmission map, and scene radiance. However, they rely on simplified physical models for estimating the transmission map, over-simplifying its complex formation, which results in imprecise modeling of underwater scattering effects. Additionally, supervised UIE methods heavily depend on synthetic data or ground truth, leading to limited generalization capabilities due to the substantial domain gap presented in different underwater scenarios. To tackle these challenges, we propose a Learnable physical model-guided unsupervised domain adaptation framework for Underwater Image Enhancement, dubbed LUIE. LUIE learns to predict background light, depth, and scene radiance from an underwater image. We incorporate a learnable network to estimate the transmission map based on the predicted depth map. To minimize the inter-domain gap between synthetic and real underwater images, we introduce a bi-directional domain adaptation method that alternates the background light from each domain. Experimental results demonstrate the effectiveness of our proposed method compared to existing approaches, and high-level experiment results validate that our enhanced underwater results. Experiments in real-world settings on underwater ROVs platform with NVIDIA Jetson AGX Xavier further confirm the effectiveness and efficiency of our work.

EEG-Based Motor Imagery Recognition Framework via Multisubject Dynamic Transfer and Iterative Self-Training.

A robust decoding model that can efficiently deal with the subject and period variation is urgently needed to apply the brain-computer interface (BCI) system. The performance of most electroencephalogram (EEG) decoding models depends on the characteristics of specific subjects and periods, which require calibration and training with annotated data prior to application. However, this situation will become unacceptable as it would be difficult for subjects to collect data for an extended period, especially in the rehabilitation process of disability based on motor imagery (MI). To address this issue, we propose an unsupervised domain adaptation framework called iterative self-training multisubject domain adaptation (ISMDA) that focuses on the offline MI task. First, the feature extractor is purposefully designed to map the EEG to a latent space of discriminative representations. Second, the attention module based on dynamic transfer matches the source domain and target domain samples with a higher coincidence degree in latent space. Then, an independent classifier oriented to the target domain is employed in the first stage of the iterative training process to cluster the samples of the target domain through similarity. Finally, a pseudolabel algorithm based on certainty and confidence is employed in the second stage of the iterative training process to adequately calibrate the error between prediction and empirical probabilities. To evaluate the effectiveness of the model, extensive testing has been performed on three publicly available MI datasets, the BCI IV IIa, the High gamma dataset, and Kwon et al. datasets. The proposed method achieved 69.51%, 82.38%, and 90.98% cross-subject classification accuracy on the three datasets, which outperforms the current state-of-the-art offline algorithms. Meanwhile, all results demonstrated that the proposed method could address the main challenges of the offline MI paradigm.

Unsupervised domain adaptation with hard-sample dividing and processing strategy

In Unsupervised Domain Adaptation(UDA), the hard samples cause serious negative transfer to model performance. However, the existing UDA methods don’t fully pay attention to the fine-grained processing of hard samples in target and source domains. To address these problems, the UDA framework with Hard-sample Dividing and Processing Strategy (HDPS) is proposed in this paper. In HDPS, we define sample division criteria in target domain and source domain respectively and divide the samples into easy samples, hard samples with low confidence and hard samples with high confidence. We further design systematic processing strategy of the three types of samples. Specifically, we firstly define geometric metrics based on the class prototype to divide the hard samples and eliminate them in source domain. We further design two weight allocation strategies based on sample classification entropy in target domain. Finally, we build a teacher-student guidance mechanism to learn the discriminative features of hard samples. The HDPS framework can be easily loaded to other methods as a plug-in, can promote the diversity feature learning of samples, and significantly improve the discriminant performance of the model. Extensive experiments on six benchmark datasets verify that HDPS is effective and superior to most existing UDA methods.

Uncertainty inspired domain adaptation network for rail surface defect segmentation

Rail surface defect detection is an essential part of railroad maintenance to prevent safety accidents. Recently, deep learning-based defect detection algorithms have shown impressive detection performance. However, the performance improvement from deep neural networks is based on a strong assumption that the training images have the same data distribution as the test images. This requirement is unrealistic in practical railroad surface defect detection. The appearance of rails varies significantly in different railroad sections due to changes in working conditions and maintenance status. In this paper, we propose a novel uncertainty-inspired unsupervised domain adaptation framework (UIUDA) to improve the generalization of deep learning models on data with distribution differences. Specifically, domain adversarial learning is used to align the feature distributions of training and test data. To mitigate the negative transfer caused by global alignment, we design a locally aligned domain adaptation strategy. In addition, self-training is introduced to refine the decision boundaries of the model in the test data. In this process, we propose an uncertainty-inspired evaluation method to remove noisy pseudo labels. We conducted domain adaptation experiments with RSDD and NEU-RSDD as the source domain and FRSD dataset as the target domain. The results show that UIUDA can effectively improve the generalization ability of the model, and the mIoU metric of defect segmentation is 80.17%.

Structure enhanced prototypical alignment for unsupervised cross-domain node classification

Graph Neural Networks (GNNs) have demonstrated remarkable success in graph node classification task. However, their performance heavily relies on the availability of high-quality labeled data, which can be time-consuming and labor-intensive to acquire for graph-structured data. Therefore, the task of transferring knowledge from a label-rich graph (source domain) to a completely unlabeled graph (target domain) becomes crucial. In this paper, we propose a novel unsupervised graph domain adaptation framework called Structure Enhanced Prototypical Alignment (SEPA), which aims to learn domain-invariant representations on non-IID (non-independent and identically distributed) data. Specifically, SEPA captures class-wise semantics by constructing a prototype-based graph and introduces an explicit domain discrepancy metric to align the source and target domains. The proposed SEPA framework is optimized in an end-to-end manner, which could be incorporated into various GNN architectures. Experimental results on several real-world datasets demonstrate that our proposed framework outperforms recent state-of-the-art baselines with different gains.

Loose to compact feature alignment for domain adaptive object detection

Recently, great achievements have been made for deep learning based object detection methods. But their performance drops significantly when domain shifts occur. To address this problem, in this work we propose a loose to compact feature alignment method under an unsupervised domain adaptation framework. The entire feature alignment is performed in a divide and conquer manner, so as to distribute the alignment difficulties at two steps. At the first step, we loosen the goal at both image and instance levels. At the image level, a new Mask Guided Foreground Alignment (MGFA) module is proposed to make the alignment focus more on easier foreground regions, leaving the more diverse and more difficult background regions to the second step; at the instance level, we propose a Class-Wise Instance Alignment (CWIA) module with separated domain classifiers for different categories so as to ease the alignment. At the second step, the alignment is performed per pixel and per instance, achieving a more compact and better aligned feature space. We conduct experiments on three different adaptation scenarios, where we achieve comparable results, demonstrating the effectiveness of our proposed method.

Unsupervised Domain Adaptation for Medical Image Segmentation Using Transformer With Meta Attention.

Image segmentation is essential to medical image analysis as it provides the labeled regions of interest for the subsequent diagnosis and treatment. However, fully-supervised segmentation methods require high-quality annotations produced by experts, which is laborious and expensive. In addition, when performing segmentation on another unlabeled image modality, the segmentation performance will be adversely affected due to the domain shift. Unsupervised domain adaptation (UDA) is an effective way to tackle these problems, but the performance of the existing methods is still desired to improve. Also, despite the effectiveness of recent Transformer-based methods in medical image segmentation, the adaptability of Transformers is rarely investigated. In this paper, we present a novel UDA framework using a Transformer for building a cross-modality segmentation method with the advantages of learning long-range dependencies and transferring attentive information. To fully utilize the attention learned by the Transformer in UDA, we propose Meta Attention (MA) and use it to perform a fully attention-based alignment scheme, which can learn the hierarchical consistencies of attention and transfer more discriminative information between two modalities. We have conducted extensive experiments on cross-modality segmentation using three datasets, including a whole heart segmentation dataset (MMWHS), an abdominal organ segmentation dataset, and a brain tumor segmentation dataset. The promising results show that our method can significantly improve performance compared with the state-of-the-art UDA methods.

M-Net: a novel unsupervised domain adaptation framework based on multi-kernel maximum mean discrepancy for fault diagnosis of rotating machinery

AbstractThe intelligent fault diagnosis model has made a significant development, whose high-precision results rely on a large amount of labeled data. However, in the actual industrial environment, it is very difficult to obtain a large amount of labeled data. It will make it difficult for the fault diagnosis model to converge with limited labeled industrial data. To address this paradox, we propose a novel unsupervised domain adaptation framework (M-Net) for fault diagnosis of rotating machinery, which only requires unlabeled industrial data. The M-Net will be pretrained using the labeled data, which can be accessed through the labs. In this stage, we propose a multi-scale feature extractor that can extract and fuse multi-scale features. This operation will generalize the features further. Then, we will align the distribution of the labeled data and unlabeled industrial data using the generator model based on multi-kernel maximum mean discrepancy. This will reduce the distribution distance between the labeled data and the unlabeled industrial data. For now, the unsupervised domain adaptation problem has shifted to a semi-supervised domain adaptation problem. The results, obtained through experimental comparison, demonstrate that the M-Net can achieve an accuracy of over 99.99% with labeled data and a maximum transfer accuracy of over 99% with unlabeled industrial data.

StAC-DA: Structure aware cross-modality domain adaptation framework with image and feature-level adaptation for medical image segmentation.

Convolutional neural networks (CNNs) have achieved state-of-the-art results in various medical image segmentation tasks. However, CNNs often assume that the source and target dataset follow the same probability distribution and when this assumption is not satisfied their performance degrades significantly. This poses a limitation in medical image analysis, where including information from different imaging modalities can bring large clinical benefits. In this work, we present an unsupervised Structure Aware Cross-modality Domain Adaptation (StAC-DA) framework for medical image segmentation. StAC-DA implements an image- and feature-level adaptation in a sequential two-step approach. The first step performs an image-level alignment, where images from the source domain are translated to the target domain in pixel space by implementing a CycleGAN-based model. The latter model includes a structure-aware network that preserves the shape of the anatomical structure during translation. The second step consists of a feature-level alignment. A U-Net network with deep supervision is trained with the transformed source domain images and target domain images in an adversarial manner to produce probable segmentations for the target domain. The framework is evaluated on bidirectional cardiac substructure segmentation. StAC-DA outperforms leading unsupervised domain adaptation approaches, being ranked first in the segmentation of the ascending aorta when adapting from Magnetic Resonance Imaging (MRI) to Computed Tomography (CT) domain and from CT to MRI domain. The presented framework overcomes the limitations posed by differing distributions in training and testing datasets. Moreover, the experimental results highlight its potential to improve the accuracy of medical image segmentation across diverse imaging modalities.

Toward Accurate Cardiac MRI Segmentation With Variational Autoencoder-Based Unsupervised Domain Adaptation.

Accurate myocardial segmentation is crucial in the diagnosis and treatment of myocardial infarction (MI), especially in Late Gadolinium Enhancement (LGE) cardiac magnetic resonance (CMR) images, where the infarcted myocardium exhibits a greater brightness. However, segmentation annotations for LGE images are usually not available. Although knowledge gained from CMR images of other modalities with ample annotations, such as balanced-Steady State Free Precession (bSSFP), can be transferred to the LGE images, the difference in image distribution between the two modalities (i.e., domain shift) usually results in a significant degradation in model performance. To alleviate this, an end-to-end Variational autoencoder based feature Alignment Module Combining Explicit and Implicit features (VAMCEI) is proposed. We first re-derive the Kullback-Leibler (KL) divergence between the posterior distributions of the two domains as a measure of the global distribution distance. Second, we calculate the prototype contrastive loss between the two domains, bringing closer the prototypes of the same category across domains and pushing away the prototypes of different categories within or across domains. Finally, a domain discriminator is added to the output space, which indirectly aligns the feature distribution and forces the extracted features to be more favorable for segmentation. In addition, by combining CycleGAN and VAMCEI, we propose a more refined multi-stage unsupervised domain adaptation (UDA) framework for myocardial structure segmentation. We conduct extensive experiments on the MSCMRSeg 2019, MyoPS 2020 and MM-WHS 2017 datasets. The experimental results demonstrate that our framework achieves superior performances than state-of-the-art methods.

BTMuda: A Bi-Level Multi-Source Unsupervised Domain Adaptation Framework for Breast Cancer Diagnosis

BTMuda: A Bi-Level Multi-Source Unsupervised Domain Adaptation Framework for Breast Cancer Diagnosis

MarsScapes and UDAFormer: A Panorama Dataset and a Transformer-Based Unsupervised Domain Adaptation Framework for Martian Terrain Segmentation

MarsScapes and UDAFormer: A Panorama Dataset and a Transformer-Based Unsupervised Domain Adaptation Framework for Martian Terrain Segmentation

Unsupervised Domain Adaptation for RF-Based Gesture Recognition

Human gesture recognition with Radio Frequency (RF) signals has attained acclaim due to the omnipresence, privacy protection, and broad coverage nature of RF signals. These gesture recognition systems rely on neural networks trained with a large number of labeled data. However, the recognition model trained with data under certain conditions would suffer from significant performance degradation when applied in practical deployment, which limits the application of gesture recognition systems. In this paper, we propose an unsupervised domain adaptation framework for RF-based gesture recognition aiming to enhance the performance of the recognition model in new conditions by making effective use of the unlabeled data from new conditions. We first propose pseudo-labeling and consistency regularization to utilize unlabeled data for model training and eliminate the feature discrepancies in different domains. Then we propose a confidence constraint loss to enhance the effectiveness of pseudo-labeling, and design two corresponding data augmentation methods based on the characteristic of the RF signals to strengthen the performance of the consistency regularization, which can make the framework more effective and robust. Furthermore, we propose a cross-match loss to integrate the pseudo-labeling and consistency regularization, which makes the whole framework simple yet effective. Extensive experiments demonstrate that the proposed framework could achieve 4.35% and 2.25% accuracy improvement comparing with the state-of-the-art methods on public WiFi dataset and millimeter wave (mmWave) radar dataset, respectively.

A Novel Unsupervised Clustering and Domain Adaptation Framework for Rotating Machinery Fault Diagnosis

Deep learning-based fault diagnosis methods require a large number of labeled datasets. However, considering the changing operating conditions, it is impractical to obtain labeled datasets for all cases. Therefore, this study proposes a new unsupervised clustering and domain adaptation framework to circumvent data deficiency and domain issues. The proposed framework comprises two steps: unsupervised clustering and domain adaptation. In the unsupervised clustering, an expectation-maximization adversarial autoencoder, which combines an expectation-maximization algorithm with an adversarial autoencoder, is used for feature extraction and subspace mapping. Subsequently, the mapped features are clustered using a Gaussian mixture model. In the domain adaptation, a domain synchronization that is based on the symmetric Kullback-Leibler divergence metric is used to infer the relationship between the source and target domain clusters. The experiments on two rolling-element-bearing datasets validate the effectiveness of our method. Specifically, our method performs domain adaptation without retraining, which is promising for real industrial applications.

ATFA: Adversarial Time–Frequency Attention network for sensor-based multimodal human activity recognition

Human activity recognition (HAR) is a key component of mobile and ubiquitous computing. With the increasing number of sensors embedded in devices such as smartphones and smartwatches, models are enhanced to recognize more complex human behaviors. However, this also leads to increased data heterogeneity, caused by diverse user backgrounds, health conditions, and sensing environments, which makes it difficult to align the multimodal data distributions across individuals. This paper proposes a novel unsupervised domain adaptation framework via Adversarial Time-Frequency Attention (ATFA) to efficiently adapt models to new users. In particular, the proposed attention-based modality fusion module captures and fuses important modalities based on their context to reduce redundant information. Additionally, the network explores frequency domain features to improve performance in recognizing human activities. Extensive experiments are conducted on three publicly available HAR datasets to demonstrate the superiority of our proposed method compared to state-of-the-art baselines.

Shape-Consistent One-Shot Unsupervised Domain Adaptation for Rail Surface Defect Segmentation

Deep neural networks have greatly improved the performance of rail surface defect segmentation when the test samples have the same distribution as the training samples. However, in practical inspection scenarios, the rail surface exhibits variations in appearance due to different service time and natural conditions. Conventional deep learning models show limited generalization in scenes with distribution differences. To address this problem, we propose a novel one-shot unsupervised domain adaptation framework. Specifically, we introduce a shape consistent style transfer module that performs pixel-level distribution alignment between the training and test images. Based on the one-shot test image, the training image is reconstructed to have the same appearance as the test image. Meanwhile, we employ a multi-task learning strategy to prevent content distortion of the reconstructed images. To improve the robustness of the model to distribution differences, we design an edge-aware defect segmentation model and train the model using the reconstructed training images. The experimental results show that our method effectively improves the robustness of the model to distribution differences and achieves satisfying results in the task of rail surface defect segmentation.

A novel one-to-multiple unsupervised domain adaptation framework for abdominal organ segmentation.

Abdominal multi-organ segmentation in multi-sequence magnetic resonance images (MRI) is of great significance in many clinical scenarios, e.g., MRI-oriented pre-operative treatment planning. Labeling multiple organs on a single MR sequence is a time-consuming and labor-intensive task, let alone manual labeling on multiple MR sequences. Training a model by one sequence and generalizing it to other domains is one way to reduce the burden of manual annotation, but the existence of domain gap often leads to poor generalization performance of such methods. Image translation-based unsupervised domain adaptation (UDA) is a common way to address this domain gap issue. However, existing methods focus less on keeping anatomical consistency and are limited by one-to-one domain adaptation, leading to low efficiency for adapting a model to multiple target domains. This work proposes a unified framework called OMUDA for one-to-multiple unsupervised domain-adaptive segmentation, where disentanglement between content and style is used to efficiently translate a source domain image into multiple target domains. Moreover, generator refactoring and style constraint are conducted in OMUDA for better maintaining cross-modality structural consistency and reducing domain aliasing. The average Dice Similarity Coefficients (DSCs) of OMUDA for multiple sequences and organs on the in-house test set, the AMOS22 dataset and the CHAOS dataset are 85.51%, 82.66% and 91.38%, respectively, which are slightly lower than those of CycleGAN(85.66% and 83.40%) in the first two data sets and slightly higher than CycleGAN(91.36%) in the last dataset. But compared with CycleGAN, OMUDA reduces floating-point calculations by about 87 percent in the training phase and about 30 percent in the inference stage respectively. The quantitative results in both segmentation performance and training efficiency demonstrate the usability of OMUDA in some practical scenes, such as the initial phase of product development.

Semantically preserving adversarial unsupervised domain adaptation network for improving disease recognition from chest x-rays.

Supervised deep learning methods have been successfully applied in medical imaging. However, training deep learning systems often requires ample annotated data. Due to cost and time restrictions, not all collected medical images, e.g., chest x-rays (CXRs), can be labeled in practice. To classify these unlabeled images, a solution may involve adopting a model trained with sufficient labeled data in relevant domains (with both source and target being CXRs). However, domain shift may cause the trained model not able to generalize well on unlabeled target datasets. This work aims to develop a novel unsupervised domain adaptation (UDA) framework to improve recognition performance on unlabeled target data. We present a semantically preserving adversarial UDA network, i.e., SPA-UDA net, with the potential to bridge the domain gap, by reconstructing the images in the target domain via an adversarial encode-and-reconstruct translation architecture. To preserve the class-specific semantic information (i.e., with or without disease) of the original images when translating, a semantically consistent framework is embedded. This framework is designed to guarantee that fine-grained disease-related information on the original images can be safely transferred. Furthermore, the proposed SPA-UDA net does not require paired images from source and target domains when training, which reduces the cost of arranging data significantly and is ideal for UDA. We evaluate the proposed SPA-UDA net on two public CXR datasets for lung disease recognition. The experimental results show that the proposed framework achieves significant performance improvements compared to other state-of-the-art UDA methods.

Unsupervised Domain Adaptation for Medical Image Segmentation by Selective Entropy Constraints and Adaptive Semantic Alignment

Generalizing a deep learning model to new domains is crucial for computer-aided medical diagnosis systems. Most existing unsupervised domain adaptation methods have made significant progress in reducing the domain distribution gap through adversarial training. However, these methods may still produce overconfident but erroneous results on unseen target images. This paper proposes a new unsupervised domain adaptation framework for cross-modality medical image segmentation. Specifically, We first introduce two data augmentation approaches to generate two sets of semantics-preserving augmented images. Based on the model's predictive consistency on these two sets of augmented images, we identify reliable and unreliable pixels. We then perform a selective entropy constraint: we minimize the entropy of reliable pixels to increase their confidence while maximizing the entropy of unreliable pixels to reduce their confidence. Based on the identified reliable and unreliable pixels, we further propose an adaptive semantic alignment module which performs class-level distribution adaptation by minimizing the distance between same class prototypes between domains, where unreliable pixels are removed to derive more accurate prototypes. We have conducted extensive experiments on the cross-modality cardiac structure segmentation task. The experimental results show that the proposed method significantly outperforms the state-of-the-art comparison algorithms. Our code and data are available at https://github.com/fengweie/SE_ASA.