Unsupervised Domain Adaptation Research Articles

Prototype-Optimized unsupervised domain adaptation via dynamic Transformer encoder for sensor drift compensation in electronic nose systems

In the field of electronic nose systems, sensor drift poses a significant challenge, affecting the reliability and accuracy of gas detection. Current solutions often require labeled data and fail to generalize well across different domains. This paper presents a novel, unsupervised domain adaptation framework for sensor drift compensation, leveraging a dynamic Transformer-based encoder and prototype learning. Our approach extracts semantic representations from source domain data and aligns instances to prototypes for knowledge transfer across domains. A dynamic prototype-guided classification model is deployed for drift compensation. The key contributions of this work include the introduction of prototype-optimized unsupervised learning and the development of an end-to-end drift compensation model. Experimental results on two gas sensor datasets demonstrate superior performance over existing unsupervised and semi-supervised methods, validating the effectiveness of our approach.

DCST: Dual Cross-Supervision for Transformer-based Unsupervised Domain Adaptation

Unsupervised Domain Adaptation aims to leverage a source domain with ample labeled data to tackle tasks on an unlabeled target domain. However, this poses a significant challenge, particularly in scenarios exhibiting significant disparities between the two domains. Prior methods often fall short in challenging domains due to the impact of incorrect pseudo-labeling noise and the limits of handcrafted domain alignment rules. In this paper, we propose a novel method called DCST (Dual Cross-Supervision Transformer), which improves upon existing methods in two key aspects. Firstly, vision transformer is combined with dual cross-supervision learning strategy to enforce consistency learning from different domains. The network accomplishes domain-specific self-training and cross-domain feature alignment in an adaptive manner. Secondly, due to the presence of noise in challenging domain, and the need to reduce the risks of model collapse and overfitting, we propose a Domain Shift Filter. Specifically, this module allows the model to leverage the memory of source domain features to facilitate a smooth transition. It can also improve the effectiveness of knowledge transfer between domains with significant gaps. We conduct extensive experiments on four benchmark datasets and achieved the best classification results, including 94.3% on Office-31, 86.0% on Office-Home, 89.3% on VisDA-2017, and 48.8% on DomainNet. Code is available in https://github.com/Yislight/DCST.

A two-layer regression network for robust and accurate domain adaptation

Naturally, classification models suffer from severe performance degradation when they are tested on datasets different from the ones used in training. Unsupervised domain adaptation helps to improve the generalizability of a pre-trained model by transferring knowledge from the labeled source domain (i.e., training dataset) to the unlabeled target domain (i.e., test dataset). The typical way of domain adaptation is to align the data distributions in embedding spaces between source and target domains. However, most existing works only enhance cross-domain consistency in the latent space disregarding the impact of source samples on the target classification task. Besides, discovering a subspace shared by two domains first and then training a transfer classifier separately hardly ensures that the obtained target features are suited for the classification model. To address these issues, in this work, we propose a two-layer regression network for domain adaptation (TRN-DA). TRN-DA learns class-wise domain invariant feature representations by jointly optimizing three tasks: supervised classification of source domain, unsupervised reconstruction of target domain, and alignment of cross-domain distributions. Furthermore, we introduce the concept of weight (importance) for source instances so that the resulting classifier can adapt well to the target domain. We formulate the domain adaptation problem as a unified optimization problem and solve it in an iterative way. Experimental results on different cross-domain image classification tasks demonstrate the encouraging performance of our TRN-DA compared to several recent state-of-the-art methods.

Learning Domain Invariant Features for Unsupervised Indoor Depth Estimation Adaptation

Predicting depth maps from monocular images has made an impressive performance in the past years. However, most depth estimation methods are trained with paired image-depth map data or multi-view images (e.g., stereo pair and monocular sequence), which suffer from expensive annotation costs and poor transferability. Although unsupervised domain adaptation methods are introduced to mitigate the reliance on annotated data, rare works focus on the unsupervised cross-scenario indoor monocular depth estimation. In this article, we propose to study the generalization of depth estimation models across different indoor scenarios in an adversarial-based domain adaptation paradigm. Concretely, a domain discriminator is designed for discriminating the representation from source and target domains, while the feature extractor aims to confuse the domain discriminator by capturing domain-invariant features. Further, we reconstruct depth maps from latent representations with the supervision of labeled source data. As a result, the feature extractor learned features possess the merit of both domain-invariant and low source risk, and the depth estimator can deal with the domain shift between source and target domains. We conduct the cross-scenario and cross-dataset experiments on the ScanNet and NYU-Depth-v2 datasets to verify the effectiveness of our method and achieve impressive performance.

MURDA: Multisource Unsupervised Raman Spectroscopy Domain Adaptation Model with Reconstructed Target Domains for Medical Diagnosis Assistance.

Artificial intelligence combined with Raman spectroscopy for disease diagnosis is on the rise. However, these methods require a large amount of annotated spectral data for modeling to achieve high diagnostic accuracy. Annotating labels consumes significant medical resources and time. To reduce dependence on labeled medical data resources, we propose a method called Multisource Unsupervised Raman Spectroscopy Domain Adaptation Model with Reconstructed Target Domains (MURDA). It transfers knowledge learned from source domain data sets of different diseases to an unlabeled target domain data set. Compared to knowledge transfer from a single source domain, knowledge from multiple disease source domains provides more generalized knowledge. Considering the diversity of autoimmune diseases and the limited sample size, we apply MURDA to assist in the medical diagnosis of autoimmune diseases. Additionally, we propose a Double-Branch Multiscale Convolutional Self-Attention (DMCS) feature extractor that is more suitable for spectral data feature extraction. On three sets of serum Raman spectroscopy data sets for autoimmune diseases, the multisource domain adaptation diagnostic accuracy of MURDA was superior to traditional single source and multisource models, with accuracy rates of 73.6%, 83.4%, and 82.9%, respectively. Compared with pure source tasks without domain adaptation, it improved by 15.1%, 36%, and 21.6%, respectively. This demonstrates the effectiveness of Raman spectroscopy combined with MURDA in diagnosing autoimmune diseases. We investigated the important decision dependency peaks in migration tasks, providing assistance for future research on artificial intelligence combined with Raman spectroscopy for diagnosing autoimmune diseases. Furthermore, to validate the effectiveness and generalization performance of MURDA, we conducted experiments on the publicly available RRUFF data set, exploring the application of multisource unsupervised domain adaptation in more Raman spectroscopy scenarios.

Robust Remote Sensing Scene Interpretation Based on Unsupervised Domain Adaptation

Deep learning models excel in interpreting the exponentially growing amounts of remote sensing data; however, they are susceptible to deception and spoofing by adversarial samples, posing catastrophic threats. The existing methods to combat adversarial samples have limited performance in robustness and efficiency, particularly in complex remote sensing scenarios. To tackle these challenges, an unsupervised domain adaptation algorithm is proposed for the accurate identification of clean images and adversarial samples by exploring a robust generative adversarial classification network that can harmonize the features between clean images and adversarial samples to minimize distribution discrepancies. Furthermore, linear polynomial loss as a replacement for cross-entropy loss is integrated to guide robust representation learning. Additionally, we leverage the fast gradient sign method (FGSM) and projected gradient descent (PGD) algorithms to generate adversarial samples with varying perturbation amplitudes to assess model robustness. A series of experiments was performed on the RSSCN7 dataset and SIRI-WHU dataset. Our experimental results illustrate that the proposed algorithm performs exceptionally well in classifying clean images while demonstrating robustness against adversarial perturbations.

Overcoming learning bias via Prototypical Feature Compensation for source-free domain adaptation

The focus of Source-free Unsupervised Domain Adaptation (SFUDA) is to effectively transfer a well-trained model from the source domain to an unlabelled target domain. During the target domain adaptation, the source domain data is no longer accessible. Prevalent methodologies attempt to synchronize the data distributions between the source and target domains, utilizing pseudo-labels to impart categorical information, which has made some progress in improving the model’s performance. However, performance impairments persist due to the introduction of learning bias from the source model and the impact of noisy pseudo-labels generated for the target domain. In this research, we reveal that the central cause for feature misalignment during domain transition is the learning bias, which is generated by the discrepancy of information between source and target domain data. The source domain data may contain distinguishable features that do not appear on the target domain, which causes the pre-trained source model to fail to work during domain adaptation. To overcome the information discrepancy, we propose a Prototypical Feature Compensation (PFC) Network. The network extracts representative feature maps of the source domain. Then use them to minimize the discrepancy information in the target domain feature maps. This mechanism facilitates feature alignment across different domains, allowing the model to generate more accurate categorical data through pseudo-labelling. The experimental results and ablation studies demonstrate exceptional performance on three SFUDA datasets and provide evidence of the proposed PFC method’s ability to adjust the feature distribution of both source and target domain data, ensuring their overlap in the latent space.

Towards Discriminative Class-Aware Domain Alignment via Coding Rate Reduction for Unsupervised Adversarial Domain Adaptation

Unsupervised domain adaptation (UDA) methods, based on adversarial learning, employ the means of implicit global and class-aware domain alignment to learn the symmetry between source and target domains and facilitate the transfer of knowledge from a labeled source domain to an unlabeled target domain. However, these methods still face misalignment and poor target generalization due to small inter-class domain discrepancy and large intra-class discrepancy of target features. To tackle these challenges, we introduce a novel adversarial learning-based UDA framework named Coding Rate Reduction Adversarial Domain Adaptation (CR2ADA) to better learn the symmetry between source and target domains. Integrating conditional domain adversarial networks with domain-specific batch normalization, CR2ADA learns robust domain-invariant features to implement global domain alignment. For discriminative class-aware domain alignment, we propose the global and local coding rate reduction methods in CR2ADA to maximize inter-class domain discrepancy and minimize intra-class discrepancy of target features. Additionally, CR2ADA combines minimum class confusion and mutual information to further regularize the diversity and discriminability of the learned features. The effectiveness of CR2ADA is demonstrated through experiments on four UDA datasets. The code can be obtained through email or GitHub.

Subfield-level crop yield mapping without ground truth data: A scale transfer framework

Ongoing advances in satellite remote sensing data and machine learning methods have enabled crop yield estimation at various spatial and temporal resolutions. While yield mapping at broader scales (e.g., state or county level) has become common, mapping at finer scales (e.g., field or subfield) has been limited by the lack of ground truth data for model training and evaluation. Here we present a scale transfer framework, named Quantile loss Domain Adversarial Neural Networks (QDANN), that leverages knowledge from county-level datasets to map crop yields at the subfield level. Based on the strategy of unsupervised domain adaptation, QDANN is trained on labeled county-level data and unlabeled subfield-level data, with no requirement for yield information at the subfield level. We evaluate the proposed method applied to Landsat imagery and Gridmet weather data for maize, soybean, and winter wheat fields in the United States, using as reference data yield monitor records from roughly one million field-year observations. The model is compared with several process-based and machine learning-based benchmark approaches that train on simulated yield records or county-level data. QDANN-estimated yields achieved an R2 score (RMSE) of 48 % (2.29 t/ha), 32 % (0.85 t/ha), and 39 % (1.40 t/ha) for maize, soybean, and winter wheat in comparison with the ground-based yield measures, respectively. These performances are higher than benchmark approaches and are nearly as good as models trained on field-level data. When aggregated to the county level, the improvement achieved by QDANN is more pronounced and the R2 scores (RMSE) improved to 78 % (0.98 t/ha), 62 % (0.37 t/ha), and 53 % (1.00 t/ha) for maize, soybean, and winter wheat, respectively. This study demonstrates that the proposed scale transfer framework can serve as a reliable approach for yield mapping at the subfield level when there is no access to fine-scale yield information. Based on the QDANN model, we have generated and made publicly available 30-m annual yield maps for major crop-producing states in the U.S. since 2008.

Unsupervised domain adaptation bearing fault diagnosis method based on joint feature alignment

In this paper, the issue of cross-condition fault diagnosis of bearing is studied. During actual operation, the conditions of bearing vary due to changes in factors such as rotation speed and load, and the data distribution between different working conditions varies. Deep learning models that perform well in one condition are not ideal when applied to another condition directly. To address this problem, a novel unsupervised domain adaptation fault diagnosis method based on joint feature alignment is proposed in this paper. 1D-CNN is used as a weight-shared feature extractor to extract the features from both the source and target domains. The discrepancies in marginal and conditional distributions between the source and target domains are comprehensively considered by multi-layer multi-bandwidth Cauchy kernel maximum mean discrepancy (MB-CMMD) and mutual information (MI). The domain drift is reduced by aligning the feature representations of source and target domains. The network after feature alignment demonstrates a notable enhancement in the diagnostic accuracy of unlabeled samples within the target domain. The experimental results demonstrate that, in comparison to other domain adaptation approaches, The proposed approach can significantly enhance the accuracy of fault diagnosis while realizing feature alignment.

RT-SNDETR: real-time supernova detection via end-to-end image transformers

ABSTRACT In large-scale astronomical surveys, traditional supernova detection pipelines rely on complex and relatively inefficient image differencing techniques. This paper proposes an end-to-end deep-learning supernova detection network, the Real-Time SuperNova DEtection TRansformer (RT-SNDETR). This network partially replaces traditional pipelines by integrating image differencing, source detection, and Real-bogus classification, achieving a speed 51.49 times that of the fastest image differencing method, SFFT. Additionally, it remains competitive with methods like YOLO v8, offering a well-balanced trade-off between speed and accuracy. Experimental results highlight RT-SNDETR’s superior performance, with an average precision(AP) of 96.30 per cent on synthetic samples and 76.60 per cent on real supernova samples. It significantly outperforms other detection networks, including RT-DETR (+5.6 per cent AP on synthetic/+5.1 per cent AP on real samples) and Cascade R-CNN (+8.9 per cent AP on synthetic/ +28.6 per cent AP on real samples). The incorporation of CycleGAN-based data generation methods plays a significant role in enhancing RT-SNDETR’s performance. These methods simulate realistic PSF variations, enabling the object detection network to learn more robust features and improving its generalization to real supernovae data. Additionally, by integrating unsupervised domain adaptation techniques, RT-SNDETR achieves an AP of 81.70 per cent on real SDSS supernova survey samples. This study demonstrates RT-SNDETR’s potential to significantly enhance both the speed and accuracy of supernova detection, making it a highly effective solution for large-scale astronomical surveys.

Advancing Nighttime Object Detection through Image Enhancement and Domain Adaptation

Due to the lack of annotations for nighttime low-light images, object detection in low-light images has always been a challenging problem. Achieving high-precision results at night is also an issue. Additionally, we aim to use a single nighttime dataset to complete the knowledge distillation task while improving the detection accuracy of object detection models under nighttime low-light conditions and reducing the computational cost of the model, especially for small targets and objects contaminated by special nighttime lighting. This paper proposes a Nighttime Unsupervised Domain Adaptation Network (NUDN) based on knowledge distillation to address these issues. To improve the detection accuracy of nighttime images, high-confidence bounding box predictions from the teacher and region proposals from the student are first fused, allowing the teacher to perform better in subsequent training, thus generating a combination of high-confidence and low-confidence pseudo-labels. This combination of feature information is used to guide model training, enabling the model to extract feature information similar to that of source images in nighttime low-light images. Nighttime images and pseudo-labels undergo random size transformations before being used as input for the student, enhancing the model’s generalization across different scales. To address the scarcity of nighttime datasets, we propose a nighttime-specific augmentation pipeline called LightImg. This pipeline enhances nighttime features, transforming them into daytime features and reducing issues such as backlighting, uneven illumination, and dim nighttime light, enabling cross-domain research using existing nighttime datasets. Our experimental results show that NUDN can significantly improve nighttime low-light object detection accuracy on the SHIFT and ExDark datasets. We conduct extensive experiments and ablation studies to demonstrate the effectiveness and efficiency of our work.

Crowd Counting Using Meta-Test-Time Adaptation.

Machine learning algorithms are commonly used for quickly and efficiently counting people from a crowd. Test-time adaptation methods for crowd counting adjust model parameters and employ additional data augmentation to better adapt the model to the specific conditions encountered during testing. The majority of current studies concentrate on unsupervised domain adaptation. These approaches commonly perform hundreds of epochs of training iterations, requiring a sizable number of unannotated data of every new target domain apart from annotated data of the source domain. Unlike these methods, we propose a meta-test-time adaptive crowd counting approach called CrowdTTA, which integrates the concept of test-time adaptation into the meta-learning framework and makes it easier for the counting model to adapt to the unknown test distributions. To facilitate the reliable supervision signal at the pixel level, we introduce uncertainty by inserting the dropout layer into the counting model. The uncertainty is then used to generate valuable pseudo labels, serving as effective supervisory signals for adapting the model. In the context of meta-learning, one image can be regarded as one task for crowd counting. In each iteration, our approach is a dual-level optimization process. In the inner update, we employ a self-supervised consistency loss function to optimize the model so as to simulate the parameters update process that occurs during the test phase. In the outer update, we authentically update the parameters based on the image with ground truth, improving the model's performance and making the pseudo labels more accurate in the next iteration. At test time, the input image is used for adapting the model before testing the image. In comparison to various supervised learning and domain adaptation methods, our results via extensive experiments on diverse datasets showcase the general adaptive capability of our approach across datasets with varying crowd densities and scales.

A global reweighting approach for cross-domain semantic segmentation

Unsupervised domain adaptation semantic segmentation attracts much research attention due to the expensive pixel-level annotation cost. Since the adaptation difficulty of samples is different, the weight of samples should be set independently, which is called reweighting. However, existing reweighting methods only calculate local reweighting information from predicted results or context information in batch images of two domains, which may lead to over-alignment or under-alignment problems. To handle this issue, we propose a global reweighting approach. Specifically, we first define the target centroid distance, which describes the distance between the source batch data and the target centroid. Then, we employ a Fréchet Inception Distance metric to evaluate the domain divergence and embed it into the target centroid distance. Finally, a global reweighting strategy is proposed to enhance the knowledge transferability in the source domain supervision. Extensive experiments demonstrate that our approach achieves competitive performance and helps to improve performance in other methods.

Gini Coefficient-Based Feature Learning for Unsupervised Cross-Domain Classification with Compact Polarimetric SAR Data

Remote sensing image classification usually needs many labeled samples so that the target nature can be fully described. For synthetic aperture radar (SAR) images, variations of the target scattering always happen to some extent due to the imaging geometry, weather conditions, and system parameters. Therefore, labeled samples in one image could not be suitable to represent the same target in other images. The domain distribution shift of different images reduces the reusability of the labeled samples. Thus, exploring cross-domain interpretation methods is of great potential for SAR images to improve the reuse rate of existing labels from historical images. In this study, an unsupervised cross-domain classification method is proposed that utilizes the Gini coefficient to rank the robust and stable polarimetric features in both the source and target domains (GRFST) such that an unsupervised domain adaptation (UDA) can be achieved. This method selects the optimal features from both the source and target domains to alleviate the domain distribution shift. Both fully polarimetric (FP) and compact polarimetric (CP) SAR features are explored for crop-domain terrain type classification. Specifically, the CP mode refers to the hybrid dual-pol mode with an arbitrary transmitting ellipse wave. This is the first attempt in the open literature to investigate the representing abilities of different CP modes for cross-domain terrain classification. Experiments are conducted from four aspects to demonstrate the performance of CP modes for cross-data, cross-scene, and cross-crop type classification. Results show that the GRFST-UDA method yields a classification accuracy of 2% to 12% higher than the traditional UDA methods. The degree of scene similarity has a certain impact on the accuracy of cross-domain crop classification. It was also found that when both the FP and circular CP SAR data are used, stable, promising results can be achieved.

Deep Learning-based Unsupervised Domain Adaptation via a Unified Model for Prostate Lesion Detection Using Multisite Biparametric MRI Datasets.

Purpose To determine whether the unsupervised domain adaptation (UDA) method with generated images improves the performance of a supervised learning (SL) model for prostate cancer (PCa) detection using multisite biparametric (bp) MRI datasets. Materials and Methods This retrospective study included data from 5150 patients (14 191 samples) collected across nine different imaging centers. A novel UDA method using a unified generative model was developed for PCa detection using multisite bpMRI datasets. This method translates diffusion-weighted imaging (DWI) acquisitions, including apparent diffusion coefficient (ADC) and individual diffusion-weighted (DW) images acquired using various b values, to align with the style of images acquired using b values recommended by Prostate Imaging Reporting and Data System (PI-RADS) guidelines. The generated ADC and DW images replace the original images for PCa detection. An independent set of 1692 test cases (2393 samples) was used for evaluation. The area under the receiver operating characteristic curve (AUC) was used as the primary metric, and statistical analysis was performed via bootstrapping. Results For all test cases, the AUC values for baseline SL and UDA methods were 0.73 and 0.79 (P < .001), respectively, for PCa lesions with PI-RADS score of 3 or greater and 0.77 and 0.80 (P < .001) for lesions with PI-RADS scores of 4 or greater. In the 361 test cases under the most unfavorable image acquisition setting, the AUC values for baseline SL and UDA were 0.49 and 0.76 (P < .001) for lesions with PI-RADS scores of 3 or greater and 0.50 and 0.77 (P < .001) for lesions with PI-RADS scores of 4 or greater. Conclusion UDA with generated images improved the performance of SL methods in PCa lesion detection across multisite datasets with various b values, especially for images acquired with significant deviations from the PI-RADS-recommended DWI protocol (eg, with an extremely high b value). Keywords: Prostate Cancer Detection, Multisite, Unsupervised Domain Adaptation, Diffusion-weighted Imaging, b Value Supplemental material is available for this article. © RSNA, 2024.

TriLA: Triple-Level Alignment Based Unsupervised Domain Adaptation for Joint Segmentation of Optic Disc and Optic Cup.

Cross-domain joint segmentation of optic disc and optic cup on fundus images is essential, yet challenging, for effective glaucoma screening. Although many unsupervised domain adaptation (UDA) methods have been proposed, these methods can hardly achieve complete domain alignment, leading to suboptimal performance. In this paper, we propose a triple-level alignment (TriLA) model to address this issue by aligning the source and target domains at the input level, feature level, and output level simultaneously. At the input level, a learnable Fourier domain adaptation (LFDA) module is developed to learn the cut-off frequency adaptively for frequency-domain translation. At the feature level, we disentangle the style and content features and align them in the corresponding feature spaces using consistency constraints. At the output level, we design a segmentation consistency constraint to emphasize the segmentation consistency across domains. The proposed model is trained on the RIGA+ dataset and widely evaluated on six different UDA scenarios. Our comprehensive results not only demonstrate that the proposed TriLA substantially outperforms other state-of-the-art UDA methods in joint segmentation of optic disc and optic cup, but also suggest the effectiveness of the triple-level alignment strategy.

FPL+: Filtered Pseudo Label-Based Unsupervised Cross-Modality Adaptation for 3D Medical Image Segmentation.

Adapting a medical image segmentation model to a new domain is important for improving its cross-domain transferability, and due to the expensive annotation process, Unsupervised Domain Adaptation (UDA) is appealing where only unlabeled images are needed for the adaptation. Existing UDA methods are mainly based on image or feature alignment with adversarial training for regularization, and they are limited by insufficient supervision in the target domain. In this paper, we propose an enhanced Filtered Pseudo Label (FPL+)-based UDA method for 3D medical image segmentation. It first uses cross-domain data augmentation to translate labeled images in the source domain to a dual-domain training set consisting of a pseudo source-domain set and a pseudo target-domain set. To leverage the dual-domain augmented images to train a pseudo label generator, domain-specific batch normalization layers are used to deal with the domain shift while learning the domain-invariant structure features, generating high-quality pseudo labels for target-domain images. We then combine labeled source-domain images and target-domain images with pseudo labels to train a final segmentor, where image-level weighting based on uncertainty estimation and pixel-level weighting based on dual-domain consensus are proposed to mitigate the adverse effect of noisy pseudo labels. Experiments on three public multi-modal datasets for Vestibular Schwannoma, brain tumor and whole heart segmentation show that our method surpassed ten state-of-the-art UDA methods, and it even achieved better results than fully supervised learning in the target domain in some cases.

Integrating multimodal contrastive learning with prototypical domain alignment for unsupervised domain adaptation of time series

Unsupervised domain adaptation (UDA) addresses the challenge of transferring knowledge from a labeled source domain to an unlabeled target domain. This task is particularly critical for time series data, characterized by unique temporal dynamics. However, existing methods often fail to capture these temporal dependencies, leading to domain discrepancies and loss of semantic information. In this study, we propose a novel framework for the unsupervised domain adaptation of time series (UDATS) that integrates Multimodal Contrastive Adaptation (MCA) and Prototypical Domain Alignment (PDA). MCA leverages image encoding techniques and prompt learning to capture complex temporal patterns while preserving semantic information. PDA constructs multimodal prototypes, combining visual and textual features to align target domain samples accurately. Our framework demonstrates superior performance across various application domains, including human activity recognition, mortality prediction, and fault detection. Experiments show our method effectively addresses domain discrepancies while preserving essential semantic content, outperforming state-of-the-art models.

Rainforest: A three-stage distribution adaptation framework for unsupervised time series domain adaptation

Solving the unsupervised domain adaptation (UDA) task in time series is of great significance for practical applications, such as human activity recognition and machine fault diagnosis. Compared to UDA for computer vision, UDA in time series is more challenging due to the dynamics of time series data and the complex dependencies among different time steps. Existing UDA methods for time series fail to adequately capture the temporal dependencies, limiting their ability to learn domain-invariant temporal patterns. Furthermore, most UDA methods only focus on distribution adaptation on the backbone network without considering how the classifier adapts to the data distribution of the target domain. In this paper, we propose Rainforest, a three-stage UDA framework for time series. We first pre-train the backbone network through a self-supervised method called bidirectional autoregression, so that the model can comprehensively learn the temporal dependencies in time series. Next, we propose a novel meta-learning-based distribution adaptation method to achieve the joint alignment of the global and local distributions while encouraging the model to adaptively reduce the temporal dynamic differences among different domains. Finally, we design a pseudo-label-guided fine-tuning strategy to help the classifier estimate the data distribution of the target domain more accurately. Extensive experiments on four real-world time series datasets show that our Rainforest outperforms state-of-the-art methods, with an average improvement of 2.19% in accuracy and 2.41% in MF1-score.