Pseudo Labeling Research Articles

Self-supervised based clustering for retinal optical coherence tomography images.

In response to the inadequacy of manual analysis in meeting the rising demand for retinal optical coherence tomography (OCT) images, a self-supervised learning-based clustering model was implemented. A public dataset was utilized, with 83,484 OCT images with categories of choroidal neovascularization (CNV), diabetic macular edema (DME), drusen, and normal fundus. This study employed the Semantic Pseudo Labeling for Image Clustering (SPICE) framework, a self-supervised learning-based method, to cluster unlabeled OCT images into binary and four categories, and the performances were compared with baseline models. We also analysed feature distribution using t-SNE, and explored the cluster centers, attention maps, and misclassified images. In addition, DME and CNV subsets were clustered binarily, and the results were interpreted by two retinal specialists. SPICE demonstrated superior performance in binary and four categories classification tasks, achieving the accuracy of 0.886 and 0.846, respectively. In t-SNE analysis, the four types exhibited significant clustering into distinct groups. The cluster centers corresponded to the human labels, and the heat map revealed that the model focused on important biomarkers. The misclassified images exposed similar features to the inaccurate classes. The model also grouped DME and CNV into two distinct categories respectively. Self-supervised clustering effectively distinguished disease variances and revealed common features, with a notable capability to detect disease heterogeneity through biomarkers.

Inter-seasons and Inter-households Domain Adaptation Based on DANNs and Pseudo Labeling for Non-Intrusive Occupancy Detection

Inter-seasons and Inter-households Domain Adaptation Based on DANNs and Pseudo Labeling for Non-Intrusive Occupancy Detection

Alleviating confirmation bias in perpetually dynamic environments: Continuous unsupervised domain adaptation-based condition monitoring (CUDACoM)

MotivationDeep learning (DL) has revolutionized condition monitoring (CoM) in mechanical systems by reducing manual signal processing. However, DL's industrial integration is limited due to low robustness against distribution shifts. Existing CoM approaches focus on adaptating to one-time distribution shift. We introduce Continuous Unsupervised Domain Adaptation-based CoM (CUDACoM), tackling continuous distribution shifts in systems under perpetually dynamic conditions. MethodologyCUDACoM mitigates confirmation bias, detrimental in long-domain sequences, by introducing two novel strategies (1) Fresh Initialization and (2) In-Domain Pseudo Labeling. Fresh Initialization maintains high model plasticity, while In-Domain pseudo-labeling improves pseudo-label accuracy, enhancing model adaptability. These strategies reduce confirmation bias, crucial for robust self-training, making CUDACoM ideal for long-sequence domain adaptation in perpetually dynamic environments. ResultsCUDACoM outperforms state-of-the-art (SOTA) adversarial and self-training approaches. Validated through two practical case studies: a 200% (RPM) change and gradual sensor degradation across 40 noise levels. These challenging case studies show stronger data shifts than the commonly used standard benchmarking datasets. The second case is especially novel, formulating robustness to noise as a domain adaptation problem. CUDACoM achieved a test accuracy of 0.937 in the RPM case (vs. SOTA's 0.770) and 0.849 in the sensor degradation case (vs. SOTA's 0.751). ImpactThis study addresses the overlooked challenges of employing DL for CoM in perpetually dynamic environments, particularly, the confirmation bias. With CUDACoM's computational efficiency, we provide a practical solution that enhances reliability by facilitating the integration of robust DL into industrial CoM systems.

Integrating pseudo labeling with contrastive clustering for transformer-based semi-supervised action recognition

Integrating pseudo labeling with contrastive clustering for transformer-based semi-supervised action recognition

Heterogeneous domain adaptation via incremental discriminative knowledge consistency

Heterogeneous domain adaptation is a challenging problem in transfer learning since samples from the source and target domains reside in different feature spaces with different feature dimensions. The key problem is how to minimize some gaps (e.g., data distribution mismatch) presented in two heterogeneous domains and produce highly discriminative representations for the target domain. In this paper, we attempt to address these challenges with the proposed incremental discriminative knowledge consistency (IDKC) method, which integrates cross-domain mapping, distribution matching, discriminative knowledge preservation, and domain-specific geometry structure consistency into a unified learning model. Specifically, we attempt to learn a domain-specific projection to project original samples into a common subspace in which the marginal distribution is well aligned and the discriminative knowledge consistency is preserved by leveraging the labeled samples from both domains. Moreover, domain-specific structure consistency is enforced to preserve the data manifold from the original space to the common feature space in each domain. Meanwhile, we further apply pseudo labeling to unlabeled target samples based on the feature correlation and retain pseudo labels with high feature correlation coefficients for the next iterative learning. Our pseudo-labeling strategy expands the number of labeled target samples in each category and thus enforces class-discriminative knowledge consistency to produce more discriminative feature representations for the target domain. Extensive experiments on several standard benchmarks for object recognition, cross-language text classification, and digit classification tasks verify the effectiveness of our method.

Source-free domain adaptation via dynamic pseudo labeling and Self-supervision

Recently, unsupervised domain adaptation (UDA) has attracted extensive interest in relieving the greedy requirement of vanilla deep learning for labeled data. It seeks for a solution to adapt the knowledge from a well-labeled training dataset (source domain) to another unlabeled target dataset (target domain). However, in some practical scenarios, the source domain data is inaccessible for a variety of reasons, and only a model trained on it can be provided, thus deriving a more challenging task, i.e., source-free unsupervised domain adaptation (SFUDA). Some pseudo labeling-based methods have been proposed to solve it by predicting pseudo labels for the unlabeled target domain data. Nevertheless, incorrectly designated pseudo labels will impose an adverse impact on the network adaptation. To alleviate this issue, we propose a dynamic confidence-based pseudo labeling strategy for SFUDA in this paper. Unlike those methods that first rigidly assign pseudo labels to all target domain data and then try to weaken the effect of incorrect pseudo labels in training, we proactively label the target samples with higher confidence in a dynamic manner. To further relieve the impact of incorrect pseudo labels, we harness the collaborative learning to constrain the consistency of the network and impose an additional soft supervision. Besides, we also investigate the possible problem brought by our labeling strategy, i.e., the neglect of wavering samples near the decision boundary, and solve it by injecting the self-supervised learning into our model. Experiments on three UDA benchmark datasets demonstrate the state-of-the-art performance of our proposed method. The code is publicly available at https://github.com/meowpass/DPLS.

CoNPL: Consistency training framework with noise-aware pseudo labeling for dense pose estimation

Dense pose estimation faces hurdles due to the lack of costly precise pixel-level IUV labels. Existing methods aim to overcome it by regularizing model outputs or interpolating pseudo labels. However, conventional geometric transformations often fall short, and pseudo labels may introduce unwanted noise, leading to continued challenges in rectifying inaccurate estimations. We introduced a novel Consistency training framework with Noise-aware Pseudo Labeling (CoNPL) to tackle problems in learning from unlabeled pixels. CoNPL employs both weak and strong augmentations in a shared model to enhance robustness against aggressive transformations. To address noisy pseudo labels, CoNPL integrates a Noise-aware Pseudo Labeling (NPL) module, which consists of a Noise-Aware Module (NAM), and Noise-Resistant Learning (NRL) modules. NAM identifies misclassifications and incorrect UV coordinates using binary classification and regression, while NRL dynamically adjusts loss weights to filter out uncertain samples, thereby stabilizing learning from pseudo labels. Our method demonstrates a + 2.0% improvement in AP on the DensePose-COCO benchmark across different networks, achieving state-of-the-art performance. On the Ultrapose and DensePose-Chimps benchmark, our method also demonstrates a + 2.7% and + 3.0% improvement in AP.

Semantic contrast with uncertainty-aware pseudo label for lumbar semi-supervised classification

BackgroundLumbar disc herniation (LDH) is a prevalent spinal disease that can result in severe pain, with Magnetic resonance imaging (MRI) serving as a commonly diagnostic tool. However, annotating numerous MRI images, necessary for deep learning based LDH diagnosis, can be challenging and labor-intensive. Semi-supervised learning, mainly utilizing pseudo labeling and consistency regularization, can leverage limited labeled images and abundant unlabeled images. However, consistency regularization solely focuses on maintaining the semantic consistency of transformed unlabeled data but fails to utilize the semantic information from labeled data to guide the unlabeled data, and additionally, pseudo labeling is prone to confirmation bias. MethodWe propose SeCoFixMatch, an innovative approach that seamlessly integrates semantic contrast and uncertainty-aware pseudo labeling into semi-supervised learning. Semantic contrast constraints the semantic consistency between labeled and unlabeled images. Pseudo labels are generated by combining predictive confidence and uncertainty, with uncertainty computing by optimizing the Kullback-Leibler (KL) loss between predictive and target Dirichlet distribution. ResultsComparison with other semi-supervised models and ablation experiment with varying labeled data demonstrate the effectiveness and generalization of proposed model. Notably, SeCoFixMatch, trained with just 40 labels, outperforms the baseline model trained with 200 labels, reducing the annotation effort by a remarkable 80%. ConclusionsProposed pseudo labeling algorithm generates more precise pseudo labels for semantic contrastive learning and semantic contrastive learning facilitates better feature representation, thereby further improving the prediction accuracy of pseudo label. The mutual reinforcement of pseudo labeling and semantic contrast constraints boosts the performance of semi-supervised algorithm.

Mask the Unknown: Assessing Different Strategies to Handle Weak Annotations in the MICCAI2023 Mediastinal Lymph Node Quantification Challenge

Pathological lymph node delineation is crucial in cancer diagnosis, progression assessment, and treatment planning. The MICCAI 2023 Lymph Node Quantification Challenge published the first public dataset for pathological lymph node segmentation in the mediastinum. As lymph node annotations are expensive, the challenge was formed as a weakly supervised learning task, where only a subset of all lymph nodes in the training set have been annotated. For the challenge submission, multiple methods for training on these weakly supervised data were explored, including noisy label training, loss masking of unlabeled data, and an approach that integrated the TotalSegmentator toolbox as a form of pseudo labeling in order to reduce the number of unknown voxels. Furthermore, multiple public TCIA datasets were incorporated into the training to improve the performance of the deep learning model. Our submitted model achieved a Dice score of 0.628 and an average symmetric surface distance of 5.8~mm on the challenge test set. With our submitted model, we accomplished the third rank in the MICCAI2023 LNQ challenge. A finding of our analysis was that the integration of all visible, including non-pathological lymph nodes improved the overall segmentation performance on pathological lymph nodes of the test set. Furthermore, segmentation models trained only on clinically enlarged lymph nodes, as given in the challenge scenario, could not generalize to smaller pathological lymph nodes. The code and model for the challenge submission are available at <a href='https://gitlab.lrz.de/compai/MediastinalLymphNodeSegmentation'>https://gitlab.lrz.de/compai/MediastinalLymphNodeSegmentation</a>

Improving Object Detection Accuracy with Self-Training Based on Bi-Directional Pseudo Label Recovery

Semi-supervised training methods need reliable pseudo labels for unlabeled data. The current state-of-the-art methods based on pseudo labeling utilize only high-confidence predictions, whereas poor confidence predictions are discarded. This paper presents a novel approach to generate high-quality pseudo labels for unlabeled data. It utilizes predictions with high- and low-confidence levels to generate refined labels and then validates the accuracy of those predictions through bi-directional object tracking. The bi-directional object tracker leverages both past and future information to recover missing labels and increase the accuracy of the generated pseudo labels. This method can also substantially reduce the effort and time needed in label creation compared to the conventional manual labeling. The proposed method utilizes a buffer to accumulate detection labels (bounding boxes) predicted by the object detector. These labels are refined for accuracy though forward and backward tracking, ultimately constructing the final set of pseudo labels. The method is integrated in the YOLOv5 object detector and tested on the BDD100K dataset. Through the experiments, we demonstrate the effectiveness of the proposed scheme in automating the process of pseudo label generation with notably higher accuracy than the recent state-of-the-art pseudo label generation schemes. The results show that the proposed method outperforms previous methods in terms of mean average precision (mAP), label generation accuracy, and speed. Using the bi-directional recovery method, an increase in mAP@50 for the BDD100K dataset by 0.52% is achieved, and for the Waymo dataset, it provides an improvement of mAP@50 by 8.7% to 9.9% compared to 8.1% of the existing method when pre-training with 10% of the dataset. An improvement by 2.1% to 2.9% is achieved as compared to 1.7% of the existing method when pre-training with 20% of the dataset. Overall, the improved method leads to a significant enhancement in detection accuracy, achieving higher mAP scores across various datasets, thus demonstrating its robustness and effectiveness in diverse conditions.

Facial Action Unit detection based on multi-task learning strategy for unlabeled facial images in the wild

Facial Action Unit (AU) detection often relies on highly-cost accurate labeling or inaccurate pseudo labeling techniques in recent years. How to introduce large amounts of unlabeled facial images in the wild into supervised AU detection frameworks has become a challenging problem. Additionally, nearly every type of AUs has the problem of unbalanced positive and negative samples. Inspired by other multi-task learning frameworks, we first propose a multi-task learning strategy boosting AU detection in the wild through jointing facial landmark detection and AU domain separation and reconstruction. Our introduced dual domains facial landmark detection framework can solve the lack of accurate facial landmark coordinates during the AU domain separation and reconstruction training process, while the parameters of homostructural facial extraction modules from these two similar facial tasks are shared. Moreover, we propose a pixel-level feature alignment scheme to maintain the consistency of features obtained from two separation and reconstruction processes. Furthermore, a weighted asymmetric loss is proposed to change the contribution of positive and negative samples of each type of AUs to model parameters updating. Experimental results on three widely used benchmarks demonstrate our superiority to most state-of-the-art methods for AU detection.

Consistency-guided pseudo labeling for transductive zero-shot learning

Zero-shot learning (ZSL) aims to recognize unseen classes during training. Transductive methods have advanced in ZSL, however, often rely on pseudo labels based on confidence scores, leading to semantic misalignment between unseen-class image features and corresponding class semantic descriptions due to noisy pseudo labels. In this paper, we introduce a novel Consistency-Guided Pseudo-Labeling (CGPL) to generate high-quality pseudo labels, achieving robust mapping from visual to semantic space for unseen classes. CGPL incorporates a large-scale vision-language model as a collaborator with the ZSL model to generate high-quality pseudo-labels. Then, pseudo-labeled samples with consistent prediction of two models are added to the training set, to learn the visual-to-semantic mapping for unseen classes. Furthermore, we design a quasi-classification loss based on reconstructed unseen prototypes to learn accurate visual-semantic mapping. Consequently, CGPL is further encouraged to obtain higher-quality pseudo labels, and progressively learn the precise visual-semantic mapping for unseen classes throughout the iterative process. Our extensive experimental results across four benchmark datasets highlight the superior performance of CGPL in both CZSL and GZSL settings.

PR-PL: A Novel Prototypical Representation Based Pairwise Learning Framework for Emotion Recognition Using EEG Signals

Affective brain-computer interface based on electroencephalography (EEG) is an important branch in the field of affective computing. However, the individual differences in EEG emotional data and the noisy labeling problem in the subjective feedback seriously limit the effectiveness and generalizability of existing models. To tackle these two critical issues, we propose a novel transfer learning framework with Prototypical Representation based Pairwise Learning ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">PR-PL</i> ). The discriminative and generalized EEG features are learned for emotion revealing across individuals and the emotion recognition task is formulated as pairwise learning for improving the model tolerance to the noisy labels. More specifically, a prototypical learning is developed to encode the inherent emotion-related semantic structure of EEG data and align the individuals' EEG features to a shared common feature space under consideration of the feature separability of both source and target domains. Based on the aligned feature representations, pairwise learning with an adaptive pseudo labeling method is introduced to encode the proximity relationships among samples and alleviate the label noises effect on modeling. Extensive results on two benchmark databases (SEED and SEED-IV) under four different cross-validation evaluation protocols validate the model reliability and stability across subjects and sessions. Compared to the literature, the average enhancement of emotion recognition across four different evaluation protocols is 2.04% (SEED) and 2.58% (SEED-IV). The source code is available at <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><uri>https://github.com/KAZABANA/PR-PL</uri></i> .

THE NO TRAIN NO GAIN SYSTEM FOR O-COCOSDA AND VLSP 2022 - A-MSV SHARED TASK: ASIAN MULTILINGUAL SPEAKER VERIFICATION

This paper proposes a semi-supervised multilingual speaker verification (MSV) system submitted for the 2 tasks, MSV for the Asian language inside the training set (T01) and outside the training set (T02) in O-COCOSDA and VLSP challenge 2022.To solve the problem, our strategy is training a baseline acoustic model with given labeled data (MSV CommonVoice) andfine-tuning the trained acoustic model with both given labeled data and given unlabeled data (MSV Youtube). To achieve the fine-tuning step, the unlabeled data is converted to labeled data by pseudo labeling technique using the clustering method with the embedding vectors extracted from the trained acoustic model. Besides, we also apply test-time augmentation, back-end scoring, and score normalization with the AS-Norm technique to improve the result. When evaluated on the VLSP 2022 challenge's given test set, our best system with baseline ECAPA-TDNN achieves an equal error rate (EER) of 2.296% in T01 and 3.3296% in T02, which ranks second rank in both two tasks.

Semi-supervised TEE Segmentation via Interacting with SAM Equipped with Noise-Resilient Prompting

Semi-supervised learning (SSL) is a powerful tool to address the challenge of insufficient annotated data in medical segmentation problems. However, existing semi-supervised methods mainly rely on internal knowledge for pseudo labeling, which is biased due to the distribution mismatch between the highly imbalanced labeled and unlabeled data. Segmenting left atrial appendage (LAA) from transesophageal echocardiogram (TEE) images is a typical medical image segmentation task featured by scarcity of professional annotations and diverse data distributions, for which existing SSL models cannot achieve satisfactory performance. In this paper, we propose a novel strategy to mitigate the inherent challenge of distribution mismatch in SSL by, for the first time, incorporating a large foundation model (i.e. SAM in our implementation) into an SSL model to improve the quality of pseudo labels. We further propose a new self-reconstruction mechanism to generate both noise-resilient prompts to demonically improve SAM’s generalization capability over TEE images and self-perturbations to stabilize the training process and reduce the impact of noisy labels. We conduct extensive experiments on an in-house TEE dataset; experimental results demonstrate that our method achieves better performance than state-of-the-art SSL models.

Innovative approach for predicting daily reference evapotranspiration using improved shallow and deep learning models in a coastal region: A comparative study

Accurate and reliable estimation of Reference Evapotranspiration (ETo) is crucial for water resources management, hydrological processes, and agricultural production. The FAO-56 Penman-Monteith (FAO-56PM) approach is recommended as the standard model for ETo estimation; nevertheless, the absence of comprehensive meteorological variables at many global locations frequently restricts its implementation. This study compares shallow learning (SL) and deep learning (DL) models for estimating daily ETo against the FAO-56PM approach based on various statistic metrics and graphic tool over a coastal Red Sea region, Sudan. A novel approach of the SL model, the Catboost Regressor (CBR) and three DL models: 1D-Convolutional Neural Networks (1D-CNN), Long Short-Term Memory (LSTM), and Gated Recurrent Unit (GRU) were adopted and coupled with a semi-supervised pseudo-labeling (PL) technique. Six scenarios were developed regarding different input combinations of meteorological variables such as air temperature (Tmin, Tmax, and Tmean), wind speed (U2), relative humidity (RH), sunshine hours duration (SSH), net radiation (Rn), and saturation vapor pressure deficit (es-ea). The results showed that the PL technique reduced the systematic error of SL and DL models during training for all the scenarios. The input combination of Tmin, Tmax, Tmean, and RH reflected higher performance than other combinations for all employed models. The CBR-PL model demonstrated good generalization abilities to predict daily ETo and was the overall superior model in the testing phase according to prediction accuracy, stability analysis, and less computation cost compared to DL models. Thus, the relatively simple CBR-PL model is highly recommended as a promising tool for predicting daily ETo in coastal regions worldwide which have limited climate data.

Exploiting Cross-Modal Prediction and Relation Consistency for Semisupervised Image Captioning.

The task of image captioning aims to generate captions directly from images via the automatically learned cross-modal generator. To build a well-performing generator, existing approaches usually need a large number of described images (i.e., supervised image-sentence pairs), requiring a huge effects on manual labeling. However, in real-world applications, a more general scenario is that we only have limited amount of described images and a large number of undescribed images. Therefore, a resulting challenge is how to effectively combine the undescribed images into the learning of cross-modal generator (i.e., semisupervised image captioning). To solve this problem, we propose a novel image captioning method by exploiting the cross-modal prediction and relation consistency (CPRC), which aims to utilize the raw image input to constrain the generated sentence in the semantic space. In detail, considering that the heterogeneous gap between modalities always leads to the supervision difficulty while using the global embedding directly, CPRC turns to transform both the raw image and corresponding generated sentence into the shared semantic space, and measure the generated sentence from two aspects: 1) prediction consistency: CPRC utilizes the prediction of raw image as soft label to distill useful supervision for the generated sentence, rather than employing the traditional pseudo labeling and 2) relation consistency: CPRC develops a novel relation consistency between augmented images and corresponding generated sentences to retain the important relational knowledge. In result, CPRC supervises the generated sentence from both the informativeness and representativeness perspectives, and can reasonably use the undescribed images to learn a more effective generator under the semisupervised scenario. The experiments show that our method outperforms state-of-the-art comparison methods on the MS-COCO "Karpathy" offline test split under complex nonparallel scenarios, for example, CPRC achieves at least 6% improvements on the CIDEr-D score.

Rapid: Zero-shot Domain Adaptation for Code Search with Pre-trained Models

Code search, which refers to the process of identifying the most relevant code snippets for a given natural language query, plays a crucial role in software maintenance. However, current approaches heavily rely on labeled data for training, which results in performance decreases when confronted with cross-domain scenarios including domain-specific or project-specific situations. This decline can be attributed to their limited ability to effectively capture the semantics associated with such scenarios. To tackle the aforementioned problem, we propose a ze R o-shot dom A in ada P tion with pre-tra I ned mo D els framework for code search named RAPID. The framework first generates synthetic data by pseudo labeling, then trains the CodeBERT with sampled synthetic data. To avoid the influence of noisy synthetic data and enhance the model performance, we propose a mixture sampling strategy to obtain hard negative samples during training. Specifically, the mixture sampling strategy considers both relevancy and diversity to select the data that are hard to be distinguished by the models. To validate the effectiveness of our approach in zero-shot settings, we conduct extensive experiments and find that RAPID outperforms the CoCoSoDa and UniXcoder model by an average of 15.7% and 10%, respectively, as measured by the MRR metric. When trained on full data, our approach results in an average improvement of 7.5% under the MRR metric using CodeBERT. We observe that as the model’s performance in zero-shot tasks improves, the impact of hard negatives diminishes. Our observation also indicates that fine-tuning CodeT5 for generating pseudo labels can enhance the performance of the code search model, and using only 100-shot samples can yield comparable results to the supervised baseline. Furthermore, we evaluate the effectiveness of RAPID in real-world code search tasks in three GitHub projects through both human and automated assessments. Our findings reveal RAPID exhibits superior performance, e.g., an average improvement of 18% under the MRR metric over the top-performing model.

Exploring Feature Representation Learning for Semi-Supervised Medical Image Segmentation.

This article presents a simple yet effective two-stage framework for semi-supervised medical image segmentation. Unlike prior state-of-the-art semi-supervised segmentation methods that predominantly rely on pseudo supervision directly on predictions, such as consistency regularization and pseudo labeling, our key insight is to explore the feature representation learning with labeled and unlabeled (i.e., pseudo labeled) images to regularize a more compact and better-separated feature space, which paves the way for low-density decision boundary learning and therefore enhances the segmentation performance. A stage-adaptive contrastive learning method is proposed, containing a boundary-aware contrastive loss that takes advantage of the labeled images in the first stage, as well as a prototype-aware contrastive loss to optimize both labeled and pseudo labeled images in the second stage. To obtain more accurate prototype estimation, which plays a critical role in prototype-aware contrastive learning, we present an aleatoric uncertainty-aware method to generate higher quality pseudo labels. Aleatoric-uncertainty adaptive (AUA) adaptively regularizes prediction consistency by taking advantage of image ambiguity, which, given its significance, is underexplored by existing works. Our method achieves the best results on three public medical image segmentation benchmarks.

Self-Supervised Autoregressive Domain Adaptation for Time Series Data.

Unsupervised domain adaptation (UDA) has successfully addressed the domain shift problem for visual applications. Yet, these approaches may have limited performance for time series data due to the following reasons. First, they mainly rely on the large-scale dataset (i.e., ImageNet) for source pretraining, which is not applicable for time series data. Second, they ignore the temporal dimension on the feature space of the source and target domains during the domain alignment step. Finally, most of the prior UDA methods can only align the global features without considering the fine-grained class distribution of the target domain. To address these limitations, we propose a SeLf-supervised AutoRegressive Domain Adaptation (SLARDA) framework. In particular, we first design a self-supervised (SL) learning module that uses forecasting as an auxiliary task to improve the transferability of source features. Second, we propose a novel autoregressive domain adaptation technique that incorporates temporal dependence of both source and target features during domain alignment. Finally, we develop an ensemble teacher model to align class-wise distribution in the target domain via a confident pseudo labeling approach. Extensive experiments have been conducted on three real-world time series applications with 30 cross-domain scenarios. The results demonstrate that our proposed SLARDA method significantly outperforms the state-of-the-art approaches for time series domain adaptation. Our source code is available at: https://github.com/mohamedr002/SLARDA.