Rethinking Domain Adaptation and Generalization in the ERA Of Clip

Objective. Myoelectric pattern recognition (MPR) has shown satisfactory performance under ideal laboratory conditions. Nevertheless, the individual variances lead to dramatic performance degradation in cross-user MPR applications. It is crucial to enable the myoelectric interface to adapt to multiple users’ surface electromyography (sEMG) distributions in practical. Approach. Domain adaptation (DA) is a promising approach to tackle cross-user challenges due to its ability to diminish the divergence between individual users’ EMG distributions and escalate model generalization performance. However, existing DA methods in sEMG control are based on single-source domain adaptation (SDA). SDA solely mixes multiple training users’ data as a combined source domain and attempts to align with a novel user. This simple data mixing manner ignores the sEMG distribution variations between disparate training users, leading to an insufficient variance elimination and lower performance. To this end, this paper proposes a multi-source synchronize domain adaptation framework with both DA and domain generalization (DG) capability. This multi-source framework aligns each source user and the new user in individual feature spaces, which better transfers the knowledge of existing users to the new user. Moreover, we retain the source-combined data to preserve the effectiveness of SDA. The property was further confirmed by evaluating the performance of the proposed method on data from nine subjects performing six tasks. Main results. Experiment results prove that the proposed multi-source framework achieved both positive DG and DA performance in a cross-user classification manner. Significance. This work demonstrates the usability and feasibility of the proposed multi-source framework in cross-user myoelectric control.

Unsupervised domain adaptation (UDA) aims to mitigate the performance drop due to the distribution shift between the training and testing datasets. UDA methods have achieved performance gains for models trained on a source domain with labeled data to a target domain with only unlabeled data. The standard feature extraction method in domain adaptation has been convolutional neural networks (CNNs). Recently, attention-based transformer models have emerged as effective alternatives for computer vision tasks. In this paper, we benchmark three attention-based architectures, specifically vision transformer (ViT), shifted window transformer (SWIN), and dual attention vision transformer (DAViT), against convolutional architectures ResNet, HRNet and attention-based ConvNext, to assess the performance of different backbones for domain generalization and adaptation. We incorporate these backbone architectures as feature extractors in the source hypothesis transfer (SHOT) framework for UDA. SHOT leverages the knowledge learned in the source domain to align the image features of unlabeled target data in the absence of source domain data, using self-supervised deep feature clustering and self-training. We analyze the generalization and adaptation performance of these models on standard UDA datasets and aerial UDA datasets. In addition, we modernize the training procedure commonly seen in UDA tasks by adding image augmentation techniques to help models generate richer features. Our results show that ConvNext and SWIN offer the best performance, indicating that the attention mechanism is very beneficial for domain generalization and adaptation with both transformer and convolutional architectures. Our ablation study shows that our modernized training recipe, within the SHOT framework, significantly boosts performance on aerial datasets.

Traditional domain adaptation methods attempted to learn the shared representation for distribution matching between source domain and target domain where the individual information in both domains was not characterized. Such a solution suffers from the mixing problem of individual information with the shared features which considerably constrains the performance for domain adaptation. To relax this constraint, it is crucial to extract both shared information and individual information. This study captures both information via a new domain separation network where the shared features are extracted and purified via separate modeling of individual information in both domains. In particular, a hybrid adversarial learning is incorporated in a separation network as well as an adaptation network where the associated discriminators are jointly trained for domain separation and adaptation according to the minmax optimization over separation loss and domain discrepancy, respectively. Experiments on different tasks show the merit of using the proposed adversarial domain separation and adaptation.

Domain adaptation (DA) aims to generalize a learning model across training and testing data despite the mismatch of their data distributions. In light of a theoretical estimation of the upper error bound, we argue, in this article, that an effective DA method for classification should: 1) search a shared feature subspace where the source and target data are not only aligned in terms of distributions as most state-of-the-art DA methods do but also discriminative in that instances of different classes are well separated and 2) account for the geometric structure of the underlying data manifold when inferring data labels on the target domain. In comparison with a baseline DA method which only cares about data distribution alignment between source and target, we derive three different DA models for classification, namely, close yet discriminative DA (CDDA), geometry-aware DA (GA-DA), and discriminative and GA-DA (DGA-DA), to highlight the contribution of CDDA based on 1), GA-DA based on 2), and, finally, DGA-DA implementing jointly 1) and 2). Using both the synthetic and real data, we show the effectiveness of the proposed approach which consistently outperforms the state-of-the-art DA methods over 49 image classification DA tasks through eight popular benchmarks. We further carry out an in-depth analysis of the proposed DA method in quantifying the contribution of each term of our DA model and provide insights into the proposed DA methods in visualizing both real and synthetic data.

There are a variety of Domain Adaptation (DA) scenarios subject to label sets and domain configurations, including closed-set and partial-set DA, as well as multi-source and multi-target DA. It is notable that existing DA methods are generally designed only for a specific scenario, and may underperform for scenarios they are not tailored to. To this end, this paper studies Versatile Domain Adaptation (VDA), where one method can handle several different DA scenarios without any modification. Towards this goal, a more general inductive bias other than the domain alignment should be explored. We delve into a missing piece of existing methods: class confusion, the tendency that a classifier confuses the predictions between the correct and ambiguous classes for target examples, which is common in different DA scenarios. We uncover that reducing such pairwise class confusion leads to significant transfer gains. With this insight, we propose a general loss function: Minimum Class Confusion (MCC). It can be characterized as (1) a non-adversarial DA method without explicitly deploying domain alignment, enjoying faster convergence speed; (2) a versatile approach that can handle four existing scenarios: Closed-Set, Partial-Set, Multi-Source, and Multi-Target DA, outperforming the state-of-the-art methods in these scenarios, especially on one of the largest and hardest datasets to date ( $$7.3\%$$ on DomainNet). Its versatility is further justified by two scenarios proposed in this paper: Multi-Source Partial DA and Multi-Target Partial DA. In addition, it can also be used as a general regularizer that is orthogonal and complementary to a variety of existing DA methods, accelerating convergence and pushing these readily competitive methods to stronger ones. Code is available at https://github.com/thuml/Versatile-Domain-Adaptation .

Due to more abundant data sources, more various objects of interest, and more time-consuming annotations, there is a large amount of out-of-distribution (OOD) data in the remote sensing field, on which the performance of high-accuracy image segmentation models trained under ideal experimental conditions generally degrades dramatically. Domain adaptation (DA) consequently comes into being, which aims to learn the predictor for the label-scarce target domain of interest with the help of the label-sufficient source domain in the presence of the distribution difference, namely, domain shift, between the two domains. However, the off-the-shelf DA methods for image segmentation not only struggle to cope with the more complex domain shift problems in remote sensing imagery but also almost cannot process heterogeneous data directly without information loss. While the current heterogeneous DA methods mostly still rely on some supervision information from the target domain, which is typically inaccessible in the real world. To overcome these drawbacks, we propose the multilevel heterogeneous unsupervised DA (UDA) method, termed MHDA, which unifies the instance-level DA based on cycle consistency, the feature-level DA based on contrastive learning, and the decision-level DA based on task consistency into a framework to more effectively handle the complex domain shift and heterogeneous data. After that, extensive DA experiments are conducted on the International Society for Photogrammetry and Remote Sensing (ISPRS) dataset, the BigCity dataset constructed by ourselves, and the Wuhan University (WHU) dataset, to explore the effect of each module in MHDA, the necessity of heterogeneous DA, and the effectiveness of multilevel DA. And the results demonstrate that MHDA can achieve superior performance on the remote sensing image segmentation task, compared with several state-of-the-art DA methods.

Summary Introduction : There has been a rapid development of deep learning (DL) models for medical imaging. However, DL requires a large labeled dataset for training the models. Getting large-scale labeled data remains a challenge, and multi-center datasets suffer from heterogeneity due to patient diversity and varying imaging protocols. Domain adaptation (DA) has been developed to transfer the knowledge from a labeled data domain to a related but unlabeled domain in either image space or feature space. DA is a type of transfer learning (TL) that can improve the performance of models when applied to multiple different datasets. Objective : In this survey, we review the state-of-the-art DL-based DA methods for medical imaging. We aim to summarize recent advances, highlighting the motivation, challenges, and opportunities, and to discuss promising directions for future work in DA for medical imaging. Methods : We surveyed peer-reviewed publications from leading biomedical journals and conferences between 2017-2020, that reported the use of DA in medical imaging applications, grouping them by methodology, image modality, and learning scenarios. Results : We mainly focused on pathology and radiology as application areas. Among various DA approaches, we discussed domain transformation (DT) and latent feature-space transformation (LFST). We highlighted the role of unsupervised DA in image segmentation and described opportunities for future development. Conclusion : DA has emerged as a promising solution to deal with the lack of annotated training data. Using adversarial techniques, unsupervised DA has achieved good performance, especially for segmentation tasks. Opportunities include domain transferability, multi-modal DA, and applications that benefit from synthetic data.

In leveraging manifold learning in domain adaptation (DA), graph embedding-based DA methods have shown their effectiveness in preserving data manifold through the Laplace graph. However, current graph embedding DA methods suffer from two issues: 1). they are only concerned with preservation of the underlying data structures in the embedding and ignore sub-domain adaptation, which requires taking into account intra-class similarity and inter-class dissimilarity, thereby leading to negative transfer; 2). manifold learning is proposed across different feature/label spaces separately, thereby hindering unified comprehensive manifold learning. In this paper, starting from our previous DGA-DA, we propose a novel DA method, namely A ttention R egularized Laplace G raph-based D omain A daptation (ARG-DA), to remedy the aforementioned issues. Specifically, by weighting the importance across different sub-domain adaptation tasks, we propose the A ttention R egularized Laplace Graph for class aware DA, thereby generating the attention regularized DA. Furthermore, using a specifically designed FEEL strategy, our approach dynamically unifies alignment of the manifold structures across different feature/label spaces, thus leading to comprehensive manifold learning. Comprehensive experiments are carried out to verify the effectiveness of the proposed DA method, which consistently outperforms the state of the art DA methods on 7 standard DA benchmarks, i.e., 37 cross-domain image classification tasks including object, face, and digit images. An in-depth analysis of the proposed DA method is also discussed, including sensitivity, convergence, and robustness.

Developing personalized gait phase prediction models is difficult because acquiring accurate gait phases requires expensive experiments. This problem can be addressed via semi-supervised domain adaptation (DA), which minimizes the discrepancy between the source and target subject features. However, classical DA models have a trade-off between accuracy and inference speed. Whereas deep DA models provide accurate prediction results with a slow inference speed, shallow DA models produce less accurate results with a fast inference speed. To achieve both high accuracy and fast inference, a dual-stage DA framework is proposed in this study. The first stage uses a deep network for precise DA. Then, a pseudo-gait-phase label of the target subject is obtained using the first-stage model. In the second stage, a shallow but fast network is trained using the pseudo-label. Because computation for DA is not conducted in the second stage, an accurate prediction can be accomplished even with the shallow network. Test results show that the proposed DA framework reduces the prediction error by 1.04% compared with a shallow DA model while maintaining its fast inference speed. The proposed DA framework can be used to provide fast personalized gait prediction models for real-time control systems such as wearable robots.

Deep models must learn robust and transferable representations in order to perform well on new domains. While domain transfer methods (e.g., domain adaptation, domain generalization) have been proposed to learn transferable representations across domains, they are typically applied to ResNet backbones pre-trained on ImageNet. Thus, existing works pay little attention to the effects of pre-training on domain transfer tasks. In this paper, we provide a broad study and in-depth analysis of pre-training for domain adaptation and generalization, namely: network architectures, size, pre-training loss, and datasets. We observe that simply using a state-of-the-art backbone outperforms existing state-of-the-art domain adaptation baselines and set new baselines on Office-Home and DomainNet improving by 10.7% and 5.5%. We hope that this work can provide more insights for future domain transfer research.KeywordsTransfer learningPre-trainingDomain generalizationDomain adaptation

A universal transfer network for machinery fault diagnosis

Background. Spinal muscular atrophy 5q (SMA) is a severe genetic neuromuscular disorder, which is primarily manifested through musclar weakness. Previously, cognitive development in the natural course of SMA was considered normal. The introduction of etiopathogenetic therapy has altered the disease trajectory, led to new phenotypes, improved survival rates, and outlined the importance of studying the development of emotional, cognitive, and communicative domains, and adaptive behavior in SMA patients.Aim. To conduct a comprehensive assessment of emotional, cognitive, and adaptive domains, as well as speech development, in patients with genetically confirmed SMA, including cases, which were identified through newborn screening programs and were asymptomatic at the initiation of etiopathogenetic therapy, and to identify factors influencing neuropsychic development in SMA patients.Materials and methods. The study included 87 SMA patients receiving etiopathogenetic therapy, aged 0–12 years (median age at testing – 57.0 [37.0; 103.0] months). The Developmental Profile-3 (DP-3) instrument was used to assess neuropsychic development. Statistical analysis was performed using SPSS Statistics v.26.0 (IBM, USA).Results. Children who received therapy at the presymptomatic stage (6.9 % of the cohort) showed no deficits in any assessed developmental domains. These results significantly differed from those of SMA types 1, 2, and 3 in motor skills (padj <0.001) and adaptive behavior (padj ≤0.026). Patients with SMA types 1, 2, and 3 exhibited severe motor impairments (reduced motor skills in 93.0 %, 89.7 %, and 88.9 % of children, respectively) and adaptive deficits (impairments in ≥55 % of each group). SMA type 1 patients additionally demonstrated delays in social emotional (39.5 %), cognitive (30.2 %), and communicative (39.5 %) domains. Children with lower functional status (“lying”) had more pronounced delays in adaptive, social emotional, and cognitive domains (p ≤0.048). In SMA type 1, fewer SMN2 gene copies and earlier disease onset correlated with more severe deficits in emotional, cognitive, and adaptive domains, as well as in speech development (SMN2 copies: p ≤0.034; age of onset: p ≤0.012). SMA type 1 patients with dysphagia showed lower scores across all subscales except motor skills (p ≤0.015). Chronic respiratory insufficiency was associated with reduced scores in all five subscales: in SMA type 1, motor skills, adaptive, social emotional, and cognitive domains were affected (p ≤0.045); in SMA type 2, adaptive, social emotional, and cognitive domains were affected (p ≤0.018). Delayed therapy initiation correlated with lower motor and adaptive scores in SMA types 1 (p ≤0.012), 2 (p ≤0.002), and 3 (p ≤0.048), and with worse social emotional and cognitive outcomes in SMA type 2 (p = 0.001).Conclusion. SMA patients exhibit not only motor impairments but also adaptive and socialization deficits, as well as delays in communicative and cognitive development. A standardized approach to identifying these impairments should be developed, and developing tailored rehabilitation methods is important as well. Initiating etiopathogenetic therapy at the presymptomatic stage may prevent neuropsychiatric manifestations of SMA.

Unsupervised domain adaptation methods utilize feature re-presentations of instances in the source and target domains to eliminate domain shifts. It is worth noting that the instance features are closely related to the entire distribution of the domain, and the current information after adaptation in the execution of the domain adaptation task is closely related to the original features. Common methods are based on only one of these pieces of information and do not make sufficient use of them. We develop the Self-Reinforcing Feedback Domain Adaptation Channel (SRFC). Pioneeringly, on the feature representation of the network, SRFC fuses global and instance information simultaneously, and utilizes the past history and current information in domain adaptation, so that the information can be effectively enhanced to better complete the domain adaptation. Through the designed self-reinforcing feedback mechanism, SRFC skillfully integrates multi-level information in a robust way in the process of domain adaptation, and actively enhances the availability and comprehensive value of features in domain adaptation with manageable continuous feedback. Experiments on benchmark datasets verify the advantages of SRFC fusion information for instance feature enhancement and domain adaptation. The modular Self-Reinforcing Feedback Domain Adaptation Channel has scalability and R &D potential, and we hope that it can be extended to more domain adaptation networks using enhanced instance representations to better accomplish different tasks.

Unsupervised Domain Adaption (DA) is an approach for adapting a data-driven model to new data without labels. Recent work on Remaining Useful Lifetime (RUL) estimation of aero engines yielded promising results for this approach. However, the current evaluation framework for DA is of limited significance when used for RUL estimation. It assumes a use case where a large number of fully degraded systems are available for adaption, which makes unsupervised DA in itself unnecessary. It is shown that the current framework overestimates adaption performance and obscures potential, negative effects of DA on performance. We propose a novel evaluation framework for unsupervised DA, specialized in RUL estimation, that takes the number of available systems and their grade of degradation into account. It enables an informed performance comparison of DA methods. We detail the framework’s capabilities on two DA methods and show that unsupervised DA delivers improved RUL estimations under real-life scenarios, as well.

Domain adaptation methods have demonstrated considerable potential in enhancing the performance of fault diagnosis models across diverse operating conditions. While single-target domain adaptation has been extensively explored, its efficacy is reduced in scenarios involving multi-target and incremental-target domains. However, such scenarios are common in real-world projects, necessitating the exploration of more sophisticated, multi-domain, and incremental domain adaptation methods. In response, we propose an iterative-cluster domain adaptation (ICDA) method, which is tailored to address incremental multi-target domain adaptation challenges. On the basis of single-target domain adaptation, ICDA enhances domain alignment through cyclic iteration and clustering algorithms. Notably, it has been demonstrated to be effective in a range of domain adaptation scenarios, including single-target, multi-target, and incremental-target adaptation, even in the unlabeled target domains. The efficacy and superiority of ICDA over baseline models has been substantiated through validation on the CWRU and WINDEY datasets.