Single-source Domain Adaptation Research Articles

Single-/Multi-Source Domain Adaptation via domain separation: A simple but effective method

Domain adaptation (DA) aims to reduce knowledge gap between domains and improve the prediction ability of models in the target domain. However, the representations learned from feature extraction network often contain redundant information, which is harmful to domain alignment. In addition, many methods only focus on either single-source DA task or multi-source DA task, which limits their real-world applications. In this paper, we propose a simple but effective method called Category and Domain Features Augmentation (CDFA), which consists of two components: Contrastive Classifier Network (CCN) and Domain-specific Learning Network (DSLN). CDFA can remove the specific representation at the feature extraction stage to alleviate transfer difficulty, where CCN is used to increase the probabilistic output of samples and avoid misclassification, and DSLN facilitates the separation of redundant information from all representations by learning domain-specific representations. Empirical evaluations on several cross-domain benchmarks under single-source and multi-source DA scenarios illustrate the competitive performance of CDFA with respect to the state-of-the-art.

Knowledge correlation graph-guided multi-source interaction domain adaptation network for rotating machinery fault diagnosis

Leveraging generalized knowledge from multiple source domains with rich labels to the target domain without labeled data is a more realistic and challenging issue compared with single-source domain adaptation. Furthermore, the distribution discrepancies between each source domain and the expansion of data categories increase the difficulty of aligning each source domain with the target domain. To alleviate these issues, a knowledge correlation graph-guided multi-source interaction domain adaptation network (KCGMIDAN) is developed for rotating machinery fault diagnosis. Firstly, a random mini-batch is randomly selected to update comprehensive feature representations (CFR) extracted from each data category across all domains, thus promoting the knowledge interaction of acquired CFR between the current and the next epochs. Then, a knowledge correlation graph (KCG) is established on all CFR to boost knowledge propagation among various domains. To improve the compactness of characteristics within the same category and the separation of various categories, two losses are designed in this procedure to place constraints on the relationships between categories. Finally, query samples are added into KCG to construct the extended KCG, and the recognition of samples is completed by using built deep graph network based on the extended KCG. Extensive experimental results verify that KCGMIDAN can achieve better recognition performance than existing methods.

Globally Localized Multisource Domain Adaptation for Cross-Domain Fault Diagnosis With Category Shift.

Deep learning has demonstrated splendid performance in mechanical fault diagnosis on condition that source and target data are identically distributed. In engineering practice, however, the domain shift between source and target domains significantly limits the further application of intelligent algorithms. Despite various transfer techniques proposed, either they focus on single-source domain adaptation (SDA) or they utilize multisource domain globally or locally, which both cannot address the cross-domain diagnosis effectively, especially with category shift. To this end, we propose globally localized multisource DA for cross-domain fault diagnosis with category shift. Specifically, we construct a GlocalNet to fuse multisource information comprehensively, which consists of a feature generator and three classifiers. By optimizing the Wasserstein discrepancy of classifiers locally and accumulative higher order multisource moment globally, multisource DA is achieved from domain and class levels thus to reduce the shift on domain and category. To refine the classifier at sample level, a distilling strategy is presented. Finally, an adaptive weighting policy is employed for reliable result. To evaluate the effectiveness, the proposed method is compared with multiple methods on four bearing vibration datasets. Experimental results indicate the superiority and practicability of the proposed method for cross-domain fault diagnosis.

Collaborative fault diagnosis of rotating machinery via dual adversarial guided unsupervised multi-domain adaptation network

Most of the existing research on unsupervised cross-domain intelligent fault diagnosis is based on single-source domain adaptation, which fails to simultaneously utilize various source domains with enough and diverse diagnostic information in practical application scenarios. How to better extract common features from multiple domains and integrate multi-source domain knowledge for collaborative diagnosis is a main challenge. To address these problems, a dual adversarial guided unsupervised multi-domain adaptation network (DAG-MDAN) is proposed. Within the proposed framework, the edge adversarial module (EA-Module) in each set of sources-target domain adaptation sub-network is utilized to compute the source-target domain adversarial loss. And an inner adversarial module (IA-Module) is constructed to direct the extraction of common features between multi-source domains, which combined the EA-Module to form the dual adversarial training to enhance domain confusion. Besides, a multi-subnet collaborative decision module (MCD-Module) is designed to compute the confidence scores to assists the multi-subnet classifier to make better fusion decisions. The DAG-MDAN is verified by the several transfer tasks using faulty rotating machinery datasets under the different speed conditions.

Multi-source domain generalization and adaptation toward cross-subject myoelectric pattern recognition

Objective. Myoelectric pattern recognition (MPR) has shown satisfactory performance under ideal laboratory conditions. Nevertheless, the individual variances lead to dramatic performance degradation in cross-user MPR applications. It is crucial to enable the myoelectric interface to adapt to multiple users’ surface electromyography (sEMG) distributions in practical. Approach. Domain adaptation (DA) is a promising approach to tackle cross-user challenges due to its ability to diminish the divergence between individual users’ EMG distributions and escalate model generalization performance. However, existing DA methods in sEMG control are based on single-source domain adaptation (SDA). SDA solely mixes multiple training users’ data as a combined source domain and attempts to align with a novel user. This simple data mixing manner ignores the sEMG distribution variations between disparate training users, leading to an insufficient variance elimination and lower performance. To this end, this paper proposes a multi-source synchronize domain adaptation framework with both DA and domain generalization (DG) capability. This multi-source framework aligns each source user and the new user in individual feature spaces, which better transfers the knowledge of existing users to the new user. Moreover, we retain the source-combined data to preserve the effectiveness of SDA. The property was further confirmed by evaluating the performance of the proposed method on data from nine subjects performing six tasks. Main results. Experiment results prove that the proposed multi-source framework achieved both positive DG and DA performance in a cross-user classification manner. Significance. This work demonstrates the usability and feasibility of the proposed multi-source framework in cross-user myoelectric control.

Transfer Learning for Remaining Useful Life Prediction Across Operating Conditions Based on Multisource Domain Adaptation

Nowadays, transfer learning and domain adaptation are widely used in prognostics and health management of rotating machinery, greatly broadening its applications in scenarios with multiple operating conditions. However, almost all existing prognostic methods employ a single-source domain adaptation (SDA), ignoring the domain-shift within the source domain, and fail to make full use of multi-source historical data. To overcome these drawbacks, a multisource domain adaptation network (MDAN) for regression tasks is proposed in this article. MDAN learns domain-invariant features and supervision from multiple sources and provides better generalization in the target domain than traditional SDA methods. Besides, a new MDAN-based framework is presented for remaining useful life prediction of bearings under multiple operating conditions. Specifically, we input the time-frequency representation of the vibration signals into MDAN to achieve end-to-end training and inference across operating conditions. Case studies on the run-to-failure datasets validate the effectiveness of proposed method, and the comparison shows that it outperforms the state-of-the-art methods.

Multiperspective Progressive Structure Adaptation for JPEG Steganography Detection Across Domains.

The aim of steganography detection is to identify whether the multimedia data contain hidden information. Although many detection algorithms have been presented to solve tasks with inconsistent distributions between the source and target domains, effectively exploiting transferable correlation information across domains remains challenging. As a solution, we present a novel multiperspective progressive structure adaptation (MPSA) scheme based on active progressive learning (APL) for JPEG steganography detection across domains. First, the source and target data originating from unprocessed steganalysis features are clustered together to explore the structures in different domains, where the intradomain and interdomain structures can be captured to provide adequate information for cross-domain steganography detection. Second, the structure vectors containing the global and local modalities are exploited to reduce nonlinear distribution discrepancy based on APL in the latent representation space. In this way, the signal-to-noise ratio (SNR) of a weak stego signal can be improved by selecting suitable objects and adjusting the learning sequence. Third, the structure adaptation across multiple domains is achieved by the constraints for iterative optimization to promote the discrimination and transferability of structure knowledge. In addition, a unified framework for single-source domain adaptation (SSDA) and multiple-source domain adaptation (MSDA) in mismatched steganalysis can enhance the model's capability to avoid a potential negative transfer. Extensive experiments on various benchmark cross-domain steganography detection tasks show the superiority of the proposed approach over the state-of-the-art methods.

Multi-source domain adaptation of social media data for disaster management.

Labeled data scarcity at the time of an ongoing disaster has encouraged the researchers to use the labeled data from some previous disaster for training and transferring the knowledge to the current disaster task using Domain Adaptation (DA). However, often labeled data from more than one previous disaster may be available. As all deep learning models are data-hungry and perform better if fed with more annotated data, it is advisable to use data from multiple sources for training a Deep Convolutional Neural Network (DCNN). One of the easiest ways is to simply combine the data from multiple sources and use it for training. However, this arrangement is not that straightforward. The models trained on the combined data from various sources do not perform well on the target, mainly due to distribution discrepancies between multiple sources. This has motivated us to explore the challenging area of multi-source domain adaptation for disaster management. The aim is to learn the domain invariant features and representations across the domains and transfer more related knowledge to solve the target task with improved accuracy than single-source or combined-source domain adaptation. This study proposes a Multi-Source Domain Adaptation framework for Disaster Management (MSDA-DM) to classify disaster images posted on social media based on unsupervised DA with adversarial training. The empirical results obtained confirm that the proposed model MSDA-DM performs better than single-source DA by up to 10.83% and combined-source DA by up to 5.06% in terms of F1-score for different sets of source and target disaster domains. We also compare our model with current state-of-the-art models. The main challenge of multi-source DA is the choice of the relevant sources taken for training since, unlike single-source DA that handles only source-target distribution drift, the multi-source DA network has to address both source-target and source-source distribution drifts.

Multisource domain factorization network for cross-domain fault diagnosis of rotating machinery: An unsupervised multisource domain adaptation method

Unsupervised domain adaptation (DA) provides a promising approach for tackling fault diagnosis tasks of target datasets without labeled data and has been actively studied in recent years. Most of them focus only on single-source DA, compared to multisource DA (MDA), which has remarkable advantages in generalized knowledge learning and generalization performance. Nevertheless, there are very few fault diagnosis studies based on MDA, and it remains challenging to reduce multiple domain shifts to improve diagnostic performance and mitigate negative transfer during learning. To this end, a novel unsupervised MDA-based transfer learning approach called multisource domain factorization network (MDFN) is proposed in this paper, where the generalized diagnosis knowledge is learned from multiple sources and then used for diagnosing the target task. The highlights of MDFN are that the shared-space component analysis and transferability-based entropy penalty strategy are employed to significantly mitigate negative transfer from the two levels of feature representation and instance transferability and effectively learn shared feature representation. Therefore, the MDFN can extract shared features that combine domain-invariance and discriminability, thereby performing better. The results of two experimental cases on six datasets, including cross-operating-condition and cross-component diagnosis tasks, validate the effectiveness and superiority of the proposed method.

Fault Diagnosis of Rolling Bearings of Different Working Conditions Based on Multi-Feature Spatial Domain Adaptation

The running state of rolling bearings is complex in operation, and the data are generally collected under different working conditions. However, when single-source domain adaptation is used to model the heterogeneously distributed data obtained under different working conditions, the domain-invariant representations can hardly be used for representation, which directly affects the fault diagnosis rate. To this end, a method for the fault diagnosis of rolling bearings under different working conditions based on multi-feature spatial domain adaptation is proposed. Firstly, all the data from source and target domains are mapped into a feature space to learn the common representations of all domains. Secondly, the data for each pair of source and target domains are mapped into different feature spaces to get the fault feature representations under various working conditions. And the multi-domain adaptation network is used for the domain-specific distribution alignment to learn multiple domain-invariant representations. Thirdly, these representations are used to train multiple domain-specific classifiers, thus obtaining the recognition result for each domain-invariant representation. Finally, the domain-specific decision boundaries predicted by multiple classifiers are employed to align the classifiers’ output of target samples and thus to reduce the influence from different classifiers. The effectiveness and feasibility of this proposed method have been verified by diagnostic experiments conducted according to the rolling bearing data from Case Western Reserve University and Laboratory, respectively.

Attention-Based Multi-Source Domain Adaptation

Multi-source domain adaptation (MSDA) aims to transfer knowledge from multi-source domains to one target domain. Inspired by single-source domain adaptation, existing methods solve MSDA by aligning the data distributions between the target domain and each source domain. However, aligning the target domain with the dissimilar source domain would harm the representation learning. To address the above issue, an intuitive motivation of MSDA is using the attention mechanism to enhance the positive effects of the similar domains, and suppress the negative effects of the dissimilar domains. Therefore, we propose Attention-Based Multi-Source Domain Adaptation (ABMSDA) by considering the domain correlations to alleviate the effects caused by dissimilar domains. To obtain the domain correlations between source and target domains, ABMSDA firstly trains a domain recognition model to calculate the probability that the target images belong to each source domain. Based on the domain correlations, Weighted Moment Distance (WMD) is proposed to pay more attention on the source domains with higher similarities. Furthermore, Attentive Classification Loss (ACL) is developed to constrain that the feature extractor can generate the alignment and discriminative visual representations. The evaluations on two benchmarks demonstrate the effectiveness of the proposed model, e.g., an average of 6.1% improvement on the challenging DomainNet dataset.

Adversarial domain adaptation with classifier alignment for cross-domain intelligent fault diagnosis of multiple source domains

Recently, most cross-domain fault diagnosis methods focus on single source domain adaptation. However, it is usually possible to obtain multiple labeled source domains in real industrial scenarios. The question of how to use multiple source domains to extract common domain-invariant features and obtain satisfactory diagnosis results is a difficult one. This paper proposes a novel adversarial domain adaptation with a classifier alignment method (ADACL) to address the issue of multiple source domain adaptation. The main elements of ADACL consist of a universal feature extractor, multiple classifiers and a domain discriminator. The parameters of the main elements are simultaneously updated via a cross-entropy loss, a domain distribution alignment loss and a domain classifier alignment loss. Under the framework of multiple loss cooperative learning, not only is the distribution discrepancy among all domains minimized, but so is the prediction discrepancy of target domain data among all classifiers. Two experimental cases on two source domains and three source domains verify that the ADACL can remarkably enhance the cross-domain diagnostic performance under diverse operating conditions. In addition, the diagnostic performance of different methods is extensively evaluated under noisy environments with a different signal-to-noise ratio.

Multi-Source Distilling Domain Adaptation

Deep neural networks suffer from performance decay when there is domain shift between the labeled source domain and unlabeled target domain, which motivates the research on domain adaptation (DA). Conventional DA methods usually assume that the labeled data is sampled from a single source distribution. However, in practice, labeled data may be collected from multiple sources, while naive application of the single-source DA algorithms may lead to suboptimal solutions. In this paper, we propose a novel multi-source distilling domain adaptation (MDDA) network, which not only considers the different distances among multiple sources and the target, but also investigates the different similarities of the source samples to the target ones. Specifically, the proposed MDDA includes four stages: (1) pre-train the source classifiers separately using the training data from each source; (2) adversarially map the target into the feature space of each source respectively by minimizing the empirical Wasserstein distance between source and target; (3) select the source training samples that are closer to the target to fine-tune the source classifiers; and (4) classify each encoded target feature by corresponding source classifier, and aggregate different predictions using respective domain weight, which corresponds to the discrepancy between each source and target. Extensive experiments are conducted on public DA benchmarks, and the results demonstrate that the proposed MDDA significantly outperforms the state-of-the-art approaches. Our source code is released at: https://github.com/daoyuan98/MDDA.