Partial Domain Adaptation Research Articles

Multisource partial domain adaptation for bearing fault diagnosis

Abstract Domain adaptation has been widely used in fault diagnosis and dealt with data distribution discrepancies, but the labels in source and target domains are usually assumed to be identical. The complexity of the working conditions, in reality, leads to the fact that the labels in target domains are often a subset of the labels in source domains. This special case is called the partial domain problem. However, most of the existing proposed methods for solving partial domain problems are limited to single-source-domain scenarios and fail to effectively integrate multisource knowledge. Hence, this study proposes a new approach of multisource domain obfuscation-subdomain alignment (MSDO-SA) for partial domain adaptation fault diagnosis in multisource domains. Through domain obfuscation, the multisource domains are converted into a single source domain. The subdomain alignment aims at improving the generalization ability and relevance of the model, and effectively alleviates the domain shift problem. Finally, multiple partial domain fault diagnosis tasks using the CWRU dataset validate the effectiveness, robustness, and superiority of the proposed method.

Joint weight optimization for partial domain adaptation via kernel statistical distance estimation

The goal of Partial Domain Adaptation (PDA) is to transfer a neural network from a source domain (joint source distribution) to a distinct target domain (joint target distribution), where the source label space subsumes the target label space. To address the PDA problem, existing works have proposed to learn the marginal source weights to match the weighted marginal source distribution to the marginal target distribution. However, this is sub-optimal, since the neural network’s target performance is concerned with the joint distribution disparity, not the marginal distribution disparity. In this paper, we propose a Joint Weight Optimization (JWO) approach that optimizes the joint source weights to match the weighted joint source distribution to the joint target distribution in the neural network’s feature space. To measure the joint distribution disparity, we exploit two statistical distances: the distribution-difference-based L2-distance and the distribution-ratio-based χ2-divergence. Since these two distances are unknown in practice, we propose a Kernel Statistical Distance Estimation (KSDE) method to estimate them from the weighted source data and the target data. Our KSDE method explicitly expresses the two estimated statistical distances as functions of the joint source weights. Therefore, we can optimize the joint weights to minimize the estimated distance functions and reduce the joint distribution disparity. Finally, we achieve the PDA goal by training the neural network on the weighted source data. Experiments on several popular datasets are conducted to demonstrate the effectiveness of our approach. Intro video and Pytorch code are available at https://github.com/sentaochen/Joint-Weight-Optimation. Interested readers can also visit https://github.com/sentaochen for more source codes of the related domain adaptation, multi-source domain adaptation, and domain generalization approaches.

Attention guided partial domain adaptation for interpretable transfer diagnosis of rotating machinery

Domain adaptation (DA) is a well-established method to tackle the transfer diagnosis of rotating machinery. However, current DA methods encounter challenges in partial-set transfer diagnosis scenarios, where the target domain only possesses a subset of the source fault classes, due to several limitations: (1) assuming identical fault classes between the target and source domains, (2) neglecting the fine-grained differences among individual instances, and (3) lacking interpretability for transfer logic. To address these problems, this paper proposes an attention-guided partial domain adaptation (AGPDA) method for interpretable transfer diagnosis of rotating machinery. Firstly, to exclude outlier classes in the source domain, a transferability evaluator is presented to evaluate the potential benefits of source instances for domain alignment. Secondly, a novel cross-domain attention (CDA) mechanism is further developed, which leverages instance-wise attention to highlight relevant samples, enabling differentiated and effective knowledge transfer to various target samples. Finally, the CDA mechanism is implemented via a new interpretable linear cosine attention, which is specially designed to facilitate the exploration of the transfer logic. Comparison experiments demonstrate the superiority of AGPDA in both closed-set and partial-set transfer diagnosis. Furthermore, dual-perspective visualization of CDA reveals that target fault recognition primarily relies on source samples in the same class.

Gradient Alignment based Partial Domain Adaptation (GAPDA) using a domain knowledge filter for fault diagnosis of bearing

Fault diagnosis of rolling element bearings is essential to ensure the safety and reliability of industrial sites. However, changes in operating conditions can lead to variations in the distributions of the data that is collected for fault diagnosis. This, in turn, decreases the performance of deep-learning-based fault-diagnosis methods. In addition, most data in industrial settings are unlabeled, which leads to ineffectiveness of the supervised learning method. To address the issues of domain shift and unlabeled data, numerous studies have been conducted to reduce distribution discrepancies when using unlabeled data. Still, most of these studies assume that the number of labels in the training and test data are identical; this is not always true for data from industrial sites. Thus, the research outlined in this paper was pursued to address the partial domain adaptation problem, which occurs when there are fewer labels in the test data than in the training data. The proposed approach suggests two methods for applying partial domain adaptation in mechanical systems: i) a domain knowledge filter is proposed, which reflects fault characteristics in the original signal for effective feature extraction in the mechanical engineering domain, and ii) a gradient alignment module is defined to align the gradient of the statistical loss function. The method proposed herein was validated using two open-source datasets; the approach demonstrated high performance and low uncertainty, as compared to other prior methods. Additionally, physical analysis of the domain knowledge filter was conducted in this work.

A temporal partial domain adaptation network for transferable prognostics across working conditions with insufficient data

Variations in equipment working conditions cause distribution drifts in condition monitoring data, thus impeding the accuracy of remaining useful life (RUL) prediction. Although numerous transfer learning methods have been developed for cross-domain prognostics, it is still challenging to collect enough target domain data to meet training demands because of economic and safety concerns. Therefore, a temporal partial domain adaptation network (TPDAN) is proposed for transferable prognostics across working conditions without run-to-failure data in the target domain. Firstly, the domain discrepancy is appropriately measured by a stage-weighted maximum mean discrepancy (SW-MMD), allowing for the inconsistency of the RUL distribution of available data between domains. Secondly, the ranking-based feature regularization is developed to align the relative position of samples in the feature space and the label space. It facilitates the exploitation of temporal information inherent in the samples and restrains the destruction of the degradation properties of features. By performing domain adaptation based on the above tools, domain-invariant and degradation-aware features can be extracted by the TPDAN to provide reliable knowledge transfer across working conditions. The effectiveness of TPDAN has been validated, and experimental results demonstrate the compelling transferable RUL prediction performance even when run-to-failure data is unavailable in the target domain.

Partial Domain Adaptation in Remaining Useful Life Prediction With Incomplete Target Data

Partial Domain Adaptation in Remaining Useful Life Prediction With Incomplete Target Data

Maximum likelihood weight estimation for partial domain adaptation

Partial Domain Adaptation (PDA) aims to generalize a classification model from a labeled source domain to an unlabeled target domain, where the source label space contains the target label space. There are two main challenges in PDA that weaken the model's classification performance in the target domain: (i) the joint distribution of the source domain is related but different from that of the target domain, and (ii) the source outlier data, whose labels do not belong to the target label space, have a negative impact on learning the target classification model. To tackle these challenges, we propose a Maximum Likelihood Weight Estimation (MLWE) approach to estimate a weight function for the source domain. The weight function matches the joint source distribution of the relevant part to the joint target distribution, and reduces the negative impact of the source outlier data. To be specific, our approach estimates the weight function by maximizing a likelihood function, and the estimation leads to a nice convex optimization problem that has a global optimal solution. In the experiments, our approach demonstrates superior performance on popular benchmark datasets. Intro video and PyTorch code are available at https://github.com/sentaochen/Maximum-Likelihood-Weight-Estimation.

An auto-regulated universal domain adaptation network for uncertain diagnostic scenarios of rotating machinery

In recent years, domain adaptation techniques have garnered significant attention in the field of intelligent fault diagnosis for mechanical equipment. Domain adaptation techniques can effectively enable the reuse of diagnostic knowledge, thereby improving the generalization performance of the trained model. However, many intelligent diagnostic models are tailored to specific diagnostic scenarios, including closed-set domain adaptation, partial domain adaptation, and open-set domain adaptation. When these models built for specific scenarios are applied to other scenarios, their diagnostic performance will deteriorate catastrophically. This study proposes an auto-regulated universal domain adaptation network (AUDAN) for uncertain diagnostic scenarios of rotating machinery, which can handle multiple diagnostic scenarios and adaptively set threshold boundaries for each fault category based on data distribution structure to distinguish known and unknown samples. Specifically, a novel integration loss function is designed to train the proposed AUDAN. The self-adaptive reweight module is introduced into the proposed method to distribute the relative weight for each loss function automatically. Experiments on two rotating machinery datasets are carried out for validations. The experimental results demonstrate that the proposed AUDAN performs effectively in uncertain diagnostic scenarios and is promising in addressing the fault classification tasks in real industries.

A dual-weighted adversarial network for partial domain fault diagnosis of machinery

Domain adaptation provides a promising approach to cross-domain fault diagnosis of rotating machinery. While many current methods focus on scenarios where the source and target domains share identical label spaces, a prevalent situation in industrial production involves the target domain being a subset of the source domain, known as partial domain adaptation (PDA). The main challenge in PDA is the label mismatches caused by outlier classes, making the alignment between domains particularly difficult. To this end, a dual-weighted adversarial network is proposed in this paper. Specifically, a bilateral class weighting strategy (BCWS) is developed, which can effectively suppress negative migration at the decision boundary while improving the robustness of adversarial training by applying bilateral weighting to both the source and target domains. Moreover, a collaborative framework is developed to facilitate positive migration. The constructed global perceptual module (GPM) is highly correlated with the improved adversarial loss function, which adaptively adjusts the input feature map according to the accuracy of the target domain classification and pays more attention to the domain-invariant features. Experimental results for two cases, namely across different operating conditions and across different rotating components, verify the effectiveness and superiority of the proposed method for the PDA problem.

Small is Beautiful: Compressing Deep Neural Networks for Partial Domain Adaptation.

Domain adaptation is a promising way to ease the costly data labeling process in the era of deep learning (DL). A practical situation is partial domain adaptation (PDA), where the label space of the target domain is a subset of that in the source domain. Although existing methods yield appealing performance in PDA tasks, it is highly presumable that computation overhead exists in deep PDA models since the target is only a subtask of the original problem. In this work, PDA and model compression are seamlessly integrated into a unified training process. The cross-domain distribution divergence is reduced by minimizing a soft-weighted maximum mean discrepancy (SWMMD), which is differentiable and functions as regularization during network training. We use gradient statistics to compress the overparameterized model to identify and prune redundant channels based on the corresponding scaling factors in batch normalization (BN) layers. The experimental results demonstrate that our method can achieve comparable classification performance to state-of-the-art methods on various PDA tasks, with a significant reduction in model size and computation overhead.

Addressing the Overfitting in Partial Domain Adaptation With Self-Training and Contrastive Learning

Addressing the Overfitting in Partial Domain Adaptation With Self-Training and Contrastive Learning

Damage localisation using disparate damage states via domain adaptation

Abstract A significant challenge of structural health monitoring (SHM) is the lack of labeled data collected from damage states. Consequently, the collected data can be incomplete, making it difficult to undertake machine learning tasks, to detect or predict the full range of damage states a structure may experience. Transfer learning is a helpful solution, where data from (source) structures containing damage labels can be used to transfer knowledge to (target) structures, for which damage labels do not exist. Machine learning models are then developed that generalize to the target structure. In practical applications, it is unlikely that the source and the target structures contain the same damage states or experience the same environmental and operational conditions, which can significantly impact the collected data. This is the first study to explore the possibility of transfer learning for damage localisation in SHM when the damage states and the environmental variations in the source and target datasets are disparate. Specifically, using several domain adaptation methods, this article localizes severe damage states at a target structure, using labeled information from minor damage states at a source structure. By minimizing the distance between the marginal and conditional distributions between the source and the target structures, this article successfully localizes damage states of disparate severities, under varying environmental and operational conditions. The effect of partial and universal domain adaptation—where the number of damage states in the source and target datasets differ—is also explored in order to mimic realistic industrial applications of these methods.

Manifold Embedded Ensemble Partial Domain Adaptation: A Novel Partial-Set Transfer Mechanism for Gearboxes Fault Diagnosis

Manifold Embedded Ensemble Partial Domain Adaptation: A Novel Partial-Set Transfer Mechanism for Gearboxes Fault Diagnosis

Learning to Discover Knowledge: A Weakly-Supervised Partial Domain Adaptation Approach.

Domain adaptation has shown appealing performance by leveraging knowledge from a source domain with rich annotations. However, for a specific target task, it is cumbersome to collect related and high-quality source domains. In real-world scenarios, large-scale datasets corrupted with noisy labels are easy to collect, stimulating a great demand for automatic recognition in a generalized setting, i.e., weakly-supervised partial domain adaptation (WS-PDA), which transfers a classifier from a large source domain with noises in labels to a small unlabeled target domain. As such, the key issues of WS-PDA are: 1) how to sufficiently discover the knowledge from the noisy labeled source domain and the unlabeled target domain, and 2) how to successfully adapt the knowledge across domains. In this paper, we propose a simple yet effective domain adaptation approach, termed as self-paced transfer classifier learning (SP-TCL), to address the above issues, which could be regarded as a well-performing baseline for several generalized domain adaptation tasks. The proposed model is established upon the self-paced learning scheme, seeking a preferable classifier for the target domain. Specifically, SP-TCL learns to discover faithful knowledge via a carefully designed prudent loss function and simultaneously adapts the learned knowledge to the target domain by iteratively excluding source examples from training under the self-paced fashion. Extensive evaluations on several benchmark datasets demonstrate that SP-TCL significantly outperforms state-of-the-art approaches on several generalized domain adaptation tasks. Code is available at https://github.com/mc-lan/SP-TCL.

Principal Properties Attention Matching for Partial Domain Adaptation in Fault Diagnosis

Principal Properties Attention Matching for Partial Domain Adaptation in Fault Diagnosis

Dual Structural Consistent Partial Domain Adaptation Network for Intelligent Machinery Fault Diagnosis

Dual Structural Consistent Partial Domain Adaptation Network for Intelligent Machinery Fault Diagnosis

Reinforced Reweighting for Self-supervised Partial Domain Adaptation

Reinforced Reweighting for Self-supervised Partial Domain Adaptation

Partial Domain Adaptation for Building Borehole Lithology Model under Weaker Geological Prior

Partial Domain Adaptation for Building Borehole Lithology Model under Weaker Geological Prior

WCAL: Weighted and center-aware adaptation learning for partial domain adaptation

Partial domain adaptation, which aims to transfer knowledge from a source domain with rich labels to a unlabeled target domain where target class space is a subspace of source class space, is a challenging task in pattern recognition. Previous partial domain adaptation approaches tend to immerse in filtering out anomaly categories by weighting and the importance of ignoring the transferability of generated features. In the light of this, this article proposes a novel partial domain adaptation method, dubbed Weighted and Center-aware Adaptation Learning (WCAL). Specifically, WCAL presents a weighted adversarial learning module learns a category classifier to filter out the outlier categories from source domain. Also, it seeks a domain discriminator for cross-domain to further address the negative transfer. Then, the Center-aware adaptation learning module minimizes the distribution discrepancy across domains, which makes the features more transferability for the adaptation model. Extensive experiments on popular domain adaptation datasets testify that the proposed WCAL approach exceeds state-of-the-art baselines significantly with a large margin, in terms of average classification result, for example, 3.36% and 1.71% on Office-Home and Office-Caltech, respectively.

General feature spatial location and distance-based unknown Detection: A universal domain adaptation fault diagnosis framework of rotating Machinery

Recently, domain adaptation (DA) based cross-domain fault diagnosis of rotating machinery has gained much attention, where a target domain with only unlabeled data can be diagnosed by transferring diagnosis knowledge from a labeled source domain. In industrial applications, the target domain may occur only limited types of faults contained in the source domain, or even novel faults unseen in the source domain. Moreover, since the target domain is unlabeled in reality, the prior label space relationship between two domains is hard to know. Therefore, not only domain shift, it is also essential to tackle the uncertainty of both the category shift and the unknown space in actual cross-domain scenarios, which can be abstracted as a universal domain adaptation (UniDA) problem. This paper proposes a cross-domain diagnosis framework for rotating machinery under UniDA to handle the uncertainty dilemma by converting it to certainty. Specifically, a feature spatial location principle general for different label space relationships is designed, where the known classes form a hypersphere and the central region of the hypersphere is left for the possible target domain unknown samples. Then, a novel distance-based unknown detection strategy is proposed, which is friendly to different “open” degrees of the unknown space. The diagnosis results of three case studies show the effectiveness of the proposed framework under all possible label space relationships, including close set DA, partial set DA, open set DA and open-partial set DA. Further discussions demonstrate the superiority of the proposed framework in known class recognition and possible unknown detection. The proposed diagnosis framework provides a new insight for solving the uncertainty dilemma laid in actual cross-domain diagnosis scenarios under UniDA setting, which is more potential to practical engineering applications compared with many DA-based diagnosis methods tailored for specific label space relationship.