Joint Distribution Adaptation Research Articles

Riemannian Locality Preserving Method for Transfer Learning with Applications on Brain-computer Interface.

Brain-computer interfaces (BCIs) have been widely focused and extensively studied in recent years for their huge prospect of medical rehabilitation and commercial applications. Transfer learning exploits the information in the source domain and applies in another different but related domain (target domain), and is therefore introduced into the BCIs to figureout the inter-subject variances of electroencephalography (EEG) signals. In this article, a novel transfer learning method is proposed to preserve the Riemannian locality of data structure in both the source and target domains and simultaneously realize the joint distribution adaptation of both domains to enhance the effectiveness of transfer learning. Specifically, a Riemannian graph is first defined and constructed based on the Riemannian distance to represent the Riemannian geometry information. To simultaneously align the marginal and conditional distribution of source and target domains and preserve the Riemannian locality of data structure in both domains, the Riemannian graph is embedded in the joint distribution adaptation (JDA) framework and forms the proposed Riemannian locality preserving-based transfer learning (RLPTL). To validate the effect of the proposed method, it is compared with several existing methods on two open motor imagery datasets, and both multi-source domains (MSD) and single-source domains (SSD) experiments are considered. Experimental results show that the proposed method achieves the highest accuracies in MSD and SSD experiments on three datasets and outperforms eight baseline methods, which demonstrates that the proposed method creates a feasible and efficient way to realize transfer learning.

A meta transfer learning method for gearbox fault diagnosis with limited data

Intelligent diagnosis of mechanical faults is an important means to guarantee the safe maintenance of equipment. Cross domain diagnosis may lack sufficient measurement data as support, and this bottleneck is particularly prominent in high-end manufacturing. This paper presents a few-shot fault diagnosis methodology based on meta transfer learning for gearbox. To be specific, firstly, the subtasks for transfer diagnosis are constructed, and then joint distribution adaptation is conducted to align the two domain distributions; secondly, through adaptive manifold regularization, the data of target working condition is further utilized to explore the potential geometric structure of the data distribution. Meta stochastic gradient descent is explored to dynamically adjust the model’s parameter based on the obtained task information to obtain better generalization performance, ultimately to achieve transfer diagnosis of gearbox faults with few samples. The effectiveness of the approach is supported by the experimental datasets of the gearbox.

Reject inference in credit scoring based on cost-sensitive learning and joint distribution adaptation method

As traditional credit evaluation methods generally only use accepted sample modeling, the rejected data is omitted, which means the model's prediction of new customers is biased. However, reject inference can be used to solve this credit evaluation sample selection bias. This paper proposes a new reject inference method based on joint distribution adaptation (JDA) and cost-sensitive semi-supervised support vector machines (CS4VM). First, this method uses both accepted (labeled) samples and rejected (unlabeled) samples modeling, which overcomes the deviations in traditional credit evaluation methods. Second, as the accepted sample and the rejected sample distributions are different, this method reduces the distribution differences between the accepted and rejected sample sets, which ensures that the sample data conforms to the basic assumptions in the semi-supervised model, and improves the performance of the classification model. Third, this method reduces the overall cost in the actual credit business by considering both the traditional misclassification costs when mining the default samples and the different decision weights for the accepted and rejected samples. Finally, an empirical study verifies the excellent predictive performance of the proposed method and effectively reduces the total credit costs.

Transfer learning in near infrared spectroscopy for stingless bee honey quality prediction across different months

Near infrared spectroscopy (NIRS) coupled with machine learning has demonstrated its capability as an efficient secondary measurement method in various applications. However, a NIRS predictive model might be unable to maintain its accuracy when it is applied to samples that were acquired from different periods. Particularly, the qualities of stingless bee honey are affected by various uncontrolled factors, e.g. ecosystem, origins, and weather. Since it is unrealistic to have a NIRS dataset that can represent unforeseen future changes, an algorithm that can adapt existing data for new samples is worth to be investigated. Thus, this study aims to evaluate the feasibility of homogenous transfer learning approaches to overcome data constraints in developing NIRS predictive models of stingless bee honey qualities across different months. First, the near infrared transmittance spectra were acquired across honey samples prior to their respective conventional quality measurements in four different analysis months. Then, data from different analysis months were used as source and target datasets. Next, joint distribution adaptation based partial least square (JDA-PLS) and transfer component analysis based PLS (TCA-PLS) were implemented to establish NIRS predictive models of moisture, hydroxymethylfurfural (HMF), and glucose quality. Results show that the proposed TCA-PLS outperformed JDA-PLS and PLS by achieving the lowest RMSEP of 3.401 % and 0.804 mg/kg in predicting the moisture and glucose of stingless bee honey, respectively, across different months. Lastly, finding shows that both JDA and TCA were inefficacy when the label distributions of the calibration and validation datasets were similar.

Fine classification of crops based on an inductive transfer learning method with compact polarimetric SAR images

ABSTRACT Compact polarimetric synthetic aperture radar (CP SAR) reduces fully polarimetric SAR system complexity and expands the imaging swath. Generally, fine classification of crop types relies on many labeled training samples. However, due to the temporal interval of crop phenology and ground environment variations over time, training samples from one dataset usually perform poorly for another. Therefore, in this study, transfer learning is introduced to crop classification to ensure classification accuracy by improving reusability of training samples. A stable and robust inductive transfer learning method, i.e. the Transfer Bagging-based Ensemble Learning (TBEL) algorithm, is proposed. The main idea is to select an adequate number of representative samples from unlabeled datasets to characterize each class in the target domain based on limited labeled samples and construct a classifier set to classify the target domain. This study investigates CP SAR data performance in transfer learning for crop classification. The proposed algorithm in the experimental study is compared with six typical methods (Subspace Alignment (SA), CORrelation ALignment (CORAL), Joint Distribution Adaptation (JDA), Balanced Distribution Adaptation (BDA), Transfer Bagging (TrBagg), and Bagging-based Ensemble Transfer Learning (BETL)). The experimental results show that the crop classification accuracy based on the TBEL algorithm is more stable, with an improved overall classification accuracy of 2–6%. Classifying the same rice harvest stage in the cross-year domain has the highest overall accuracy of 92.2%. Wheat fields in different scenes are also classified. Based on the TBEL algorithm, the overall classification accuracy improves by 1–10% compared with typical methods, with an accuracy of at least 87.6%. Furthermore, by testing the CP mode classification performance over various crops in transfer learning, we find that the circular CP mode performs better than the linear mode in most cases. This conclusion agrees with single-scene applications and was first verified in transfer learning.

Wind turbines fault diagnosis method under variable working conditions based on AMVMD and deep discrimination transfer learning network

With the wide application of wind turbines, the bearing fault diagnosis of wind turbines has become a research hotspot. Under complex variable working conditions, the vibration signals of bearing components show non-stationary characteristics. Therefore, it is challenging to extract fault features using typical fault diagnosis methods. This paper proposes Adaptive Multivariate Variational Mode Decomposition combined with an improved Deep Discrimination Transfer Learning Network (AMVMD-IDDTLN) for bearing fault diagnosis of wind turbines under variable working conditions. First, the AMVMD method is used for the adaptive decomposition of the original signal, and use SE-ResNet18 convolutional neural network to obtain the transfer features of the source domain and target domain. Then, marginal distribution differences and conditional differences are assessed by DDM measures. The whole model is optimized by cross-entropy and improved joint distribution adaptation loss function, and the identification and classification of cross-working fault characteristics of the wind turbine- bearings are realized. The model achieves 99.48% transfer learning for the ten classifications of CWRU data set, 97% transfer learning for the four classifications of UPB data set, and 90% transfer learning for wind turbine bearing data across working conditions and across equipment. It is concluded that: Compared with similar models, the AMVMD-IDDTLN model proposed in this paper has higher diagnostic accuracy and faster convergence rate, which has certain practicality.

A lightweight fault diagnosis model for planetary gearbox using domain adaptation and model compression

This article proposes a novel lightweight attention spatiotemporal joint distribution adaptation network fault diagnosis model to address the key challenges of domain transfer and high model complexity in traditional methods. The novelty lies in 1. Using model compression techniques to reduce the complexity of the network model and improve its computational efficiency; 2. Introducing new domain adaptation and adversarial methods to solve the domain transfer problem. The effectiveness of the proposed model is verified through a transfer experiment of planetary gearbox vibration data. The experimental results show that the proposed model reduces the parameters and computational complexity to 18 % and 15 % of the original model, respectively, and has a diagnostic accuracy of over 98 % in cross-condition transfer tasks, and still maintains an accuracy of over 88 % even under high noise levels. This indicates that the proposed model is an efficient and accurate fault diagnosis model.

A New Framework Based on Supervised Joint Distribution Adaptation for Bearing Fault Diagnosis across Diverse Working Conditions

To address the degradation of diagnostic performance due to data distribution differences and the scarcity of labeled fault data, this study has focused on transfer learning-based cross-domain fault diagnosis, which attracts considerable attention. However, deep transfer learning-based methods often present a challenge due to their time-consuming and costly nature, particularly in tuning hyperparameters. For this issue, on the basis of classical features-based transfer learning method, this study introduces a new framework for bearing fault diagnosis based on supervised joint distribution adaptation and feature refinement. It first utilizes ensemble empirical mode decomposition to process raw signals, and statistical features extraction is implemented. Then, a new feature refinement module is designed to refine domain adaptation features from high-dimensional feature set by evaluating the fault distinguishability and working-condition invariance of feature data. Next, it proposes a supervised joint distribution adaptation method to conduct improved joint distribution alignment that preserves neighborhood relationships within a manifold subspace. Finally, an adaptive classifier is trained to predict fault labels of feature data across varying working conditions. To prove the cross-domain fault diagnosis performance and superiority of the proposed methods, two bearing datasets are applied for experiments, and the experimental results verify that the model built by the proposed framework can achieve desirable diagnosis performance under different working conditions and that it apparently outperforms comparative models.

Regularized Wasserstein Distance-Based Joint Distribution Adaptation Approach for Fault Detection Under Variable Working Conditions

Regularized Wasserstein Distance-Based Joint Distribution Adaptation Approach for Fault Detection Under Variable Working Conditions

Category-level selective dual-adversarial network using significance-augmented unsupervised domain adaptation for surface defect detection

Surface defect detection is very important to ensure the quality of industrial products. Traditional machine learning cannot be well extended to a non-identically distributed dataset, making surface defect detection a data-limited task. Unsupervised domain adaptation (UDA) can solve this problem by transferring knowledge from a labeled source domain to an unlabeled target domain. Adversarial learning is one of the latest heuristic methods to deal with domain shift in UDA tasks. Although impressive results have been achieved, the adversarial model still suffers from the equilibrium challenge, which may lead to under-transfer or negative transfer. To this end, we utilize joint distribution adaptation to propose a novel UDA model, named significance-augmented-based category-level selective dual-adversarial (CDASA) network, to learn a generalized model. Specifically, to promote positive transfer, we use the selective dual-adversarial (DA) learning strategy to further minimize the feature distribution difference at equilibrium to achieve better domain confusion. Meanwhile, guided by defect recognition performance, the transferability of confusion domain is measured to enhance the distributions of potential domains. In addition, to avoid under-transfer, we consider the relationship between the target data and the decision boundary, and then the significance-augmented (SA) mechanism is proposed to encourage class-level alignment. Thus, the alignment features with domain-invariant and category discrimination can be captured simultaneously. Extensive experiments on collected real industrial datasets and publicly available steel surface defect datasets confirm the effectiveness of our approach.

A Multi-Adversarial Joint Distribution Adaptation Method for Bearing Fault Diagnosis under Variable Working Conditions

Deep network fault diagnosis requires a lot of labeled data and assumes identical data distributions for training and testing. In industry, varying equipment conditions lead to different data distributions, making it challenging to maintain consistent fault diagnosis performance across conditions. To this end, this paper designs a transfer learning model named the multi-adversarial joint distribution adaptation network (MAJDAN) to achieve effective fault diagnosis across operating conditions. MAJDAN uses a one-dimensional lightweight convolutional neural network (1DLCNN) to directly extract features from the original bearing vibration signal. Combining the distance-based domain-adaptive method, maximum mean difference (MMD), with the multi-adversarial network will simultaneously reduce the conditional and marginal distribution differences between the domains. As a result, MAJDAN can efficiently acquire domain-invariant feature information, addressing the challenge of cross-domain bearing fault diagnosis. The effectiveness of the model was verified based on two sets of different bearing vibration signals, and one-to-one and one-to-many working condition migration task experiments were carried out. Simultaneously, various levels of noise were introduced to the signal to enable analysis and comparison. The findings demonstrate that the suggested approach achieves exceptional diagnostic accuracy and exhibits robustness.

Foundations and applicability of transfer learning for structural health monitoring of bridges

The number of bridges worldwide is extensive, making it financially and technically challenging for the authorities to install a structural health monitoring (SHM) system and collect large quantities of data for every bridge. Transfer learning has gained relevance in the last few years to extend the SHM concept for most bridges, while minimizing costs with monitoring systems and time with data measurement. It can be especially suitable for bridges structurally similar and replicated extensively, like overpasses integrated into highways. Therefore, this paper intends to lay down the foundations of transfer learning for SHM of bridges and to highlight the importance of the quality of knowledge transferred across different bridges for damage detection. Transfer Component Analysis, Joint Distribution Adaptation, and Maximum Independence Domain Adaptation methods are applied to data sets from different bridges, where classifiers have access to labeled training data from one bridge (source domain) and unlabeled monitoring test data from another bridge (target domain) that present similarities. The effectiveness of those methods is compared through the classification performance using real-world monitoring data sets collected from the Z-24 Bridge in Switzerland, and the PI-57 and PK 075+317 Bridges in France.

KL Divergence-based transfer learning for cross-subject eye movement recognition with EOG signals.

Human-machine interfaces (HMIs) based on Electro-oculogram (EOG) signals have been widely explored. However, due to the individual variability, it is still challenging for an EOG-based eye movement recognition model to achieve favorable results among cross-subjects. The classical transfer learning methods such as CORrelation Alignment (CORAL), Transfer Component Analysis (TCA), and Joint Distribution Adaptation (JDA) are mainly based on feature transformation and distribution alignment, which do not consider similarities/dissimilarities between target subject and source subjects. In this paper, the Kullback-Leibler (KL) divergence of the log-Power Spectral Density (log-PSD) features of horizontal EOG (HEOG) between the target subject and each source subject is calculated for adaptively selecting partial subjects that suppose to have similar distribution with target subject for further training. It not only consider the similarity but also reduce computational consumption. The results show that the proposed approach is superior to the baseline and classical transfer learning methods, and significantly improves the performance of target subjects who have poor performance with the primary classifiers. The best improvement of Support Vector Machines (SVM) classifier has improved by 13.1% for subject 31 compared with baseline result. The preliminary results of this study demonstrate the effectiveness of the proposed transfer framework and provide a promising tool for implementing cross-subject eye movement recognition models in real-life scenarios.

Unbalanced Bearing Fault Diagnosis Under Various Speeds Based on Spectrum Alignment and Deep Transfer Convolution Neural Network

Bearing fault diagnosis plays a pivotal role in the safe and reliable operation of modern mechanical systems. However, the existing fault diagnosis methods rarely deal with the problem of category imbalance and various speeds concurrently, which cannot work effectively in practical scenarios. Considering the underlying similarities of data in frequency domain, data mining under various speeds can help to reduce the deviation of domain distribution. Therefore, a novel fault diagnosis method based on spectrum alignment (SA) and deep transfer convolution neural network (DTCNN) is proposed, where the SA and data augmentation module (DAM) are designed to extract spectrum alignment features from the unbalanced bearing data. The DTCNN model based on joint distribution adaptation (JDA) is built to facilitate learning reliable domain-invariant features. Different from the existing studies, a more general transfer task with time-varying speed is considered, even with complex faults. For 14 transfer tasks in two unbalanced fault diagnosis cases under variable speed, the average accuracy, F1-score, area under curve (AUC) of the proposed method can reach more than 97.76% 97.57% 98.75%. The results show that this method has superior diagnostic effect and better generalization ability than various state-of-the-art methods.

Revisiting ‘revisiting supervised methods for effort‐aware cross‐project defect prediction’

AbstractEffort‐aware cross‐project defect prediction (EACPDP), which uses cross‐project software modules to build a model to rank within‐project software modules based on the defect density, has been suggested to allocate limited testing resource efficiently. Recently, Ni et al. proposed an EACPDP method called EASC, which used all cross‐project modules to train a model without considering the data distribution difference between cross‐project and within‐project data. In addition, Ni et al. employed the different defect density calculation strategies when comparing EASC and baseline methods. To explore the effective defect density calculation strategies and methods on EACPDP, the authors compare four data filtering methods and five transfer learning methods with EASC using four commonly used defect density calculation strategies. The authors use three classification evaluation metrics and seven effort‐aware metrics to assess the performance of methods on 11 PROMISE datasets comprehensively. The results show that (1) The classification before sorting (CBS+) defect density calculation strategy achieves the best overall performance. (2) Using balanced distribution adaption (BDA) and joint distribution adaptation (JDA) with the K‐nearest neighbour classifier to build the EACPDP model can find 15% and 14.3% more defective modules and 11.6% and 8.9% more defects while achieving the acceptable initial false alarms (IFA). (3) Better comprehensive classification performance of the methods can bring better EACPDP performance to some extent. (4) A flexible adjustment of the defect threshold λ of the CBS+ strategy contribute to different goals. In summary, the authors recommend researchers and practitioners use to BDA and JDA with the CBS+ strategy to build the EACPDP model.

The accuracy losing phenomenon in abrasive tool condition monitoring and a noval WMMC-JDA based data-driven method considered tool stochastic surface morphology

As a critically issue in the maintaining of desired part quality and manufacturing productivity, tool condition monitoring (TCM) problem is treated by multi data-driven machine learning methods with training and testing datasets generated by stochastic segmentation. For existing methods, it is assumed that samples collected from same type tools follow identical probability distribution. Different from regular tools, abrasive tools dissatisfy this assumption because of their stochastic surface morphologies. Therefore, traditional data-driven methods loss their accuracies in abrasive tool changing cases. In data driven TCM models, the macroscopic effect of abrasive tool micro-stochasticity is discovered. For the abrasive tool frequently changing cases, an abrasive tool condition monitoring (ATCM) method based on unsupervised domain adaptation is presented to deal with changes caused by stochastic surface morphology. In feature-based transfer learning, source and target domain datasets are projected into a transferred subspace to enhance the distribution alignment similarity of two domains. A weighted maximum margin criterion (WMMC) is adopted in the domain adaptation process to make the transformed samples in same class closing but segregated from those in different classes. With unsupervised domain adaptation mechanism and multilayer perceptron (MLP), the unsupervised domain adaptation is proposed to deal with traditional model invalidation after abrasive tool changes. The proposed method is verified by the robotic belt grinding experiment results. Compared with the existing data-driven method, the weighted maximum margin criterion joint distribution adaptation (WMMC-JDA) based ATCM method still maintains the predicting effectiveness after abrasive tool changes.

A New Cross-Domain Bearing Fault Diagnosis Framework Based on Transferable Features and Manifold Embedded Discriminative Distribution Adaption Under Class Imbalance

Cross-domain fault diagnosis based on transfer learning has been popularly developed to overcome inconsistent data distribution-caused degradation of diagnostic performance. However, the existing methods are typically suffering from a class imbalance of domains and lacking sufficient fault data because it is difficult to obtain the real industrial data of machinery under fault conditions. To address these problems, this work proposes a new cross-domain bearing diagnosis framework based on transferable features and manifold embedded discriminative distribution adaption. First, it applies the maximal overlap discrete wavelet packet transform to process the vibration data and extract different statistics-based features. Then, to enhance the domain adaptation performance, it designs a transferability evaluation based on the adjusted rand index and maximum mean discrepancy to quantify the fault discriminability and domain invariance of the features. After that, it proposes a novel manifold embedded discriminative joint distribution adaptation method to perform cross-domain feature discriminative joint distribution alignment in a Grassmann manifold subspace. Finally, it utilizes a random forest classifier to train the cross-domain fault diagnosis model. To verify the performances of the proposed methods, extensive experiments have been conducted on two real rolling bearing datasets. The results demonstrate that the proposed methods can achieve the desirable diagnosis results and significantly outperform comparative classical transfer learning-based models when there is the class imbalance between source and target domains.

A novel intelligent bearing fault diagnosis method based on signal process and multi-kernel joint distribution adaptation

The present research on intelligent bearing fault diagnosis assumes that the same feature distribution is used to obtain training and testing data. However, the domain shift (distribution discrepancy) issue generally occurs in both datasets because of different operational conditions. The domain adaptation techniques are preferably applied for fault diagnosis to handle the domain shift issue. Moreover, collecting sufficient testing data or labelled data in real industries is a challenging task. Therefore, the multi-kernel joint distribution adaptation (MKJDA) with dynamic distribution alignment is proposed for bearing fault diagnosis. This method dynamically joins both the marginal and conditional distributions and uses the multi-kernel to solve the non-linear problems to extract the most effective and robust representation for cross-domain issues. Moreover, it runs with the unlabelled task domain to perform the diagnosis by iteratively updating the pseudo code. The experimental results (two public datasets and one experimental dataset) demonstrated that the proposed method (MKJDA) exhibited stable and robust accuracy while conducting bearing fault diagnosis. It can effectively address the most crucial issue: intelligent diagnosis methods must re-train the model when the distribution differs between the source domain (the model is learned) and the target domain (the learned model is applied).

A fuzzy-set-based joint distribution adaptation method for regression and its application to online damage quantification for structural digital twin

Online damage quantification suffers from insufficient labeled data that weakens its accuracy. In this context, adopting the domain adaptation on historical labeled data from similar structures/damages or simulated digital twin data to assist the current diagnosis task would be beneficial. However, most domain adaptation methods are designed for classification and cannot efficiently address damage quantification, a regression problem with continuous real-valued labels. This study first proposes a novel domain adaptation method, the Online Fuzzy-set-based Joint Distribution Adaptation for Regression, to address this challenge. By converting the continuous real-valued labels to fuzzy class labels via fuzzy sets, the marginal and conditional distribution discrepancy are simultaneously measured to achieve the domain adaptation for the damage quantification task. Thanks to the superiority of the proposed method, a state-of-the-art online damage quantification framework based on domain adaptation is presented. Finally, the framework has been comprehensively demonstrated with a damaged helicopter panel, in which three types of damage domain adaptations (across different damage locations, across different damage types, and from simulation to experiment) are all conducted, proving the accuracy of damage quantification can be significantly improved in a realistic environment. It is expected that the proposed approach to be applied to the fleet-level digital twin considering the individual differences.

Predicting Cardiovascular Rehabilitation of Patients with Coronary Artery Disease Using Transfer Feature Learning.

Cardiovascular diseases represent the leading cause of death worldwide. Thus, cardiovascular rehabilitation programs are crucial to mitigate the deaths caused by this condition each year, mainly in patients with coronary artery disease. COVID-19 was not only a challenge in this area but also an opportunity to open remote or hybrid versions of these programs, potentially reducing the number of patients who leave rehabilitation programs due to geographical/time barriers. This paper presents a method for building a cardiovascular rehabilitation prediction model using retrospective and prospective data with different features using stacked machine learning, transfer feature learning, and the joint distribution adaptation tool to address this problem. We illustrate the method over a Chilean rehabilitation center, where the prediction performance results obtained for 10-fold cross-validation achieved error levels with an NMSE of 0.03±0.013 and an R2 of 63±19%, where the best-achieved performance was an error level with a normalized mean squared error of 0.008 and an R2 up to 92%. The results are encouraging for remote cardiovascular rehabilitation programs because these models could support the prioritization of remote patients needing more help to succeed in the current rehabilitation phase.