Adversarial Discriminative Domain Adaptation Research Articles

Structural damage detection based on transmissibility functions with unsupervised domain adaptation

Deep learning (DL) has been used for structural damage detection by training neural network models with a large amount of data. These trained models perform well when the test samples are from the data with an identical distribution of the training data. The distributions are always different due to numerical modelling errors and operational environmental varieties, and the data for damage scenarios is difficult to be obtained in practice. A novel method based on the joint maximum discrepancy and adversarial discriminative domain adaptation (JMDAD) for structural damage detection without labelled measurement data has been developed to address the above issues. To reduce the influence of external excitations in practice, the transmissibility function of measured structural responses is used for structural damage detection. The proposed network includes a feature generator, two classifiers and one discriminator. Firstly, the feature generator and domain discriminator are to extract features and merge their distributions at the domain level to overcome the issue of insufficient data from the target real structure. Secondly, the generator and two classifiers are optimised to align their distributions at the class level using the classification discrepancy between the two classifiers. As a result, the damage-sensitive features are extracted and aligned to eliminate modelling errors and operational environmental varieties between the source and target structures. Three case studies have been conducted in this paper, e.g., one case between two building structures with different storeys, another case between the numerical and experimental structures subjected to random and earthquake excitations and the application of Canton Tower. The results show that the proposed method is much robust to the measurement noise and operational environment and the structural damage is identified accurately without labelled measurement data from the target structure in operational environments.

Genetically optimised SMOTE-based adversarial discriminative domain adaptation for rotor fault diagnosis at variable operating conditions

Abstract For safety, reliability, and uninterrupted output of gas turbines, aviation engines, power-generating equipment, pumps, gears, compressors etc., rotor mass imbalance must be detected and diagnosed to avoid catastrophic failure. Industry 4.0 relies on predictive digital maintenance and deep learning-based convolutional neural network (CNN), which predicts defects but fails if the operating conditions change. Research studies in various fields indicate that the domain shift issue occurs due to source and target samples being from different domains, which reduces prediction capability. Moreover, research studies are scarce in examining prediction capability under varying operating speeds for rotor mass imbalance. Hence, this research proposes the adversarial discriminative domain adaptation (ADDA) technique which predicts machine failures under various operational conditions. The efficacy of ADDA has been explored by introducing 1D-CNN as a source and a target encoder inside ADDA's architecture to take advantage of CNN's feature extraction capability. Further, this research effectively tackles CNN's inherent issues of overfitting and hyperparameters value selection. Furthermore, The real-world scenario has more healthy samples than fault condition samples, causing a multiclass imbalance in sample data, which affects the classification decision boundary and causes biased prediction. Hence, the proposed methodology first addresses the class imbalance through synthetic minority oversampling (SMOTE), then genetic algorithm optimizes 1D-CNN's hyperparameters, and the effective dropout layer positioning solves the overfitting. Finally, the deep learning-based SMOTE_ADDA_GO-1D-CNN decreases domain discrepancy with adversarial discriminative domain adaptation. The proposed methodology's efficacy has been explored through F1-Score, which is used as multiclass evaluation metrics, and it has been benchmarked against standard machine learning and deep learning algorithms. The test results of the proposed methodology surpassed all of them with maximum prediction accuracy. Thus, this study contributes to rotor mass imbalance detection and diagnosis for multiclass imbalanced data under varying operational conditions by successfully overcoming potential challenges during fault prediction.

Fault severity recognition in axial piston pumps using attention-based adversarial discriminative domain adaptation neural network

Axial piston pumps are the ‘hearts’ of hydraulic systems whose fault recognition is necessary for the safety and reliability of hydraulic equipment. These pumps operate under different operating conditions and the fault recognition model trained at one operating point cannot be applicable at another operating point due to the problem of domain shifts. This paper proposes a transfer learning method for the fault severity recognition of axial piston pumps based on adversarial discriminative domain adaptation fused with a convolutional channel attention module. First, a convolutional neural network is pre-trained with labeled vibration data from the source domain, and a convolutional channel attention module is added to assign weights to different convolution kernels. Second, the trained source model is transferred to the target domain, and its parameters are updated by an adversarial training process between the labeled source data and the unlabeled target data. Finally, vibration data are collected from an axial piston pump at different fault levels under various operating conditions to validate the proposed method. Experimental results indicate that the proposed method achieves an average recognition accuracy of 98.3% and outperforms some other transfer learning methods by a large margin.

Transfer Learning for Bearing Fault Diagnosis based on Graph Neural Network with Dilated KNN and Adversarial Discriminative Domain Adaptation

Graph neural networks (GNNs) have emerged as a forefront in deep learning, notably influencing research in mechanical fault diagnosis. Transfer learning, particularly through domain adaptation (DA) techniques, has found application in machinery fault diagnosis by training models under one working condition and deploying them under another. While efforts have been made to integrate GNNs with DA techniques to alleviate data distribution discrepancies by investigating the inter-sample relationships, challenges persist: reliance on K-nearest neighbor (KNN) for graph generation emphasizes close relationships, neglecting distant ones; batch processing limits real-time fault diagnosis; and transfer between different-sized bearings is nearly unexplored. To address these limitations, a novel framework for GNN-based domain adaptation in machinery fault diagnosis is proposed. Initially, a convolutional neural network extracts node embeddings from the continuous wavelet transform graph of raw vibration signals. Subsequently, a graph generation layer based on dilated KNN captures both close and distant sample relationships, addressing the long-range dependency issue. Two GNN blocks are then applied for inter-sample relationships investigation and further feature extraction with the outputs directed to a linear classifier during source domain pretraining. Following pretraining, adversarial discriminative domain adaptation is leveraged to mitigate domain distribution discrepancies. Additionally, a novel graph construction method that combines existing training samples with a new single sample is proposed, enabling fault prediction with single instances for real-time online fault diagnosis. Evaluation on datasets with varying working conditions and bearings of different sizes demonstrates the superior performance of our method to other comparison methods.

Adversarial Distillation Adaptation Model with Sentiment Contrastive Learning for Zero-Shot Stance Detection

Zero-shot stance detection is both crucial and challenging because it demands detecting the stances of previously unseen targets in the inference stage. Learning transferable target invariant features effectively from training data is crucial for zero-shot stance detection. This paper proposes an adversarial adaptation approach for zero-shot stance detection, which applies an adversarial discriminative domain adaptation network to transfer knowledge efficiently. Specifically, the proposed model applies knowledge distillation to prevent overfitting the destination data and forgetting the learned source knowledge. Moreover, stance contrastive learning is applied to enhance the quality of feature representation for superior generalization, and sentiment information is extracted to assist with stance detection. The experimental results indicate that our model performs competitively on two benchmark datasets.

Adversarial discriminative domain adaptation for modulation classification based on Ulam stability

AbstractDomain adaptation modulation classification aims to improve the cross‐domain robustness of modulation classification, with the basic idea of mitigating the domain shift between the label‐rich source domain and label‐poor target domain in the latent common feature subspace. Due to the complex and heterogeneous wireless propagation conditions, how to construct the optimal latent common feature subspace is a hard problem. In this letter, the authors introduce Ulam stability to character the boundary of the latent common feature space. These theorems can provide guidance for cross‐domain feature extraction and eliminate the domain shift. Based on the proposed stable theorems, the authors define the Ulam loss and present a new method named Adversarial Discriminative Domain Adaptation with Ulam stability (U‐ADDA). Experiments on public datasets prove the effectiveness and generality of the method.

Semi-supervised adversarial discriminative domain adaptation.

Domain adaptation is a potential method to train a powerful deep neural network across various datasets. More precisely, domain adaptation methods train the model on training data and test that model on a completely separate dataset. The adversarial-based adaptation method became popular among other domain adaptation methods. Relying on the idea of GAN, the adversarial-based domain adaptation tries to minimize the distribution between the training and testing dataset based on the adversarial learning process. We observe that the semi-supervised learning approach can combine with the adversarial-based method to solve the domain adaptation problem. In this paper, we propose an improved adversarial domain adaptation method called Semi-Supervised Adversarial Discriminative Domain Adaptation (SADDA), which can outperform other prior domain adaptation methods. We also show that SADDA has a wide range of applications and illustrate the promise of our method for image classification and sentiment classification problems.

Deep adversarial domain adaptation for hyperspectral calibration model transfer among plant phenotyping systems

Hyperspectral imaging has widely been used for plant phenotyping applications. Calibration transfer is crucial for eliminating the prediction drift when a model developed in one plant phenotyping system needs to be deployed in another system. In this study two approaches (Domain Adversarial Neural Network - DANN and Adversarial Discriminative Domain Adaptation – ADDA) for transferring a deep learning model across hyperspectral imaging-based plant phenotyping systems were presented. The proposed techniques employ adversarial learning to minimise the discrepancy between the spectral data of two imaging facilities in the feature space. The calibration transfer analysis was examined by collecting the hyperspectral images of maize plants in two phenotyping facilities equipped with different sensors covering the visible near-infrared (VNIR) region under different lighting conditions. The capability of the suggested methods to transfer a relative water content (RWC) prediction model was investigated. The results showed that the prediction R 2 improved from 0.603 to 0.849 and 0.843 and RMSE reduced from 23.52% to 14.15% and 12.73% for DANN and ADDA, respectively. The study showed that both approaches improved the performance of the transferred model compared to a similar deep learning model without adversarial transfer. Visualisation of feature space showed that both methods lead to similar source and target feature distributions, thus indicating that a prediction model developed using these common features can successfully be applied to both imaging facilities. The main benefit of the DANN and ADDA is that they do not need the standard samples to be imaged in two phenotyping systems or the collection of labelled data to carry out the transfer. The adversarial learning-based calibration transfer can promote the sharing of prediction models across different phenotyping systems. • Differences across phenotyping facilities hinders the application of exiting models. • Calibration transfer between different phenotyping facilities is critically important. • Adversarial domain adaptation based methods were proposed for calibration transfer. • Proposed methods resulted in RWC model shared between two facilities. • Proposed methods can help to eliminate the need of full recalibration.

Knowledge distillation for BERT unsupervised domain adaptation

A pre-trained language model, BERT, has brought significant performance improvements across a range of natural language processing tasks. Since the model is trained on a large corpus of diverse topics, it shows robust performance for domain shift problems in which data distributions at training (source data) and testing (target data) differ while sharing similarities. Despite its great improvements compared to previous models, it still suffers from performance degradation due to domain shifts. To mitigate such problems, we propose a simple but effective unsupervised domain adaptation method, adversarial adaptation with distillation (AAD), which combines the adversarial discriminative domain adaptation (ADDA) framework with knowledge distillation. We evaluate our approach in the task of cross-domain sentiment classification on 30 domain pairs, advancing the state-of-the-art performance for unsupervised domain adaptation in text sentiment classification.

Research on fault diagnosis of gas turbine rotor based on adversarial discriminative domain adaption transfer learning

In the process of gas turbine rotor fault diagnosis based on data-driven, transfer learning is an effective method to solve the lack of gas turbines labeled data, which will result in domain shifts due to the data distribution difference between source domain data and target domain data under variable working condition. A gas turbine fault diagnosis method based on Adversarial Discriminative Domain Adaptation Transfer Learning Network (ADDATLN) is put forward to reduce domain offsets and improve the gas turbine fault diagnosis accuracy. In the proposed method, pre-trained deep Convolutional Neural Networks (CNN) models in the source domain is transferred to target domain data, then deep adversarial training between the source domain and target domain is adopted to adaptively optimize the model parameters of the target domain network, with the purpose of reducing domain offsets and improving gas turbine fault classification accuracy. Field test experiment results on gas turbine rotor fault diagnosis under different working conditions show that the average accuracy of the proposed method reaches 96.45%, and the average accuracy of fault diagnosis on different gas turbines with the same type achieved 95.13%. The field test results confirm that the method effectively reduces the domain differences caused by varying working conditions and different gas turbines, and improves the accuracy of gas turbine rotor fault diagnosis under variable working condition and for different gas turbines with small samples.

Profile to frontal face recognition in the wild using coupled conditional generative adversarial network

In recent years, with the advent of deep-learning, face recognition (FR) has achieved exceptional success. However, many of these deep FR models perform much better in handling frontal faces compared to profile faces. The major reason for poor performance in handling of profile faces is that it is inherently difficult to learn pose-invariant deep representations that are useful for profile FR. In this paper, the authors hypothesise that the profile face domain possesses a latent connection with the frontal face domain in a latent feature subspace. The authors look to exploit this latent connection by projecting the profile faces and frontal faces into a common latent subspace and perform verification or retrieval in the latent domain. A coupled conditional generative adversarial network (cpGAN) structure is leveraged to find the hidden relationship between the profile and frontal images in a latent common embedding subspace. Specifically, the cpGAN framework consists of two conditional GAN-based sub-networks, one dedicated to the frontal domain and the other dedicated to the profile domain. Each sub-network tends to find a projection that maximises the pair-wise correlation between the two feature domains in a common embedding feature subspace. The efficacy of the authors’ approach compared with the state of the art is demonstrated using the CFP, CMU Multi-PIE, IARPA Janus Benchmark A, and IARPA Janus Benchmark C datasets. Additionally, the authors have also implemented a coupled convolutional neural network (cpCNN) and an adversarial discriminative domain adaptation network (ADDA) for profile to frontal FR. The authors have evaluated the performance of cpCNN and ADDA and compared it with the proposed cpGAN. Finally, the authors have also evaluated the authors’ cpGAN for reconstruction of frontal faces from input profile faces contained in the VGGFace2 dataset.

Research on Multimodal Image Fusion Target Detection Algorithm Based on Generative Adversarial Network

In this paper, we propose a target detection algorithm based on adversarial discriminative domain adaptation for infrared and visible image fusion using unsupervised learning methods to reduce the differences between multimodal image information. Firstly, this paper improves the fusion model based on generative adversarial network and uses the fusion algorithm based on the dual discriminator generative adversarial network to generate high-quality IR-visible fused images and then blends the IR and visible images into a ternary dataset and combines the triple angular loss function to do migration learning. Finally, the fused images are used as the input images of faster RCNN object detection algorithm for detection, and a new nonmaximum suppression algorithm is used to improve the faster RCNN target detection algorithm, which further improves the target detection accuracy. Experiments prove that the method can achieve mutual complementation of multimodal feature information and make up for the lack of information in single-modal scenes, and the algorithm achieves good detection results for information from both modalities (infrared and visible light).

Targeted adversarial discriminative domain adaptation

Domain adaptation is a technology enabling aided target recognition and other algorithms for environments and targets with data or labeled data that is scarce. Recent advances in unsupervised domain adaptation have demonstrated excellent performance but only when the domain shift is relatively small. We proposed targeted adversarial discriminative domain adaptation (T-ADDA), a semi-supervised domain adaptation method that extends the ADDA framework. By providing at least one labeled target image per class, used as a cue to guide the adaption, T-ADDA significantly boosts the performance of ADDA and is applicable to the challenging scenario in which the sets of targets in the source and target domains are not the same. The efficacy of T-ADDA is demonstrated by cross-domain, cross-sensor, and cross-target experiments using the common digits datasets and several aerial image datasets. Results demonstrate an average increase of 15% improvement with T-ADDA over ADDA using just a few labeled images when adapting to a small domain shift and afforded a 60% improvement when adapting to large domain shifts.

Building damage detection based on multi-source adversarial domain adaptation

Building damage assessment plays an essential role during post-disaster rescue operations. Given that labeled samples are difficult to timely obtain after a disaster, transfer learning attracts increasing attention. However, different sensors employed cause considerable discrepancies not only between historical and current scenes but also among historical scenes, which could exert an effect on transfer performance. Therefore, a multi-source adversarial domain adaptation (MADA) method is proposed in this paper to fulfill the task of post-disaster building assessment. This method consists of two phases. First, imageries of several historical scenes are transformed into the same style of the current scene through the CycleGAN model with a classifier, ensuring class invariance, to be fused to make an adapted source domain. Second, feature alignment between adapted source and target domains is executed based on adversarial discriminative domain adaptation. The MADA method enhances the transformed image quality, fully utilizes relevant information in historical scenes, solves inter-scene interference problems among historical images, and improves the transfer efficiency from historical to the current disaster scene. Two experiments are conducted with Hurricane Sandy, Irma, and Maria datasets as multi-source and target domains to validate MADA’s effectiveness. Results show that the classification performance is better than other methods.

ADVERSARIAL DISCRIMINATIVE DOMAIN ADAPTATION FOR DEFORESTATION DETECTION

Abstract. Although very efficient in a number of application fields, deep learning based models are known to demand large amounts of labeled data for training. Particularly for remote sensing applications, responding to that demand is generally expensive and time consuming. Moreover, supervised training methods tend to perform poorly when they are tested with a set of samples that does not match the general characteristics of the training set. Domain adaptation methods can be used to mitigate those problems, especially in applications where labeled data is only available for a particular region or epoch, i.e., for a source domain, but not for a target domain on which the model should be tested. In this work we introduce a domain adaptation approach based on representation matching for the deforestation detection task. The approach follows the Adversarial Discriminative Domain Adaptation (ADDA) framework, and we introduce a margin-based regularization constraint in the learning process that promotes a better convergence of the model parameters during training. The approach is evaluated using three different domains, which represent sites in different forest biomes. The experimental results show that the approach is successful in the adaptation of most of the domain combination scenarios, usually with considerable gains in relation to the baselines.

Robust adversarial discriminative domain adaptation for real-world cross-domain visual recognition

Deep convolutional networks (CNNs) are able to learn robust representations and empower many computer vision tasks such as object recognition. However, when applying CNNs to industrial visual systems, they usually suffer from domain shift that exists between the training data and testing data. Such shift can be caused by different environment, types of cameras and exteriors of objects, leading to degrading performance and hindering the practical applications of CNNs in real-world visual recognition. To tackle this problem, Adversarial domain adaptation (ADA) reduces such shift by min–max optimization. However, current CNNs with ADA are hard to train due to training instability of adversarial network. In this paper, we propose a unified and easy-to-train domain adaptation framework, namely Attention-based Domain-confused Adversarial Domain ADaptation (AD3). Our method leverages both adversarial and statistical domain alignment, allows flexibility for source and target feature extractors and simultaneously performs feature-level and attention-level alignment. The statistical domain alignment promotes and stabilizes the adversarial domain learning, which reduces the manual work of tuning the hyper-parameters. The experimental results validate that our method performs better adaptation and faster convergence for adversarial domain learning than existing state-of-the-art methods on DIGITS, Office-31 and VisDA domain adaptation benchmarks.

Domain-invariant interpretable fundus image quality assessment.

Objective and quantitative assessment of fundus image quality is essential for the diagnosis of retinal diseases. The major factors in fundus image quality assessment are image artifact, clarity, and field definition. Unfortunately, most of existing quality assessment methods focus on the quality of overall image, without interpretable quality feedback for real-time adjustment. Furthermore, these models are often sensitive to the specific imaging devices, and cannot generalize well under different imaging conditions. This paper presents a new multi-task domain adaptation framework to automatically assess fundus image quality. The proposed framework provides interpretable quality assessment with both quantitative scores and quality visualization for potential real-time image recapture with proper adjustment. In particular, the present approach can detect optic disc and fovea structures as landmarks, to assist the assessment through coarse-to-fine feature encoding. The framework also exploit semi-tied adversarial discriminative domain adaptation to make the model generalizable across different data sources. Experimental results demonstrated that the proposed algorithm outperforms different state-of-the-art approaches and achieves an area under the ROC curve of 0.9455 for the overall quality classification.

Improved Techniques for Adversarial Discriminative Domain Adaptation.

Adversarial discriminative domain adaptation (ADDA) is an efficient framework for unsupervised domain adaptation in image classification, where the source and target domains are assumed to have the same classes, but no labels are available for the target domain. While ADDA has already achieved better training efficiency and competitive accuracy on image classification in comparison to other adversarial based methods, we investigate whether we can improve its performance with a new framework and new loss formulations. Following the framework of semi-supervised GANs, we first extend the discriminator output over the source classes, in order to model the joint distribution over domain and task. We thus leverage on the distribution over the source encoder posteriors (which is fixed during adversarial training) and propose maximum mean discrepancy (MMD) and reconstruction-based loss functions for aligning the target encoder distribution to the source domain. We compare and provide a comprehensive analysis of how our framework and loss formulations extend over simple multi-class extensions of ADDA and other discriminative variants of semi-supervised GANs. In addition, we introduce various forms of regularization for stabilizing training, including treating the discriminator as a denoising autoencoder and regularizing the target encoder with source examples to reduce overfitting under a contraction mapping (i.e., when the target per-class distributions are contracting during alignment with the source). Finally, we validate our framework on standard datasets like MNIST, USPS, SVHN, MNIST-M and Office-31. We additionally examine how the proposed framework benefits recognition problems based on sensing modalities that lack training data. This is realized by introducing and evaluating on a neuromorphic vision sensing (NVS) sign language recognition dataset, where the source domain constitutes emulated neuromorphic spike events converted from conventional pixel-based video and the target domain is experimental (real) spike events from an NVS camera. Our results on all datasets show that our proposal is both simple and efficient, as it competes or outperforms the state-of-the-art in unsupervised domain adaptation, such as DIFA and MCDDA, whilst offering lower complexity than other recent adversarial methods.

ADVERSARIAL DOMAIN ADAPTATION FOR THE CLASSIFICATION OF AERIAL IMAGES AND HEIGHT DATA USING CONVOLUTIONAL NEURAL NETWORKS

Abstract. Domain adaptation (DA) can drastically decrease the amount of training data needed to obtain good classification models by leveraging available data from a source domain for the classification of a new (target) domains. In this paper, we address deep DA, i.e. DA with deep convolutional neural networks (CNN), a problem that has not been addressed frequently in remote sensing. We present a new method for semi-supervised DA for the task of pixel-based classification by a CNN. After proposing an encoder-decoder-based fully convolutional neural network (FCN), we adapt a method for adversarial discriminative DA to be applicable to the pixel-based classification of remotely sensed data based on this network. It tries to learn a feature representation that is domain invariant; domain-invariance is measured by a classifier’s incapability of predicting from which domain a sample was generated. We evaluate our FCN on the ISPRS labelling challenge, showing that it is close to the best-performing models. DA is evaluated on the basis of three domains. We compare different network configurations and perform the representation transfer at different layers of the network. We show that when using a proper layer for adaptation, our method achieves a positive transfer and thus an improved classification accuracy in the target domain for all evaluated combinations of source and target domains.

An automatic diagnostic network using skew-robust adversarial discriminative domain adaptation to evaluate the severity of depression

Background and ObjectiveDeep learning provides an automatic and robust solution to depression severity evaluation. However, despite it is powerful, there is a trade-off between robust performance and the cost of manual annotation. MethodsMotivated by knowledge evolution and domain adaptation, we propose a deep evaluation network using skew-robust adversarial discriminative domain adaptation (SRADDA), which adaptively shifts its domain from a large-scale Twitter dataset to a small-scale depression interview dataset for evaluating the severity of depression. ResultsWithout top-down selection, SRADDA-based severity evaluation network achieves regression errors of 6.38 (Root Mean Square Error,RMSE) and 4.93 (Mean Absolute Error,MAE), which outperforms baselines provided by the Audio/Visual Emotion Challenge and Workshop(AVEC 2017). However, with top-down selection, the network achieves comparable results (RMSE = 5.13, MAE = 4.08). ConclusionsResults show that SRADDA not only represents features robustly, but also performs comparably to state-of-the-art results on small-scale dataset, DAIC-WOZ.