Domain Adversarial Learning Research Articles

Multi-label domain adversarial reinforcement learning for unsupervised compound fault recognition

Multi-label domain adversarial reinforcement learning for unsupervised compound fault recognition

A Domain Generalized Face Anti-Spoofing System Using Domain Adversarial Learning

This research addresses the enhancement of face anti-spoofing (FAS) in facial recognition systems (FRS) against sophisticated fraudulent activities. Prior methods primarily focus on extracting facial features like color, texture, and dynamic variations, yet these methods struggle to accurately identify common characteristics of forged faces, thereby limiting generalization in practical scenarios. This study aims to propose a novel representation learning framework incorporating adversarial learning algorithms, to segregate features into liveness-specific and domain-specific categories, emphasizing liveness-specific features for FAS advancement. Feature disentanglement is central to this approach, enabling the deep learning models to effectively discern separable latent generating factors, such as identity, liveness, appearance, and texture. This methodology enhances model interpretability, explainability, and generalization. Additionally, Grad-CAM is employed to elucidate the basis for classifications made by the architecture, increasing explainability and trustworthiness. Empirical evaluation across panoply available FAS datasets confirms the superiority, significantly improving performance and robustness over existing technologies.

AVATAR: Adversarial self-superVised domain Adaptation network for TARget domain

This paper introduces AVATAR (adversarial self-supervised domain adaptation network for target domain), a novel unsupervised domain adaptation method designed to address the challenge of transferring knowledge from a well-labeled source domain to an unlabeled target domain. AVATAR’s key contributions lie in its unique combination of adversarial learning and self-supervised learning techniques, which enhance the model’s adaptability and accuracy in new, unlabeled environments. The technical innovations of AVATAR include a domain adversarial learning component that reduces domain discrepancy by aligning source and target domain features in a shared feature space, a self-supervised learning module that leverages pseudo-labels and instance weights to refine target domain discrimination boundaries, and a sample selection strategy guided by within-cluster distances to focus on aligning informative samples while mitigating the impact of outliers. By integrating these components, AVATAR provides a comprehensive framework for tackling complex unsupervised domain adaptation tasks, particularly when the domain gap is significant. The method balances domain-invariant feature learning with domain-specific adjustments, enabling effective adaptation to dynamically changing environments. Extensive experiments on multiple benchmark datasets demonstrate AVATAR’s superior performance compared to existing domain adaptation methods, especially in scenarios with limited labeled target domain data. These results highlight AVATAR’s potential as a versatile and robust solution for domain adaptation challenges in various real-world applications.

Low-resource entity resolution with domain generalization and active learning

Entity Resolution (ER), a fundamental task in data cleaning and integration, is critical in various fields such as healthcare, e-commerce, and social networks. Traditional ER methods are constrained by the need for substantial labeled samples and the challenge of generalization to unseen domains. To address the low-resource challenge in ER, a novel two-phase framework is proposed. Initially, we introduce the Domain Generalization Entity Resolution (DGER) framework, combining domain adversarial learning and simulated target learning to improve the generalization performance on unseen target domains. Subsequently, to further adapt to the target dataset, we present a novel active learning approach called Domain-Aware Uncertainty Active Learning (DUAL), for fine-tuning the DGER model with minimal annotation cost. DUAL manually annotates target domain samples that are highly uncertain and exhibit high divergence from the source, while assigning pseudo-labels to high-confidence samples. Experimental results on multiple real-world datasets demonstrate that our framework outperforms traditional ER methods in generalizing to unseen domains. Specifically, our DGER method outperforms the best-performing ER baseline in each task, achieving an average F1 score improvement of 9.02% across eight different test tasks. Moreover, within a limited annotation budget during the active learning phase, our DUAL fine-tuning strategy for the ER model outperforms uncertainty-based active learning techniques.

Uncertainty inspired domain adaptation network for rail surface defect segmentation

Rail surface defect detection is an essential part of railroad maintenance to prevent safety accidents. Recently, deep learning-based defect detection algorithms have shown impressive detection performance. However, the performance improvement from deep neural networks is based on a strong assumption that the training images have the same data distribution as the test images. This requirement is unrealistic in practical railroad surface defect detection. The appearance of rails varies significantly in different railroad sections due to changes in working conditions and maintenance status. In this paper, we propose a novel uncertainty-inspired unsupervised domain adaptation framework (UIUDA) to improve the generalization of deep learning models on data with distribution differences. Specifically, domain adversarial learning is used to align the feature distributions of training and test data. To mitigate the negative transfer caused by global alignment, we design a locally aligned domain adaptation strategy. In addition, self-training is introduced to refine the decision boundaries of the model in the test data. In this process, we propose an uncertainty-inspired evaluation method to remove noisy pseudo labels. We conducted domain adaptation experiments with RSDD and NEU-RSDD as the source domain and FRSD dataset as the target domain. The results show that UIUDA can effectively improve the generalization ability of the model, and the mIoU metric of defect segmentation is 80.17%.

Multi-stream domain adversarial prototype network for integrated smart roller TBM main bearing fault diagnosis across various low rotating speeds

The well-being of tunnel boring machines (TBM) heavily relies on the health status of main bearings, significantly impacting both safety and operational efficiency. Investigating the operation monitoring and diagnosis algorithms of TBM main bearings in practical scenarios with limited fault data and real-time variations in low-speed heavy-load operating conditions poses a research challenge. To solve these issues, a novel domain-adversarial prototype network diagnostic algorithm with a multi-stream fusion feature encoder (MDAPN) based on smart roller monitoring data is proposed. In contrast to existing external vibration monitoring schemes for bearings, monitoring data from smart rollers can mitigate complex external interference and greatly shorten the transmission path of fault sources. The algorithm encompasses a multi-stream fusion feature encoder, domain discriminator, and prototype network classifier. Specifically, the multi-stream fusion feature encoder adopts the two-stream convolutional network to deeply extract the axial-radial fused features of rollers, incorporating shallow statistical feature information. Domain-invariant features are generated based on domain adversarial learning strategies, and the prototype network classifier reduces dependence on target domain samples. An integrated smart roller main bearing fault simulation testbed was built, conducting 6 sets of cross-domain experiments. The proposed method reached an average accuracy of 98.41 % under 5-shot, surpassing the baseline method by 6.57 % at least. This validates that the MDAPN based on smart roller state exhibits excellent fault recognition performance for the main bearing under varying operational conditions with scarce available samples.

Graph-based domain adversarial learning framework for video anomaly detection domain generalization

Graph-based domain adversarial learning framework for video anomaly detection domain generalization

Automated BIM-to-scan point cloud semantic segmentation using a domain adaptation network with hybrid attention and whitening (DawNet)

Deep learning-based point cloud semantic segmentation facilitates scene understanding and BIM modelling, but its success requires vast amount of labelled point clouds, which is laborious and time-consuming. To reduce point cloud annotation cost, researchers attempt to leverage synthetic point clouds, but the domain gap between synthetic and real point clouds deteriorates the segmentation accuracy. To address this issue, this study develops a BIM-to-Scan point cloud semantic segmentation approach to mitigate the domain gap between BIM and real point clouds, improving the segmentation performance on real point clouds. To this end, this study starts by proposing a BIM-based point cloud generation method, which uses FME and BIM models to automatically generate and label synthetic point clouds, decreasing the annotation cost. To fill the domain gap, a DawNet is invented by integrating a domain adaptation network with ZCA whitening operation and hybrid attention mechanism. Specifically, ResPointNet++ is used to extract geometric features and execute the segmentation task, which is then combined with a domain discriminator to perform domain adversarial learning, aligning the data distribution of BIM and real point clouds. To improve the performance of the DawNet, a residual learning block with whitening and a hybrid attention module are designed. These two modules help extract and exploit domain-invariant features to boost the generalisation and segmentation performance of the DawNet. Finally comprehensive experiments show that the proposed BIM-based method spends 0.5 person-hours to generate 0.45 billion labelled BIM point clouds, and that the developed DawNet achieves 17% and 11% more mIoU than ResPointNet++ and DANN. The ablation study also confirms the effectiveness of the hybrid attention module and ZCA whitening operation.

An improved method for rapid un-collapsed building extraction from post-disaster high-resolution remote sensing imagery based on multi-scale feature alignment

ABSTRACT Quick and accurate extraction of un-collapsed buildings from post-disaster High-resolution Remote Sensing Images (HRSIs) is imperative for emergency response. Pre-disaster HRSIs could serve as auxiliary data for training models to expedite this extraction process. However, the effectiveness of models trained directly on pre-disaster HRSIs tends to diminish when applied to post-disaster scenarios, mainly due to the notable discrepancies between these datasets. The current popular approach to mitigate this issue involves aligning features from pre- and post-disaster images using an unsupervised domain adversarial learning framework. However, conventional methods often fall short in reducing the substantial disparity between pre- and post-disaster images, due to a lack of comprehensive alignment of category-level and multi-scale features. To overcome these limitations, we propose the Multi-scale Global and Category-attention Features Alignment Network (MGCAN). This novel approach further refines feature alignment strategies by concurrently aligning both multi-scale global and category-attention features, thus effectively narrowing the gap between pre- and post-disaster HRSIs. Extensive experiments have demonstrated that MGCAN significantly improves the accuracy of un-collapsed building extraction from post-disaster HRSIs. Moreover, compared to other state-of-the-art domain adversarial networks, MGCAN exhibits superior performance in different disaster scenarios.

Vocoder Detection of Spoofing Speech Based on GAN Fingerprints and Domain Generalization

As an important part of the text-to-speech (TTS) system, vocoders convert acoustic features into speech waveforms. The difference in vocoders is key to producing different types of forged speech in the TTS system. With the rapid development of general adversarial networks (GANs), an increasing number of GAN vocoders have been proposed. Detectors often encounter vocoders of unknown types, which leads to a decline in the generalization performance of models. However, existing studies lack research on detection generalization based on GAN vocoders. To solve this problem, this study proposes vocoder detection of spoofed speech based on GAN fingerprints and domain generalization. The framework can widen the distance between real speech and forged speech in feature space, improving the detection model’s performance. Specifically, we utilize a fingerprint extractor based on an autoencoder to extract GAN fingerprints from vocoders. We then weight them to the forged speech for subsequent classification to learn the forged speech features with high differentiation. Subsequently, domain generalization is used to further improve the generalization ability of the model for unseen forgery types. We achieve domain generalization using domain-adversarial learning and asymmetric triplet loss to learn a better generalized feature space in which real speech is compact and forged speech synthesized by different vocoders is dispersed. Finally, to optimize the training process, curriculum learning is used to dynamically adjust the contributions of the samples with different difficulties in the training process. Experimental results show that the proposed method achieves the most advanced detection results among four GAN vocoders. The code is available at https://github.com/multimedia-infomation-security/GAN-Vocoder-detection .

Distilling Invariant Representations With Domain Adversarial Learning for Cross-Subject Children Seizure Prediction

Seizure prediction based on electroencephalogram (EEG) has great potential to improve patients’ life quality. Due to the high heterogeneity in distributions of EEG signals among different patients, conventional studies usually show poor generalization ability when transferring the model to new patients, which also leads to difficulties in clinical applications. To alleviate the challenging issue concerning cross-subject domain shift, we propose a transformer-based domain adversarial model. Our model first adopts a pre-trained general neural network to extract common features from the EEG signals of available patients. Then, we design a distiller module and a domain discriminator module to perform domain adaptation training based on a small amount of labeled data from the new-coming patient. During adaptation process, conditional domain adversarial training with the addition of label information is employed to remove patient-related information from the extracted features to learn a common seizure feature space among different patients. Our proposed seizure prediction method is evaluated on the CHB-MIT EEG database. The proposed model achieves a sensitivity of 79.5%, a false alarm rate (FPR) of 0.258/h, and an AUC of 0.814. Experimental results demonstrate that the proposed method can effectively reduce inter-patient domain disparity compared to state-of-the-art methods.

Exploring Structure Incentive Domain Adversarial Learning for Generalizable Sleep Stage Classification

Sleep stage classification is crucial for sleep state monitoring and health interventions. In accordance with the standards prescribed by the American Academy of Sleep Medicine, a sleep episode follows a specific structure comprising five distinctive sleep stages that collectively form a sleep cycle. Typically, this cycle repeats about five times, providing an insightful portrayal of the subject’s physiological attributes. The progress of deep learning and advanced domain generalization methods allows automatic and even adaptive sleep stage classification. However, applying models trained with visible subject data to invisible subject data remains challenging due to significant individual differences among subjects. Motivated by the periodic category-complete structure of sleep stage classification, we propose a Structure Incentive Domain Adversarial learning (SIDA) method that combines the sleep stage classification method with domain generalization to enable cross-subject sleep stage classification. SIDA includes individual domain discriminators for each sleep stage category to decouple subject dependence differences among different categories and fine-grained learning of domain-invariant features. Furthermore, SIDA directly connects the label classifier and domain discriminators to promote the training process. Experiments on three benchmark sleep stage classification datasets demonstrate that the proposed SIDA method outperforms other state-of-the-art sleep stage classification and domain generalization methods and achieves the best cross-subject sleep stage classification results.

Performance assessment of cross office building energy prediction in the same region using the domain adversarial transfer learning strategy

Precise and reliable building energy consumption prediction is of great significance for building energy management and future energy planning. To mitigate the adverse impact of data limitation on the performance of building energy consumption prediction model, a transfer learning strategy was introduced. However, the operation modes of different building energy systems are different, resulting in a large difference in the distribution of building energy consumption related data. In this paper, the relevant data of 25 buildings from the same region of the United States were selected. On this basis, two similarity calculation strategies are compared, and three similarity measurement methods were used to analyze the impact of different source buildings on the performance improvement of transfer learning models. The results show that choosing the appropriate source building can improve the transfer learning performance by up to 82.56% when compared to the no-transfer model (the baseline model). When higher similarity source buildings are selected, the corresponding average PIR is about 24.7% higher than the total average, which can make the transfer model performance improvement more obvious. This paper provides insights into the selection of source domains for transfer learning strategies under the same region.

An End-to-end Domain Generalization Fault Diagnosis Method Based on Adaptive Weighted Domain Adversarial Learning

Domain adaptation (DA) based fault diagnosis methods have attracted great attention for a long time, which leverages domain-invariant knowledge from a source domain (SD) to a special target domain (TD). However, in practise, there not usually exists only single SD and a specific and available TD. Mostly, data is generated from multiple domain distributions due to varied working conditions, and the TD is not always available during model training. Hence, how to use these multiple domain distribution data to build a model to generalize domain knowledge to an “unseen” TD, is an emerging and challengeable issue. In this paper, a novel domain generalization (DG) based fault diagnosis methods is proposed. Multiple domains are considered jointly to extract generalized domain-invariant features. A multiple residual blocks structure is adopted as the main feature extractor, an additional extractor is added to extract prior knowledge from statistic features. Besides, a k-means based adaptive weighted domain adversarial learning is designed to realize multiple domain confusion in a latent space while the wasserstein distance is calculated to assist in narrowing discrepancy between different domain distributions. Experiments of multiple domain distributions tasks from two datasets are conducted to evaluate the proposed method, and the results have verified the effectiveness of proposed method.

Unsupervised Domain Adaptation via Weighted Sequential Discriminative Feature Learning for Sentiment Analysis

Unsupervised domain adaptation (UDA) presents a significant challenge in sentiment analysis, especially when faced with differences between source and target domains. This study introduces Weighted Sequential Unsupervised Domain Adaptation (WS-UDA), a novel sequential framework aimed at discovering more profound features and improving target representations, even in resource-limited scenarios. WS-UDA utilizes a domain-adversarial learning model for sequential discriminative feature learning. While recent UDA techniques excel in scenarios where source and target domains are closely related, they struggle with substantial dissimilarities. This potentially leads to instability during shared-feature learning. To tackle this issue, WS-UDA employs a two-stage transfer process concurrently, significantly enhancing model stability and adaptability. The sequential approach of WS-UDA facilitates superior adaptability to varying levels of dissimilarity between source and target domains. Experimental results on benchmark datasets, including Amazon reviews, FDU-MTL datasets, and Spam datasets, demonstrate the promising performance of WS-UDA. It outperforms state-of-the-art cross-domain unsupervised baselines, showcasing its efficacy in scenarios with dissimilar domains. WS-UDA’s adaptability extends beyond sentiment analysis, making it a versatile solution for diverse text classification tasks.

Domain-Adversarial Learning for UWB NLOS Identification in Dynamic Obstacle Environments

Domain-Adversarial Learning for UWB NLOS Identification in Dynamic Obstacle Environments

Bearing Fault Diagnosis Method Based on Adversarial Transfer Learning for Imbalanced Samples of Portal Crane Drive Motor

Due to their unique structural design, portal cranes have been extensively utilized in bulk cargo and container terminals. The bearing fault of their drive motors is a critical issue that significantly impacts their operational efficiency. Moreover, the problem of imbalanced fault samples has a more pronounced influence on the application of novel fault diagnosis methods. To address this, the paper presents a new method called bidirectional gated recurrent domain adversarial transfer learning (BRDATL), specifically designed for imbalanced samples from portal cranes’ drive motor bearings. Initially, a bidirectional gated recurrent unit (Bi-GRU) is used as a feature extractor within the network to comprehensively extract features from both source and target domains. Building on this, a new Correlation Maximum Mean Discrepancy (CAMMD) method, integrating both Correlation Alignment (CORAL) and Maximum Mean Discrepancy (MMD), is proposed to guide the feature generator in providing domain-invariant features. Considering the real-time data characteristics of portal crane drive motor bearings, we adjusted the CWRU and XJTU-SY bearing datasets and conducted comparative experiments. The experimental results show that the accuracy of the proposed method is up to 99.5%, which is obviously higher than other methods. The presented fault diagnosis model provides a practical and theoretical framework for diagnosing faults in portal cranes’ field operation environments.

Domain adaptation with label-aligned sampling (DALAS) for cross-domain fault diagnosis of rotating machinery under class imbalance

Though existing cross-domain fault-diagnosis methods have shown promising results under domain shift conditions, existing approaches are only valid for class-balanced data. However, situations of class imbalance are inevitable in industrial fields, due to the difficulty in acquiring fault data from in-use machinery. Thus, if existing cross-domain fault diagnosis approaches are directly applied when domain shift and class imbalance coexist, performance degradation can occur. Thus, this research develops a domain adversarial learning network for class-imbalanced data to address situations where class imbalance and domain shift coexist; this situation is referred to as the problem of class imbalance domain adaptation (CIDA). In the proposed method, domain adversarial training is implemented for learning domain-invariant features by reducing the domain shift, and a label-aligned sampling strategy is utilized to deal with the class imbalance. In addition, for further performance enhancement of label-aligned sampling by increasing the accuracy of pseudo labels, metric learning is introduced to enhance the feature distinctiveness by expanding the distance of samples from different classes while decreasing the distance of samples from the same class. The efficiency of the proposed method is validated by applying it to various circumstances using two bearing datasets. The proposed method demonstrates superior performance compared to conventional algorithms in addressing the CIDA problem, according to quantitative and qualitative evaluations.

RUL prediction of rolling bearings across working conditions based on multi-scale convolutional parallel memory domain adaptation network

Rolling bearings are widely used in mechanical equipment, effectively determining the failure time of rolling bearings is particularly significant to ensure the safe performance of mechanical equipment. However, in industrial scenarios, the machine mainly works in the normal state for a long time, it is difficult to accumulate the same distribution of the whole life data, but the use of different distribution of data for forecasting will reduce the performance of deep learning-based prediction methods. Therefore, in order to tackle this problem, a multi-scale convolutional parallel memory domain adaptation network is investigated to forecast the residual useful life (RUL) of bearings across working conditions. Firstly, a new characteristic extractor—multi-scale convolutional parallel memory network is designed to extract spatial and temporal characteristics of bearing degradation data. At the same time, in order to minimize the distribution difference between source domain and target domain, a temporal-spatial feature alignment strategy is proposed to obtain domain invariable characteristics by combining maximum mean difference and domain adversarial learning. Finally, the availability of the proposed approach is verified using two rolling bearing data sets. The results reveal that it can efficiently forecast the RUL of rolling bearings across working conditions.

Boosting SAR Aircraft Detection Performance with Multi-Stage Domain Adaptation Training

Deep learning has achieved significant success in various synthetic aperture radar (SAR) imagery interpretation tasks. However, automatic aircraft detection is still challenging due to the high labeling cost and limited data quantity. To address this issue, we propose a multi-stage domain adaptation training framework to efficiently transfer the knowledge from optical imagery and boost SAR aircraft detection performance. To overcome the significant domain discrepancy between optical and SAR images, the training process can be divided into three stages: image translation, domain adaptive pretraining, and domain adaptive finetuning. First, CycleGAN is used to translate optical images into SAR-style images and reduce global-level image divergence. Next, we propose multilayer feature alignment to further reduce the local-level feature distribution distance. By applying domain adversarial learning in both the pretrain and finetune stages, the detector can learn to extract domain-invariant features that are beneficial to the learning of generic aircraft characteristics. To evaluate the proposed method, extensive experiments were conducted on a self-built SAR aircraft detection dataset. The results indicate that by using the proposed training framework, the average precision of Faster RCNN gained an increase of 2.4, and that of YOLOv3 was improved by 2.6, which outperformed other domain adaptation methods. By reducing the domain discrepancy between optical and SAR in three progressive stages, the proposed method can effectively mitigate the domain shift, thereby enhancing the efficiency of knowledge transfer. It greatly improves the detection performance of aircraft and offers an effective approach to address the limited training data problem of SAR aircraft detection.