Adversarial Module Research Articles

Adversarial Spatiotemporal Contrastive Learning for Electrocardiogram Signals.

Extracting invariant representations in unlabeled electrocardiogram (ECG) signals is a challenge for deep neural networks (DNNs). Contrastive learning is a promising method for unsupervised learning. However, it should improve its robustness to noise and learn the spatiotemporal and semantic representations of categories, just like cardiologists. This article proposes a patient-level adversarial spatiotemporal contrastive learning (ASTCL) framework, which includes ECG augmentations, an adversarial module, and a spatiotemporal contrastive module. Based on the ECG noise attributes, two distinct but effective ECG augmentations, ECG noise enhancement, and ECG noise denoising, are introduced. These methods are beneficial for ASTCL to enhance the robustness of the DNN to noise. This article proposes a self-supervised task to increase the antiperturbation ability. This task is represented as a game between the discriminator and encoder in the adversarial module, which pulls the extracted representations into the shared distribution between the positive pairs to discard the perturbation representations and learn the invariant representations. The spatiotemporal contrastive module combines spatiotemporal prediction and patient discrimination to learn the spatiotemporal and semantic representations of categories. To learn category representations effectively, this article only uses patient-level positive pairs and alternately uses the predictor and the stop-gradient to avoid model collapse. To verify the effectiveness of the proposed method, various groups of experiments are conducted on four ECG benchmark datasets and one clinical dataset compared with the state-of-the-art methods. Experimental results showed that the proposed method outperforms the state-of-the-art methods.

Cycle-Consistent Adversarial chest X-rays Domain Adaptation for pneumonia diagnosis

To deal with the challenge of limited annotated data for training target domain pneumonia classifier, domain adaptation methods leverage publicly available source dataset to enhance performance on the target domain. However, the existing methods lack effective feature representation to ensure a significant similarity between the source domain and the target domain. To address this problem, we propose a novel method, called Cycle-Consistent Adversarial chest X-rays Domain Adaptation (C2ADA) to diagnose pneumonia from Chest X-rays automatically, which can transfer knowledge from a publicly available large-scale source dataset to the small-scale target dataset and achieve high performance with fewer target domain samples. Specifically, C2ADA introduces the cycle-consistent adversarial module to enforce the feature extractor to learn domain-invariant features. Furthermore, unlike most existing domain adaptation methods that deal with the same tasks in the source domain and target domain, our perspective is that heterogeneous tasks can assist the target domain in learning more effectively. Therefore, C2ADA includes two subnetworks, one is for the source domain, which contains a multilabel classification task, and the other is for the target domain, which focuses on the binary classification task. To assess the effectiveness of our method, we conduct a comparative analysis with other domain adaptation approaches. The experimental results show that the proposed method can achieve 96.86% accuracy on the ChestXRay2017 dataset utilizing just 50 images.

Geo-Hgan: Unsupervised anomaly detection in geochemical data via latent space learning

Reconstructing geochemical data for anomaly detection using Generative Adversarial Networks (GANs) has become a prevalent method in identifying geochemical anomalies. However, injecting random noise into GANs can induce model instability. To mitigate this issue, we propose a novel anomaly detection model, Geo-Hgan, which integrates a dual adversarial network architecture with a Latent Space Adversarial Module (LSAM) to learn the distribution of latent variables from arbitrary data and optimize the sample reconstruction process, thereby alleviating instability during GAN training. Additionally, an encoder guided by the LSAM-pretrained GAN is employed to extract variational features, facilitating rapid and effective sample mapping into the latent space defined by LSAM. Experimental results demonstrate that under unsupervised conditions, Geo-Hgan achieves an Area Under the Curve (AUC) score of 85% across three geochemical datasets, outperforming similar models in accuracy and reconstruction capabilities. To assess its versatility and generalization ability, we extend Geo-Hgan to anomaly detection tasks in computer vision, where it achieves an average AUC score of 98.7% on the MvtecAD dataset, setting a new state-of-the-art performance in the domain. Furthermore, we propose AnomFilter, a method for setting anomaly thresholds based on the clustering hypothesis. AnomFilter identifies high-confidence anomaly samples identified by Geo-Hgan in the source domain and iteratively transfers them to the target domain. These high-confidence anomaly samples, combined with a small number of known positive samples in the target domain, enhance the accuracy of supervised geochemical anomaly detection in the target domain, which achieved an AUC score of 94%. The utilization of anomaly detection models for sample transfer learning offers a novel perspective for future work.

DF-CDM: Conditional diffusion model with data fusion for structural dynamic response reconstruction

In structural health monitoring (SHM) systems, data loss inevitably occurs and reduces the applicability of SHM techniques, such as condition assessment and damage identification. The current mainstream data-driven method, generative adversarial networks (GAN), suffers from convergence difficulty, limiting the accuracy and efficiency of response reconstruction. In this study, a conditional diffusion model with data fusion (DF-CDM) is proposed for structural dynamic response reconstruction. The original unsupervised diffusion model is improved by introducing the conditional input and modifying the deep denoising neural network to achieve supervised learning. Besides, data fusion is developed to further utilize the frequency-domain information and improve the reconstruction quality of the time-domain signals. The proposed model is validated on a three-span continuous bridge. Results show that the diffusion model with data fusion achieves the highest accuracy on the test set with R2, RMSE and MAE equaling 0.821, 0.0053 and 0.0042 respectively. Compared with the state-of-the-art GAN model, the diffusion model without the adversarial modules has a more stable training process and better reconstruction performance in both time and frequency domains. The modal identification and robustness analysis further verify the effectiveness of the proposed model. The proposed diffusion model with data fusion achieves high-quality structural response reconstruction, guaranteeing the applicability and reliability of subsequent response-based structural analysis.

Dual-Pathway Deep Hashing-Based Adversarial Learning for Cross-Modal Retrieval

The application of deep hashing methods for cross-modal retrieval has seen growing interest due to their storage efficiency and fast query execution. However, the challenge posed by the “heterogeneity gap” in multi-modal datasets cannot be understated. To address this, we present a novel framework named Dual-Pathway Deep Hashing-Based Adversarial Learning (DP-DHAL), engineered to surmount this challenge. The architecture of DP-DHAL integrates three key components: (a) a dual-pathway representation learning module tasked with extracting modality-specific features; (b) an adversarial module working to align the distributions of cross-modal features; and (c) a deep hashing module responsible for generating hash codes that uphold the similarity relationships across different modalities. Additionally, we have developed a unique Hamming triplet-margin loss function to refine the assessment of content similarities. The DP-DHAL model is trained through an adversarial process where the adversarial module’s goal is to discern cross-modal features with the aim of reducing the heterogeneity gap. Simultaneously, the representation learning module is focused on producing representations that can both deceive the adversarial module and preserve cross-modal similarities to yield distinctive hash codes. Comprehensive experiments on varied datasets have shown that our proposed method outperforms other leading cross-modal hashing techniques.

DDAGCN: an unsupervised cross-domain identification method for tie rod bolt loosening in a rod-fastening rotor system under different working conditions

The cross-domain diagnosis of tie rod bolt loosening is essential for guaranteeing the healthy operation of rod-fastening rotor (RFR) systems. The unsupervised domain adaptation (UDA) method effectively alleviates the impact of domain discrepancy and has been applied for cross-domain diagnosis. Traditional UDA methods mainly focus on the marginal and conditional distributions with fixed weights to adapt the domain distribution discrepancy. However, the fixed distribution combination cannot satisfy the requirement of feature domain alignment under different working conditions, and the relative importance of the two distributions cannot be evaluated quantitatively. This paper proposes an improved dynamic distribution adaptive graph convolutional network (DDAGCN) for the cross-domain diagnosis of tie rod bolt loosening under different working conditions. This method can quantitatively evaluate the relative significance of each distribution in representing the distribution discrepancy. First, it combines the convolutional neural network and the graph convolutional network to extract the features in the graph structure by using the connection relationship between nodes, and realizes the full extraction of neighbourhood information of nodes. Then, the dynamic distribution adaptive alignment strategy is introduced to construct the dynamic linear combination of marginal and conditional distributions, so as to measure the distribution discrepancy between domains. Meanwhile, the domain adversarial module is combined to further reduce the domain gap and finally realize feature alignment. The extracted domain invariant features can effectively enhance the generalization ability and fault identification ability of the model. The case of the public bearing dataset verifies that the effectiveness and generalization ability of the proposed method for cross-domain fault diagnosis under different working conditions is superior to other compared methods. In addition, the identification ability of the proposed method for the degree of tie rod bolt loosening is verified by the self-made bolt loosening dataset of the RFR system.

Adversarial Secret-Identity Generation Model for Face Anonymization in the Internet of Vehicles

With the rapid development of the Internet of Vehicles, the systematical cameras in vehicles can easily collect mass personal face data, which is very private on the Internet. This article focuses on solving the face protection problem caused by the Internet of Vehicles. Previous studies have proposed various face protection models based on obfuscation, neural networks, or adversarial techniques to prevent the disclosure of identity information. Even with existing face anonymization algorithms, there are several issues, including weak information protection ability, not maintaining the naturalness of a protected face, and not being suitable for various emerging face recognition systems. We propose an adversarial secret-identity generation model to protect personal identities from these problems. Concretely, our model's identity encryption generative adversarial module can generate brand new faces, which ensures naturalness while making other face identification models unable to recognize the identity. In addition, our PFS module extracts the identity set that is least relevant to the inputs and most conducive to model convergence from target data and promotes the generation of excellent faces from the perspective of data. Finally, the protected identities generated by our black-box face deidentification model can resist recognizing various advanced face identification approaches. Extensive experiments prove our model can achieve a 96.5% protection success rate, achieving competitive performance with 98.1% R1-T, 99.2% R5-T, 98.5% R1-U, and 97.1% R5-U against ArcFace and beating out various superior face anonymization algorithms. Meanwhile, we introduce a model compression algorithm to reduce the network scale of this model, demonstrating the scalability of various intelligent vehicles.

MALN: Multimodal Adversarial Learning Network for Conversational Emotion Recognition

Multimodal emotion recognition in conversations (ERC) aims to identify the emotional state of constituent utterances expressed by multiple speakers in dialogue from multimodal data. Existing multimodal ERC approaches focus on modeling the global context of the dialogue and neglect to mine the characteristic information from the corresponding utterances expressed by the same speaker. Additionally, information from different modalities exhibits commonality and diversity for emotional expression. The commonality and diversity of multimodal information are compensated for each other but not effectively exploited in previous multimodal ERC works. To tackle these issues, we propose a novel Multimodal Adversarial Learning Network (MALN). MALN first mines the speaker’s characteristics from context sequences and then incorporate them with the unimodal features. Afterward, we design a novel adversarial module AMDM to exploit both commonality and diversity from the unimodal features. Finally, AMDM fuses different modalities to generate refined utterance representations for emotion classification. Extensive experiments are conducted on two public multimodal ERC datasets, IEMOCAP and MELD. Through the experiments, MALN shows its superiority over the state-of-the-art methods.

An improved unified domain adversarial category-wise alignment network for unsupervised cross-domain sentiment classification

Cross-domain sentiment analysis (CDSA) aims to learn transferable knowledge from the source domain to facilitate the sentiment polarity classification on the target domain of lacking labeled data. Currently, two types of unsupervised domain adaptation (UDA) methods are widely used in CDSA tasks. One employs the domain adversarial strategy to extract domain-invariant features, and the other utilizes the distance metric strategy to reduce domain distribution discrepancy. However, the fine-grained domain-specific information related to categories aligned between domains is not preserved, which suppresses the performance of target-domain classification. To overcome the mentioned problem, a unified Domain Adversarial Category-wise Alignment Network (DACAN) was proposed in this paper. An integrated network was constructed with progressive multi-level feature learning. Specifically, a feature extraction module was constructed with parameter sharing between two domains at low-level text feature extraction layers. The domain adversarial module was added to enable shared knowledge transfer by extracting domain-invariant information and by updating the shared parameters at the feature extraction layers. A category-wise alignment module was built to achieve local distribution alignment at the high dimension-level semantic layers guided by fine-grained category structure information. Meanwhile, joint constraint was established with domain-invariant constraint based on domain adversarial, and domain-consistency constraint based on category-wise alignment. Comprehensive experiments were conducted on two standard Amazon review datasets. The results show that DACAN outperforms other state-of-the-art UDA methods by 0.7% and 1.1% on the two- and three-category CDSA tasks, respectively. Also, better performance results are achieved with a synergistic UDA scheme compared with a single UDA scheme.

Coronary heart disease prediction method fusing domain-adaptive transfer learning with graph convolutional networks (GCN)

Graph convolutional networks (GCNs) have achieved impressive results in many medical scenarios involving graph node classification tasks. However, there are difficulties in transfer learning for graph representation learning and graph network models. Most GNNs work only in a single domain and cannot transfer the learned knowledge to other domains. Coronary Heart Disease (CHD) is a high-mortality disease, and there are non-public and significant differences in CHD datasets for current research, which makes it difficult to perform unified transfer learning. Therefore, in this paper, we propose a novel adversarial domain-adaptive multichannel graph convolutional network (DAMGCN) that can perform graph transfer learning on cross-domain tasks to achieve cross-domain medical knowledge transfer on different CHD datasets. First, we use a two-channel GCN model for feature aggregation using local consistency and global consistency. Then, a uniform node representation is generated for different graphs using an attention mechanism. Finally, we provide a domain adversarial module to decrease the discrepancies between the source and target domain classifiers and optimize the three loss functions in order to accomplish source and target domain knowledge transfer. The experimental findings demonstrate that our model performs best on three CHD datasets, and its performance is greatly enhanced by graph transfer learning.

Consistent graph embedding network with optimal transport for incomplete multi-view clustering

Existing incomplete multi-view learning models focus on reconstructing the latent variables of multiple views by exploring complementary and consistent information among diverse views. However, filling the missing information for views results in a loss of consistency, while fusion and reconstruction between views face over-fitting problems. The optimal transport algorithm delicately measures the distance of two distributions, resulting in decreased reconstruction errors and guaranteeing consistency and complementarity across multiple views of the data. In light of this, this paper proposes the incorporation of the optimal transport algorithm into the framework of incomplete multi-view clustering. The proposed consistent graph embedding network (CGEN-OT) via optimal transport combines the adversarial module and fusion module to learn a completed latent graph embedding. Specifically, CGEN-OT utilizes an adversarial module to generate complete views and fuses them into a consistent embedding, and introduces reconstruction loss and Sinkhorn loss to jointly optimize the proposed network and obtain superior latent graph embedding and clustering performance. To further validate the clustering accuracy and convergence of the CGEN-OT, experimental evaluation was conducted on six distinct incomplete datasets. A comparison with existing state-of-the-art models highlights the superiority of the proposed framework.

A label information vector generative zero-shot model for the diagnosis of compound faults

Diagnosis of compound faults remains a challenge owing to the coupling of fault characteristics and the exponential increment of the number of possible fault types. Current compound faults diagnostic methods often require a large number of training data for each type of compound fault. In real-world scenarios, training data of compound faults are usually difficult to acquire and sometimes even inaccessible. In contrast, single fault samples are much easier to obtain. Thus in this paper inspired by the idea of zero-shot learning, we present a novel label information vector generative zero-shot model to identify unknown compound faults, using only single fault samples as the training set. This model comprises several modules, namely label information vector (LIV) definition, feature extractor, and generative adversarial modules, respectively responsible for representing the prior knowledge of specific class labels for the single fault and compound fault, extraction of fault features, and mapping the relationship between the fault features and the fault LIVs. By adversarial training between the samples and LIVs of single faults, the model can generate compound fault features using the compound fault LIVs. Thus the unknown compound faults are identified by measuring the distance between the features extracted from the testing compound fault samples and the generated features from LIVs. The proposed method is evaluated on a self-built experimental platform. The results demonstrate that without any compound fault samples in the training set, the compound fault classification accuracy of the model reaches 78.10%.

Adversarial Alignment for Source Free Object Detection

Source-free object detection (SFOD) aims to transfer a detector pre-trained on a label-rich source domain to an unlabeled target domain without seeing source data. While most existing SFOD methods generate pseudo labels via a source-pretrained model to guide training, these pseudo labels usually contain high noises due to heavy domain discrepancy. In order to obtain better pseudo supervisions, we divide the target domain into source-similar and source-dissimilar parts and align them in the feature space by adversarial learning.Specifically, we design a detection variance-based criterion to divide the target domain. This criterion is motivated by a finding that larger detection variances denote higher recall and larger similarity to the source domain. Then we incorporate an adversarial module into a mean teacher framework to drive the feature spaces of these two subsets indistinguishable. Extensive experiments on multiple cross-domain object detection datasets demonstrate that our proposed method consistently outperforms the compared SFOD methods. Our implementation is available at https://github.com/ChuQiaosong.

Neighborhood-based credibility anchor learning for universal domain adaptation

Universal domain adaptation (UniDA) aims to transfer knowledge from the source domain to the target domain in the presence of distribution shift and class mismatch. Most existing works design threshold-relied methods to reject target private classes by carefully-proposed uncertainty scoring functions which are very sensitive to thresholds. To overcome this problem, a few threshold-free methods are proposed but ignore the neighborhood structure information of the target domain, leading to poor performance. In this paper, we propose Neighborhood-based Credibility Anchor Learning (NCAL), a new threshold-free framework that fully mines the neighborhood structure information to explore better target representations. NCAL contains three key components: a class anchor learning module to learn target class distribution, a credibility-weighted conditional adversarial module to learn class-invariant features of common classes, and an open-set neighborhood clustering module to learn well-clustered features. Extensive experiments demonstrate that our method outperforms the state-of-the-art.

Collaborative fault diagnosis of rotating machinery via dual adversarial guided unsupervised multi-domain adaptation network

Most of the existing research on unsupervised cross-domain intelligent fault diagnosis is based on single-source domain adaptation, which fails to simultaneously utilize various source domains with enough and diverse diagnostic information in practical application scenarios. How to better extract common features from multiple domains and integrate multi-source domain knowledge for collaborative diagnosis is a main challenge. To address these problems, a dual adversarial guided unsupervised multi-domain adaptation network (DAG-MDAN) is proposed. Within the proposed framework, the edge adversarial module (EA-Module) in each set of sources-target domain adaptation sub-network is utilized to compute the source-target domain adversarial loss. And an inner adversarial module (IA-Module) is constructed to direct the extraction of common features between multi-source domains, which combined the EA-Module to form the dual adversarial training to enhance domain confusion. Besides, a multi-subnet collaborative decision module (MCD-Module) is designed to compute the confidence scores to assists the multi-subnet classifier to make better fusion decisions. The DAG-MDAN is verified by the several transfer tasks using faulty rotating machinery datasets under the different speed conditions.

Cross-Domain Few-Shot Hyperspectral Image Classification With Class-Wise Attention

Few-shot learning (FSL) is an effective method to solve the problem of hyperspectral image (HSI) classification with few labeled samples. It learns transferable knowledge from sufficient labeled auxiliary data to classify unseen classes with limited labeled samples for training. However, the distribution difference between auxiliary data and unseen classes results in the learned transferable knowledge not being well applied to the new task. Therefore, a class-wise attentive cross-domain FSL (CA-CFSL) framework is proposed in this article, in which a feature extractor is learned to extract data features with discriminability and domain invariance. The class-wise attention metric module (CAMM) introduces class-wise attention on the FSL framework to learn more discriminative features, which improves the interclass decision boundaries. Furthermore, an asymmetric domain adversarial module (ADAM) is designed to enhance the ability of extracting domain-invariant representations, which combines asymmetric adversarial training with embedded domain-specific information. Experimental results on four public HSI datasets demonstrate that the proposed method outperforms the existing methods.

SFDA: Domain Adaptation With Source Subject Fusion Based on Multi-Source and Single-Target Fall Risk Assessment.

In cross-subject fall risk classification based on plantar pressure, a challenge is that data from different subjects have significant individual information. Thus, the models with insufficient generalization ability can't perform well on new subjects, which limits their application in daily life. To solve this problem, domain adaptation methods are applied to reduce the gap between source and target domain. However, these methods focus on the distribution of the source and the target domain, but ignore the potential correlation among multiple source subjects, which deteriorates domain adaptation performance. In this paper, we proposed a novel method named domain adaptation with subject fusion (SFDA) for fall risk assessment, greatly improving the cross-subject assessment ability. Specifically, SFDA synchronously carries out source target adaptation and multiple source subject fusion by domain adversarial module to reduce source-target gap and distribution distance within source subjects of same class. Consequently, target samples can learn more task-specific features from source subjects to improve the generalization ability. Experiment results show that SFDA achieved mean accuracy of 79.17 % and 73.66 % based on two backbones in a cross-subject classification manner, outperforming the state-of-the-art methods on continuous plantar pressure dataset. This study proves the effectiveness of SFDA and provides a novel tool for implementing cross-subject and few-gait fall risk assessment.

DMSTL: A Deep Multi-Scale Transfer Learning Framework for Unsupervised Cross-Position Human Activity Recognition

Human activity recognition (HAR) based on wearable sensors has been a prosperous research topic in recent years. Considering that the sensor may be worn at diverse body positions, obtaining enough labeled human activity data for each body-worn position is usually expensive and labor-intensive. Furthermore, the variability and diversity of data distribution induced by different body-worn positions make the HAR model trained on data collected from one body position perform poorly for other body-worn positions. To achieve accurate HAR with low labeling cost, which we call unsupervised cross-position HAR, in this article, we propose a deep multiscale transfer learning (DMSTL) model. In the model, we first introduce an unsupervised source selection method to select the most similar source domain for transferring domain knowledge. Then, we develop a multiscale spatial–temporal Net (MSSTNet) to learn comprehensive multimodal representations from multiple feature subspaces. Finally, we design the category-level adaptation module and the domain-level adversarial module for learning domain-invariant features. We conduct extensive experiments on three public HAR data sets and demonstrate the reasonable generalization performance of the DMSTL, which remarkably outperforms other state-of-the-art baselines.

Learning for Data Synthesis: Joint Local Salient Projection and Adversarial Network Optimization for Vehicle Re-Identification

The problem of vehicle re-identification in surveillance scenarios has grown in popularity as a research topic. Deep learning has been successfully applied in re-identification tasks in the last few years due to its superior performance. However, deep learning approaches require a large volume of training data, and it is particularly crucial in vehicle re-identification tasks to have a sufficient amount of varying image samples for each vehicle. To collect and construct such a large and diverse dataset from natural environments is labor intensive. We offer a novel image sample synthesis framework to automatically generate new variants of training data by augmentation. First, we use an attention module to locate a local salient projection region in an image sample. Then, a lightweight convolutional neural network, the parameter agent network, is responsible for generating further image transformation states. Finally, an adversarial module is employed to ensure that the images in the dataset are distorted, while retaining their structural identities. This adversarial module helps to generate more appropriate and difficult training samples for vehicle re-identification. Moreover, we select the most difficult sample and update the parameter agent network accordingly to improve the performance. Our method draws on the adversarial networks strategy and the self-attention mechanism, which can dynamically decide the region selection and transformation degree of the synthesis images. Extensive experiments on the VeRi-776, VehicleID, and VERI-Wild datasets achieve good performance. Specifically, our method outperforms the state-of-the-art in MAP accuracy on VeRi-776 by 2.15%. Moreover, on VERI-Wil, a significant improvement of 7.15% is achieved.

A text matching model based on dynamic multi‐mask and augmented adversarial

AbstractThe text matching is a basic task of NLP and is important for tasks such as text retrieval, question answering, and so forth. The development of pre‐trained language models has promoted the progress of text matching tasks. However, due to the natural particularity of Chinese characters and expressions, the Chinese text matching tasks still have problems such as word segmentation difficulty, serious semantic loss, and model instability. In this paper, we propose the DAINet model, which includes DMM, AA and IO modules. We use the Dynamic Multi‐Mask module (DMM) to enhance the completeness of word segmentation. Then we use the Augmented Adversarial module (AA) to further extraction of semantic information. Finally, we use the Integrated Output module (IO) for a more stable output. We conducted experiments on LCQMC, BQ and Xiaobu datasets and compared the results with seven strong baseline models. The results showed that DAINet model made great improvement, including improving ACC value of BQ dataset to , AUC value to , ACC value of LCQMC dataset to and AUC value to . The ACC value of Xiaobu dataset was improved to and the AUC value was improved to . Further ablation experiment results show that the proposed DMM, AA and IO modules have good adaptability and improvement over existing models.