Adversarial Loss Research Articles

Digital Camouflage Generation Based on an Improved CycleGAN Network Model

Abstract This paper proposes a digital camouflage generation method based on an improved CycleGAN to produce camouflage patterns with a high degree of fusion with the background and realistic texture details. Firstly, a SE-ResNet network structure is constructed by combining the residual network ResNet with the channel attention mechanism SENet, enabling flexible adjustment of channel weights to effectively extract crucial channel features and enhance the network's perception capability of important information in images. Secondly, a color preservation loss is introduced to improve the adversarial loss function, thereby avoiding training instability and fluctuation in pattern quality. Experimental results demonstrate that the camouflage patterns generated using the proposed method achieve a Structural Similarity Index (SSIM) of 0.77 and a Peak Signal-to-Noise Ratio (PSNR) of 18.9, representing improvements of 0.27 and 3.3, respectively, compared to the original CycleGAN. This method can generate digital camouflage patterns with richer details, textures, and high fusion with the background.

Skeleton-aware Graph-based Adversarial Networks for Human Pose Estimation from Sparse IMUs

Recently, sparse-inertial human pose estimation (SI-HPE) with only a few IMUs has shown great potential in various fields. The most advanced work in this area achieved fairish results using only six IMUs. However, there are still two major issues that remain to be addressed. First, existing methods typically treat SI-HPE as a temporal sequential learning problem and often ignore the important spatial prior of skeletal topology. Second, there are far more synthetic data in their training data than real data, and the data distribution of synthetic data and real data is quite different, which makes it difficult for the model to be applied to more diverse real data. To address these issues, we propose “Graph-based Adversarial Inertial Poser (GAIP)”, which tracks body movements using sparse data from six IMUs. To make full use of the spatial prior, we design a multi-stage pose regressor with graph convolution to explicitly learn the skeletal topology. A joint position loss is also introduced to implicitly mine spatial information. To enhance the generalization ability, we propose supervising the pose regression with an adversarial loss from a discriminator, bringing the ability of adversarial networks to learn implicit constraints into full play. Additionally, we construct a real dataset that includes hip support movements and a synthetic dataset containing various motion categories to enrich the diversity of inertial data for SI-HPE. Extensive experiments demonstrate that GAIP produces results with more precise limb movement amplitudes and relative joint positions, accompanied by smaller joint angle and position errors compared to state-of-the-art counterparts. The datasets and codes are publicly available at https://cslinzhang.github.io/GAIP/ .

DAE-GAN: Underwater Image Super-Resolution Based on Symmetric Degradation Attention Enhanced Generative Adversarial Network

Underwater images often exhibit detail blurring and color distortion due to light scattering, impurities, and other influences, obscuring essential textures and details. This presents a challenge for existing super-resolution techniques in identifying and extracting effective features, making high-quality reconstruction difficult. This research aims to innovate underwater image super-resolution technology to tackle this challenge. Initially, an underwater image degradation model was created by integrating random subsampling, Gaussian blur, mixed noise, and suspended particle simulation to generate a highly realistic synthetic dataset, thereby training the network to adapt to various degradation factors. Subsequently, to enhance the network’s capability to extract key features, improvements were made based on the symmetrically structured blind super-resolution generative adversarial network (BSRGAN) model architecture. An attention mechanism based on energy functions was introduced within the generator to assess the importance of each pixel, and a weighted fusion strategy of adversarial loss, reconstruction loss, and perceptual loss was utilized to improve the quality of image reconstruction. Experimental results demonstrated that the proposed method achieved significant improvements in peak signal-to-noise ratio (PSNR) and underwater image quality measure (UIQM) by 0.85 dB and 0.19, respectively, significantly enhancing the visual perception quality and indicating its feasibility in super-resolution applications.

A sea–land clutter classification framework for over-the-horizon radar based on weighted loss semi-supervised generative adversarial network

Deep convolutional neural network has made great achievements in sea–land clutter classification for over-the-horizon radar (OTHR). The premise is that a large number of labeled training samples must be provided for a sea–land clutter classifier. In practical engineering applications, it is relatively easy to obtain label-free sea–land clutter samples. However, the labeling process is extremely cumbersome and requires expertise in the field of OTHR. To solve this problem, we propose an improved generative adversarial network, namely weighted loss semi-supervised generative adversarial network (WL-SSGAN). Specifically, we propose a joint feature matching loss by weighting the middle layer features of the discriminator of semi-supervised generative adversarial network (SSGAN). Furthermore, we propose the weighted loss of WL-SSGAN by linearly weighting the standard adversarial loss of SSGAN and the joint feature matching loss. The classification performance of WL-SSGAN is evaluated on sea–land clutter datasets. The experimental results show that WL-SSGAN can improve the performance of the fully supervised classifier with only a small number of labeled samples by utilizing a large number of unlabeled sea–land clutter samples. Further, the proposed weighted loss is superior to both the adversarial loss and the feature matching loss. Additionally, we compare WL-SSGAN with conventional semi-supervised classification methods and demonstrate that WL-SSGAN achieves the highest classification accuracy.

Generative Adversarial Network for Real‐Time Flash Drought Monitoring: A Deep Learning Study

AbstractDroughts are among the most devastating natural hazards, occurring in all regions with different climate conditions. The impacts of droughts result in significant damages annually around the world. While drought is generally described as a slow‐developing hazardous event, a rapidly developing type of drought, the so‐called flash drought has been revealed by recent studies. The rapid onset and strong intensity of flash droughts require accurate real‐time monitoring. Addressing this issue, a Generative Adversarial Network (GAN) is developed in this study to monitor flash droughts over the Contiguous United States (CONUS). GAN contains two models: (a) discriminator and (b) generator. The developed architecture in this study employs a Markovian discriminator, which emphasizes the spatial dependencies, with a modified U‐Net generator, tuned for optimal performance. To determine the best loss function for the generator, four different networks are developed with different loss functions, including Mean Absolute Error (MAE), adversarial loss, a combination of adversarial loss with Mean Square Error (MSE), and a combination of adversarial loss with MAE. Utilizing daily datasets collected from NLDAS‐2 and Standardized Soil Moisture Index (SSI) maps, the network is trained for real‐time daily SSI monitoring. Comparative assessments reveal the proposed GAN's superior ability to replicate SSI values over U‐Net and Naïve models. Evaluation metrics further underscore that the developed GAN successfully identifies both fine‐ and coarse‐scale spatial drought patterns and abrupt changes in the SSI temporal patterns that is important for flash drought identification.

CFOA: Exploring transferable adversarial examples by content feature optimization with attention guidance

Deep neural networks are known to be highly susceptible to adversarial examples, created by adding small and carefully-designed perturbations on original images. In black-box scenarios, attacking neural networks is challenging since the information about models is not available. Generating transferable adversarial examples on surrogate models is a viable way. However, current mainstream transfer-based attacks seldomly consider the relationship between transferability and content features of the image. To effectively enhance the attack success rate at the feature level, we newly propose an attack method called Content Feature Optimization Attack (CFOA). This novel CFOA approach operates on white-box models, aiming to generate transferable adversarial examples by targeting the content feature representations of images in the feature space. The method involves the extraction and parameterization of content features, followed by the iterative generation of feature-level perturbations guided by attention loss and adversarial loss. Ultimately, the adversarial examples generated by CFOA exhibit strong transferability on other black-box models. Experimental results show that our method produces adversarial examples with higher success rates in transferability compared to state-of-the-art methods, and importantly, they can effectively evade certain defense mechanisms, presenting a challenge for adversarial defense.

Improving the spatial resolution of solar images using super-resolution diffusion generative adversarial networks

High-spatial-resolution solar images contribute to the study of small-scale structures on the Sun. The Helioseismic and Magnetic Imager (HMI) conducts continuous full-disk observations of the Sun at a fixed cadence, accumulating a wealth of observational data. However, the spatial resolution of HMI images is not sufficient to analyze the small-scale structures of solar activity. We present a new super-resolution (SR) method based on generative adversarial networks (GANs) and denoising diffusion probabilistic models (DDPMs) that can increase the spatial resolution of HMI images by a factor four. We propose a method called super-resolution diffusion GANs (SDGAN), which combines GANs and DDPMs for the SR reconstruction of HMI images. SDGAN progressively maps low-resolution (LR) images to high-resolution (HR) images through a conditional denoising process. It employs conditional GANs to simulate the denoising distribution and optimizes model results using nonsaturating adversarial loss and perceptual loss. This approach enables fast and high-quality reconstruction of solar images. We used high-spatial-resolution images from the Goode Solar Telescope (GST) as HR images and created a data set consisting of paired images from HMI and GST. We then used this data set to train SDGAN for the purpose of reconstructing HMI images with four times the original spatial resolution. The experimental results demonstrate that SDGAN can obtain high-quality HMI reconstructed images with just four denoising steps.

SVCGAN: Speaker Voice Conversion Generative Adversarial Network for Children’s Speech Conversion and Recognition

Automatic speech recognition (ASR) refers to a technological process that entails the conversion of spoken language into written text. However, the acoustic distinctions between children’s speech and adult speech are substantial, rendering the automatic speech recognition system trained on adult speech inadequate for effectively recognizing children’s speech. To overcome this issue, in this study, we propose speaker conversion generative adversarial network (SVCGAN). SVCGAN is a novel non-parallel voice conversion model, which enhances three key areas: log-cosh loss, semantic-similarity loss, and third adversarial loss. Therefore, the incorporation of these losses better protects semantic information for young children during voice conversion process and improves the quality of the converted speech. Additionally, the character error rate (CER) of children’s speech recognition can benefit from children’s speech transformed into adult speech. Experimental results suggest that SVCGAN demonstrates superior performance across multiple dimensions compared to both CycleGAN-VC3 and MaskCycleGAN-VC models. It encompasses training efficiency, semantic information similarity, voice type similarity, sound naturalness and intelligibility, which leads to a reduction in the CER of speech recognition for young children.

Style transfer and recognition of infrared images of transformers under different lighting based on CycleGAN

In order to solve the problem that the transformer in the infrared image of the transformer is difficult to be accurately detected due to strong light irradiation under different lighting conditions, a method for unifying the illumination and equipment identification of the infrared image of the transformer based on CycleGAN and yolov3 is proposed. CycleGAN is used to unify the illumination intensity of the infrared image of the transformer, and an adversarial loss function and a cycle consistency loss function are introduced to ensure high-quality generation of infrared images, and the expanded infrared sample library is richer. Finally, the yolov3 network is trained using the uniformly illuminated image set to achieve accurate identification of transformers. The experimental results show that the infrared image of the transformer that has been unified by illumination can identify the transformer with an accuracy of 95.2%, which is 3.1% higher than before enhancement.

Seismic data reconstruction method using generative adversarial network based on moment reconstruction error constraint.

The seismic data acquired are usually spatially undersampled due to the constraints of the field acquisition environment. However, the removal of multiple waves, offsets, and inversions requires high regularity and integrity of seismic data. Therefore, reasonable data reconstruction methods are usually applied to the missing data in the indoor processing stage to recover regular seismic data. The traditional reconstruction methods for seismic data reconstruction are generally based on some assumptions (e.g., assuming that the data satisfies linearity or sparsity, etc.) and have some limitations of use. To overcome the applicability problem of traditional seismic data reconstruction methods, this article proposes a generative adversarial network (GAN) seismic data reconstruction method based on moment reconstruction error constraints. The method can extract the deep features of the data nonlinearly without any assumptions. First, the error function in the GAN is improved, and the commonly used joint error function of adversarial loss plus L1/L2 amplitude reconstruction loss is improved to a new error function consisting of adversarial loss and moment reconstruction loss weighting. Then, an adversarial network data reconstruction generation method based on the moment reconstruction error constraint is given. Next, an experimental analysis of different types of data missing was carried out using theoretical model data, and the study method was analyzed by interpolation errors. Finally, actual seismic data is used to further validate the effect of the research method. The experimental results show that the improved algorithm performs superiorly in dealing with the data reconstruction problem. Compared with the error function of conventional GAN optimization, the reconstruction results of GAN based on the moment reconstruction error constraint have better amplitude preservation.

MKTN: Adversarial-Based Multifarious Knowledge Transfer Network from Complementary Teachers

With the demands for light deep networks models in various front-end devices, network compression has attracted increasing interest for reducing model sizes yet without sacrificing much model accuracy. This paper presents a multifarious knowledge transfer network (MKTN) that aims to produce a compact yet powerful student network from two complementary teacher networks. Instead of learning homogeneous features, the idea is to pre-train one teacher to capture generative and low-level image features under a reconstruction objective, and another teacher to capture discriminative and task-specific features under the same objective as the student network. During knowledge transfer, the student learns multifarious and complementary knowledge from the two teacher networks under the guidance of the proposed adversarial loss and feature loss respectively. Experimental results indicate that the proposed training losses can effectively guide the student to learn spatial-level and pixel-level information as distilled from teacher networks. On the other hand, our study over a number of widely used datasets shows that transferring multifarious features from complementary teachers equipped with different types of knowledge helps to teach a compact yet powerful student effectively.

Towards Squeezing-Averse Virtual Try-On via Sequential Deformation

In this paper, we first investigate a visual quality degradation problem observed in recent high-resolution virtual try-on approach. The tendency is empirically found that the textures of clothes are squeezed at the sleeve, as visualized in the upper row of Fig.1(a). A main reason for the issue arises from a gradient conflict between two popular losses, the Total Variation (TV) and adversarial losses. Specifically, the TV loss aims to disconnect boundaries between the sleeve and torso in a warped clothing mask, whereas the adversarial loss aims to combine between them. Such contrary objectives feedback the misaligned gradients to a cascaded appearance flow estimation, resulting in undesirable squeezing artifacts. To reduce this, we propose a Sequential Deformation (SD-VITON) that disentangles the appearance flow prediction layers into TV objective-dominant (TVOB) layers and a task-coexistence (TACO) layer. Specifically, we coarsely fit the clothes onto a human body via the TVOB layers, and then keep on refining via the TACO layer. In addition, the bottom row of Fig.1(a) shows a different type of squeezing artifacts around the waist. To address it, we further propose that we first warp the clothes into a tucked-out shirts style, and then partially erase the texture from the warped clothes without hurting the smoothness of the appearance flows. Experimental results show that our SD-VITON successfully resolves both types of artifacts and outperforms the baseline methods. Source code will be available at https://github.com/SHShim0513/SD-VITON.

Teacher as a Lenient Expert: Teacher-Agnostic Data-Free Knowledge Distillation

Data-free knowledge distillation (DFKD) aims to distill pretrained knowledge to a student model with the help of a generator without using original data. In such data-free scenarios, achieving stable performance of DFKD is essential due to the unavailability of validation data. Unfortunately, this paper has discovered that existing DFKD methods are quite sensitive to different teacher models, occasionally showing catastrophic failures of distillation, even when using well-trained teacher models. Our observation is that the generator in DFKD is not always guaranteed to produce precise yet diverse samples using the existing representative strategy of minimizing both class-prior and adversarial losses. Through our empirical study, we focus on the fact that class-prior not only decreases the diversity of generated samples, but also cannot completely address the problem of generating unexpectedly low-quality samples depending on teacher models. In this paper, we propose the teacher-agnostic data-free knowledge distillation (TA-DFKD) method, with the goal of more robust and stable performance regardless of teacher models. Our basic idea is to assign the teacher model a lenient expert role for evaluating samples, rather than a strict supervisor that enforces its class-prior on the generator. Specifically, we design a sample selection approach that takes only clean samples verified by the teacher model without imposing restrictions on the power of generating diverse samples. Through extensive experiments, we show that our method successfully achieves both robustness and training stability across various teacher models, while outperforming the existing DFKD methods.

Sharpness-Aware Model-Agnostic Long-Tailed Domain Generalization

Domain Generalization (DG) aims to improve the generalization ability of models trained on a specific group of source domains, enabling them to perform well on new, unseen target domains. Recent studies have shown that methods that converge to smooth optima can enhance the generalization performance of supervised learning tasks such as classification. In this study, we examine the impact of smoothness-enhancing formulations on domain adversarial training, which combines task loss and adversarial loss objectives. Our approach leverages the fact that converging to a smooth minimum with respect to task loss can stabilize the task loss and lead to better performance on unseen domains. Furthermore, we recognize that the distribution of objects in the real world often follows a long-tailed class distribution, resulting in a mismatch between machine learning models and our expectations of their performance on all classes of datasets with long-tailed class distributions. To address this issue, we consider the domain generalization problem from the perspective of the long-tail distribution and propose using the maximum square loss to balance different classes which can improve model generalizability. Our method's effectiveness is demonstrated through comparisons with state-of-the-art methods on various domain generalization datasets. Code: https://github.com/bamboosir920/SAMALTDG.

ACQ: Improving generative data-free quantization via attention correction

Data-free quantization aims to achieve model quantization without accessing any authentic sample. It is significant in an application-oriented context involving data privacy. Converting noise vectors into synthetic samples through a generator is a popular data-free quantization method, which is called generative data-free quantization. However, there is a difference in attention map between synthetic samples and authentic samples. This is always ignored and restricts the quantization performance. First, since synthetic samples of the same class are prone to have homogenous attention, the quantized network can only learn limited intra-class visual features. Second, synthetic samples in eval mode and training mode exhibit different attention. Hence, the statistical distribution matching tends to be inaccurate. ACQ is proposed in this paper to fix the attention of synthetic samples. Regarding intra-class attention homogeneity, we introduce an attention center matching loss aimed at achieving coarse-grained matching of attention of synthetic samples. Additionally, we have devised an adversarial loss based on pairs of samples with identical conditions. On one hand, this mechanism prevents the generator from mode collapse due to excessive attention on conditional information. On the other hand, it augments the separation between intra-class samples, thus further enhancing intra-class attention diversity. To improve the attention similarity of synthetic samples in different network modes, we introduce a consistency penalty to guarantee accurate statistical distribution matching. The experimental results demonstrate that ACQ effectively improves the attention problems of synthetic samples. Under various training settings, ACQ achieves the best quantization performance. For the 4-bit quantization of Resnet18 and Resnet50, ACQ reaches 67.55 % and 72.23 % accuracy, respectively.

A dual-weighted adversarial network for partial domain fault diagnosis of machinery

Domain adaptation provides a promising approach to cross-domain fault diagnosis of rotating machinery. While many current methods focus on scenarios where the source and target domains share identical label spaces, a prevalent situation in industrial production involves the target domain being a subset of the source domain, known as partial domain adaptation (PDA). The main challenge in PDA is the label mismatches caused by outlier classes, making the alignment between domains particularly difficult. To this end, a dual-weighted adversarial network is proposed in this paper. Specifically, a bilateral class weighting strategy (BCWS) is developed, which can effectively suppress negative migration at the decision boundary while improving the robustness of adversarial training by applying bilateral weighting to both the source and target domains. Moreover, a collaborative framework is developed to facilitate positive migration. The constructed global perceptual module (GPM) is highly correlated with the improved adversarial loss function, which adaptively adjusts the input feature map according to the accuracy of the target domain classification and pays more attention to the domain-invariant features. Experimental results for two cases, namely across different operating conditions and across different rotating components, verify the effectiveness and superiority of the proposed method for the PDA problem.

Appearance debiased gaze estimation via stochastic subject-wise adversarial learning

Recently, appearance-based gaze estimation has been attracting attention in computer vision, and remarkable improvements have been achieved using various deep learning techniques. Despite such progress, most methods aim to infer gaze vectors from images directly, which causes overfitting to person-specific appearance factors. In this paper, we address these challenges and propose a novel framework: Stochastic subject-wise Adversarial gaZE learning (SAZE), which trains a network to generalize the appearance of subjects. We design a Face generalization Network (Fgen-Net) using a face-to-gaze encoder and face identity classifier and a proposed adversarial loss. The proposed loss generalizes face appearance factors so that the identity classifier inferences a uniform probability distribution. In addition, the Fgen-Net is trained by a learning mechanism that optimizes the network by reselecting a subset of subjects at every training step to avoid overfitting. Our experimental results verify the robustness of the method in that it yields state-of-the-art performance, achieving 3.89°and 4.42°on the MPIIFaceGaze and EyeDiap datasets, respectively. Furthermore, we demonstrate the positive generalization effect by conducting further experiments using face images involving different styles generated from the generative model.

Open-set adversarial domain match for electronic nose drift compensation and unknown gas recognition

In practical applications, electronic noses (EN) must tackle not only sensor drift but also resist interference from unknown gases. In prolonged open environments, electronic noses often encounter unknown gases that cannot be predicted in advance. Current gas detection methods typically rely on prior knowledge of target analytes but struggle to comprehensively address sensor drift and interference from unknown gases. Such as indoor air quality monitoring, long-term stability is crucial, and not all relevant analytes can be predetermined. However, the recent research cannot solve both the sensor drift and unknown gas intrusion problem simultaneously well. In this work, we unify above problems in to the open-set risk boundary. We propose an open-set adversarial domain match (OSADM) model and introduce the considers of open-set domain adaptation (OSDA). OSDA trains a target classifier through matching the domain distribution to recognize the known and unknown gases. First, a binary adversarial loss divides the class boundary. Secondly, adversarial domain adaptation unifies the distribution of different domains. Compared with the metric methods, it avoids complex distribution computation and parameter adjustment to reduce negative transfer. Extensive experimental results on two benchmark datasets, Gas Sensor Array Drift and Twin gas sensor arrays Dataset show that OSADM outperformance of the existing open-set models.

Adaptive multi-text union for stable text-to-image synthesis learning

Generative Adversarial Networks (GANs) have significantly boosted the performance of text-to-image generation tasks in recent years. To train the generator, losses like reconstruction loss or adversarial loss between the generated image and ground truth image are widely adopted by recent works. These losses are all built over one assumption: the given text descriptions can describe the corresponding image perfectly. Unfortunately, this assumption is not satisfied in many cases, especially in datasets like COCO with complicated scenes due to the variance of annotator experience and focal point. This paper addresses this issue by proposing a multi-text-to-image training framework, which adaptively adjusts the weights of all the text descriptions corresponding to a specific image to generate the union description features. With the union description features, the generator can generate more visual-consistent images and mitigate the negative optimization caused by incomplete or inconsistent text descriptions. To better measure the similarity between generated images and multi-text descriptions, we also reformulate the process of multi-modal matching loss to better measure the similarity between image and multi-text descriptions. Extensive experiments on relevant benchmarks CUB and COCO prove the proposed method’s effectiveness and superiority compared to state-of-the-art methods.

An AI-Based Low-Risk Lung Health Image Visualization Framework Using LR-ULDCT.

In this article, we proposean AI-based low-risk visualization framework for lung health monitoring using low-resolution ultra-low-dose CT (LR-ULDCT). We present a novel deep cascade processing workflow to achieve diagnostic visualization on LR-ULDCT (<0.3 mSv) at par high-resolution CT (HRCT) of 100 mSV radiation technology. To this end, we build a low-risk and affordable deep cascade network comprising three sequential deep processes: restoration, super-resolution (SR), and segmentation. Given degraded LR-ULDCT, the first novel network unsupervisedly learns restoration function from augmenting patch-based dictionaries and residuals. The restored version is then super-resolved (SR) for target (sensor) resolution. Here, we combine perceptual and adversarial losses in novel GAN to establish the closeness between probability distributions of generated SR-ULDCT and restored LR-ULDCT. Thus SR-ULDCT is presented to the segmentation network that first separates the chest portion from SR-ULDCT followed by lobe-wise colorization. Finally, we extract five lobes to account for the presence of ground glass opacity (GGO) in the lung. Hence, our AI-based system provides low-risk visualization of input degraded LR-ULDCT to various stages, i.e., restored LR-ULDCT, restored SR-ULDCT, and segmented SR-ULDCT, and achieves diagnostic power of HRCT. We perform case studies by experimenting on real datasets of COVID-19, pneumonia, and pulmonary edema/congestion while comparing our results with state-of-the-art. Ablation experiments are conducted for better visualizing different operating pipelines. Finally, we present a verification report by fourteen (14) experienced radiologists and pulmonologists.