Multi-scale Discriminators Research Articles

GFPNet: Generalizable Face Privacy Network with Dynamic Defense Training

In certain specific scenarios, there is a risk of privacy leakage in terms of the soft biometric attributes on a person’s face. However, existing face privacy-enhancing techniques suffer from limited generalizability, meaning that they can only induce misclassification in a specific classifier but fail to generalize this effect well to arbitrary attribute classifiers. Moreover, existing methods reverse attributes to improve face privacy, but this may result in privacy recovery. To address those problems, we propose GFPNet, a novel privacy-enhancing model that can provide generalizable and reliable privacy to face images. The key factor for improving generalizability is that GFPNet uses defense training, which is an effective way to improve model robustness, to dynamically strengthen the mediocre auxiliary attribute classifier during iterative training. Specifically, the generalizability of GFPNet is enhanced in the game between attack and defense, where the generator attempts to deceive the auxiliary attribute classifier and the classifier defends against the generator’s attack by defense training. Furthermore, instead of reversing attributes, skewing attributes to one side is used to avoid attribute recovery. GFPNet also integrates a face matcher, multi-scale discriminator, and Demiguise Attack to improve face matching and image quality. Extensive experiments demonstrate that GFPNet has excellent generalizability to arbitrary attribute classifiers and satisfactory face-matching utility.

Improved generative adversarial networks for denoising fabric defect images

In order to solve the problems of noise, blurring, and loss of detail in fabric defect images due to camera location and factory environments, a fabric defect image denoising method based on improved generative adversarial network (GAN) is proposed. A mathematical model is developed to simulate the noisy images of fabric blemishes in a real factory environment. GAN, as a deep learning method widely used in the field of computer vision, generates high-quality data samples through the competition between generators and discriminators. In this study, the traditional GAN is improved by introducing multiple loss terms and adopting a new adaptive weighting strategy in order to make the denoised image closer to the original clear image while better preserving the structural and detailed features of the image. In addition, the network structure is improved by introducing a multiscale discriminator, a convolutional block attention module (CBAM) and a residual network to capture feature information at different spatial scales more effectively and to promote the propagation of low-level features. The experimental results show that the method can effectively remove fabric defective image noise, and its subjective effect and objective indexes are better than similar algorithms. The research results are of great value to quality inspectors, related researchers, and image denoising technology professionals in the textile industry. These results help to improve the efficiency and accuracy of fabric defect detection and image denoising processing, thus promoting the development of related technologies.

Autoregressive GAN for Semantic Unconditional Head Motion Generation

In this work, we address the task of unconditional head motion generation to animate still human faces in a low-dimensional semantic space from a single reference pose. Different from traditional audio-conditioned talking head generation that seldom puts emphasis on realistic head motions, we devise a GAN-based architecture that learns to synthesize rich head motion sequences over long duration while maintaining low error-accumulation levels. In particular, the autoregressive generation of incremental outputs ensures smooth trajectories, while a multi-scale discriminator on input pairs drives generation toward better handling of high- and low-frequency signals and less mode collapse. We experimentally demonstrate the relevance of the proposed method and show its superiority compared to models that attained state-of-the-art performances on similar tasks.

Retrieval of Three-Dimensional Wave Surfaces from X-Band Marine Radar Images Utilizing Enhanced Pix2Pix Model

In this study, we propose a novel method for retrieving the three-dimensional (3D) wave surface from sea clutter using both simulated and measured data. First, the linear wave superposition model and modulation principle are employed to generate simulated datasets comprising 3D wave surfaces and corresponding sea clutter. Subsequently, we develop a Pix2Pix model enhanced with a self-attention mechanism and a multiscale discriminator to effectively capture the nonlinear relationship between the simulated 3D wave surfaces and sea clutter. The model’s performance is evaluated through error analysis, comparisons of wave number spectra, and differences in wave surface reconstructions using a dedicated test set. Finally, the trained model is applied to reconstruct wave surfaces from sea clutter data collected aboard a ship, with results benchmarked against those derived from the Schrödinger equation. The findings demonstrate that the proposed model excels in preserving high-frequency image details while ensuring precise alignment between reconstructed images. Furthermore, it achieves superior retrieval accuracy compared to traditional approaches, highlighting its potential for advancing wave surface retrieval techniques.

Triboelectric Nanogenerator-Embedded Intelligent Self-Aligning Roller Bearing with the Capability of Self-Sensing, Monitoring, and Fault Diagnosis.

Monitoring the dynamic behaviors of self-aligning roller bearings (SABs) is vital to guarantee the stability of various mechanical systems. This study presents a novel self-powered, intelligent, and self-aligning roller bearing (I-SAB) with which to monitor rotational speeds and bias angles; it also has an application in fault diagnosis. The designed I-SAB is compactly embedded with a novel sweep-type triboelectric nanogenerator (TENG). The TENG is realized within the proposed I-SAB using a comb-finger electrode pair and a flannelette triboelectric layer. A floating, sweeping, and freestanding mode is utilized, which can prevent collisions and considerably enhance the operational life of the embedded TENG. Experiments are subsequently conducted to optimize the output performance and sensing sensitivity of the proposed I-SAB. The results of a speed-sensing experiment show that the characteristic frequencies of triboelectric current and voltage signals are both perfectly proportional to the rotational speed, indicating that the designed I-SAB has the self-sensing capability for rotational speed. Additionally, as both the bias angle and rotational speed of the SAB increase, the envelope amplitudes of the triboelectric voltage signals generated by the I-SAB rise at a rate of 0.0057 V·deg-1·rpm-1. To further demonstrate the effectiveness of the triboelectric signals emitted from the designed I-SAB in terms of self-powered fault diagnosis, a Multi-Scale Discrimination Network (MSDN), based on the ResNet18 architecture, is proposed in order to classify the various fault conditions of the SAB. Using the triboelectric voltage and current signals emitted from the designed I-SAB as inputs, the proposed MSDN model yields excellent average diagnosis accuracies of 99.8% and 99.1%, respectively, indicating its potential for self-powered fault diagnosis.

Spectral network based on lattice convolution and adversarial training for noise-robust speech super-resolution.

Speech super-resolution aims to predict a high-resolution speech signal from its low-resolution counterpart. The previous models usually perform this task at a fixed sampling rate, reconstructing only high-frequency spectrogram components and merging them with low-frequency ones in noise-free cases. These methods achieve high accuracy, but they are less effective in real-world settings, where ambient noise and flexible sampling rates are presented. To develop a robust model that fits practical applications, in this work, we introduce Super Denoise Net (SDNet), a neural network for noise-robust super-resolution with flexible input sampling rates. To this end, SDNet's design includes gated and lattice convolution blocks for enhanced repair and temporal-spectral information capture. The frequency transform blocks are employed to model long frequency dependencies, and a multi-scale discriminator is proposed to facilitate the multi-adversarial loss training. The experiments show that SDNet outperforms current state-of-the-art noise-robust speech super-resolution models on multiple test sets, indicating its robustness and effectiveness in real-world scenarios.

TransImg: A Translation Algorithm of Visible-to-Infrared Image Based on Generative Adversarial Network

Infrared images of sensitive targets are difficult to obtain and cannot meet the design and training needs of target detection and tracking algorithms for mobile platforms such as aircraft. This paper proposes an image translation algorithm TransImg, which can achieve visible light image translation to the infrared domain to enrich the dataset. First, the algorithm designed a generator structure consisting of a deep residual connected encoder and a region perception feature fusion module to enhance feature learning, thereby avoiding issues such as generating infrared images with insufficient details in the transfer task. Afterward, a multi-scale discriminator and a composite loss function were designed to further improve the transfer effect. Finally, an automatic mixed-precision training strategy was designed for the overall migration algorithm architecture to accelerate the training and generation of infrared images. Experiments have shown that the image translation algorithm TransImg has good algorithm accuracy, and the infrared image generated by visible light image translation has richer texture details, faster generation speed, and lower video memory consumption, and the performance exceeds the mainstream traditional algorithm, and the generated images can meet the requirements of target detection and tracking algorithms design and training for mobile platforms such as aircraft.

A dual encoder LDCT image denoising model based on cross-scale skip connections

LDCT image denoising is crucial in medical imaging as it aims to minimize patient radiation exposure while maintaining diagnostic image quality. However, current convolutional neural network-based denoising methods struggle to incorporate global contexts, often focusing solely on local features. This limitation poses a significant challenge. To address this, a dual encoder denoising model is introduced that utilizes the Transformer model’s proficiency in capturing long-range dependencies and global context. This model integrates the Transformer branch and the convolutional branch in the encoder. By concatenating the features of these two different branches, the model can capture both global and local image features, substantially enhancing denoising efficacy. A cross-scale skip connection mechanism is introduced to integrate the encoder’ s low-level features with the decoder’ s high-level features, enriching contextual information and preserving image details. In addition, to meet the requirements of multi-scale feature fusion, the decoder is equipped with different multi-scale convolution modules to optimize feature processing. The number of layers in these modules gradually decreases as the depth of the decoder increases. In order to enhance the discriminative ability of the model, a multi-scale discriminator is also introduced, which effectively improves the recognition ability of the image by extracting features from four different scales. Consequently, our approach demonstrates remarkable performance in reducing noise and improving LDCT image quality, as evidenced by the substantial improvements in PSNR (17.75%) and SSIM (7.31%) values.

High-Quality Image Compression Algorithm Design Based on Unsupervised Learning.

Increasingly massive image data is restricted by conditions such as information transmission and reconstruction, and it is increasingly difficult to meet the requirements of speed and integrity in the information age. To solve the urgent problems faced by massive image data in information transmission, this paper proposes a high-quality image compression algorithm based on unsupervised learning. Among them, a content-weighted autoencoder network is proposed to achieve image compression coding on the basis of a smaller bit rate to solve the entropy rate optimization problem. Binary quantizers are used for coding quantization, and importance maps are used to achieve better bit allocation. The compression rate is further controlled and optimized. A multi-scale discriminator suitable for the generative adversarial network image compression framework is designed to solve the problem that the generated compressed image is prone to blurring and distortion. Finally, through training with different weights, the distortion of each scale is minimized, so that the image compression can achieve a higher quality compression and reconstruction effect. The experimental results show that the algorithm model can save the details of the image and greatly compress the memory of the image. Its advantage is that it can expand and compress a large number of images quickly and efficiently and realize the efficient processing of image compression.

Generation of NIR Spectral Band from RGB Image with Wavelet Domain Spectral Extrapolation Generative Adversarial Network

Near-infrared (NIR) imaging exhibits outstanding penetration capabilities and robust anti-interference characteristics. However, acquiring high-resolution and high-fidelity NIR images is difficult due to the limited mobility, high cost, and low resolution of NIR imaging hardware. In this paper, we proposed a new end-to-end Wavelet Domain Spectral Extrapolation Generative Adversarial Network (WSEGAN) to generate highly realistic NIR images from RGB images. Since RGB and NIR images have different types of noise and artifacts, which affects the quality of NIR images generation. We design a generator with a discrete wavelet transform and an attention mechanism to capture multi-resolution contextual information, and a multi-scale discriminator to capture detailed and global features. Then, the proposed approach is evaluated by three different datasets to achieve optimal results regarding both visual effects and quantitative evaluation. The normalized difference vegetation index (NDVI) is utilized to validate the effectiveness of the generated NIR images. The result demonstrates a strong correlation between the generated images and the actual vegetation distribution. More importantly, the proposed network is testified to generate NIR images to be fused with the RGB image source for agricultural target detection tasks. In dark conditions, results show that using multi-modal data instead of RGB images improves mAP0.5 detection accuracy by 4% on the CAPSICUM dataset and by 8% on the KIWI dataset across five object detection methods. This follows the physical significance of the NIR imaging and demonstrates the potential use of the NIR images generated by the proposed method.

Improvement in Image Quality of Low-Dose CT of Canines with Generative Adversarial Network of Anti-Aliasing Generator and Multi-Scale Discriminator.

Computed tomography (CT) imaging is vital for diagnosing and monitoring diseases in both humans and animals, yet radiation exposure remains a significant concern, especially in animal imaging. Low-dose CT (LDCT) minimizes radiation exposure but often compromises image quality due to a reduced signal-to-noise ratio (SNR). Recent advancements in deep learning, particularly with CycleGAN, offer promising solutions for denoising LDCT images, though challenges in preserving anatomical detail and image sharpness persist. This study introduces a novel framework tailored for animal LDCT imaging, integrating deep learning techniques within the CycleGAN architecture. Key components include BlurPool for mitigating high-resolution image distortion, PixelShuffle for enhancing expressiveness, hierarchical feature synthesis (HFS) networks for feature retention, and spatial channel squeeze excitation (scSE) blocks for contrast reproduction. Additionally, a multi-scale discriminator enhances detail assessment, supporting effective adversarial learning. Rigorous experimentation on veterinary CT images demonstrates our framework's superiority over traditional denoising methods, achieving significant improvements in noise reduction, contrast enhancement, and anatomical structure preservation. Extensive evaluations show that our method achieves a precision of 0.93 and a recall of 0.94. This validates our approach's efficacy, highlighting its potential to enhance diagnostic accuracy in veterinary imaging. We confirm the scSE method's critical role in optimizing performance, and robustness to input variations underscores its practical utility.

Image style migration method based on transformer generative adversarial network combined with contrast learning strategy

Abstract Aiming at the problems of large loss of image patch embedding information and lack of stylized features in the generated image during transformer style migration, a style migration model combining contrast learning with transformer backbone generative adversarial network is established. Firstly, overlapping patch embedding is used to serialize the input images to get richer features; then the style and content contrast loss of contrast learning is used to mine the non-significant information between stylized images to reduce the style and content loss; finally, a multi-scale discriminator discriminates the true and false of stylized images to optimize the generator to get more stylistic features. The results on MS-COCO and WikiArt datasets show that the model improves the effectiveness and transformation efficacy of stylized images in style migration.

Multidomain Object Detection Framework Using Feature Domain Knowledge Distillation.

Object detection techniques have been widely studied, utilized in various works, and have exhibited robust performance on images with sufficient luminance. However, these approaches typically struggle to extract valuable features from low-luminance images, which often exhibit blurriness and dim appearence, leading to detection failures. To overcome this issue, we introduce an innovative unsupervised feature domain knowledge distillation (KD) framework. The proposed framework enhances the generalization capability of neural networks across both low- and high-luminance domains without incurring additional computational costs during testing. This improvement is made possible through the integration of generative adversarial networks and our proposed unsupervised KD process. Furthermore, we introduce a region-based multiscale discriminator designed to discern feature domain discrepancies at the object level rather than from the global context. This bolsters the joint learning process of object detection and feature domain distillation tasks. Both qualitative and quantitative assessments shown that the proposed method, empowered by the region-based multiscale discriminator and the unsupervised feature domain distillation process, can effectively extract beneficial features from low-luminance images, outperforming other state-of-the-art approaches in both low- and sufficient-luminance domains.

A Digital Animation Generation Model Based on Cycle Adversarial Neural Network

This paper proposes a digital animation generation model based on Cycle Adversarial Neural Network (CycleGAN). Compared with the classical CycleGAN, this research presents a multi-attention approach to enhance the network’s generalization. Specifically, a style enhancement module and a style cross-attention mechanism are introduced into the generator network structure, which enables the model to better parse the structural information of the content image and realize the accurate matching of content features and style features. Furthermore, the introduction of a multi-scale discriminator with a fused attention mechanism enhances the preservation of content information from the source image in the output animated image. The experiments conducted in this study demonstrate that the model proposed in this paper exhibits superior performance in generating digital animation. This model not only enhances the realism and variety of the generated effects, but also achieves significant improvements in the coherence and stability of long time-series animation. The contribution of this paper lies in the introduction of the multi-attention mechanism, which enhances the generalization of the network and is of great theoretical and practical significance for the development of the field of digital animation generation.

Class-conditional domain adaptation for semantic segmentation

Semantic segmentation is an important sub-task for many applications. However, pixel-level ground-truth labeling is costly, and there is a tendency to overfit to training data, thereby limiting the generalization ability. Unsupervised domain adaptation can potentially address these problems by allowing systems trained on labelled datasets from the source domain (including less expensive synthetic domain) to be adapted to a novel target domain. The conventional approach involves automatic extraction and alignment of the representations of source and target domains globally. One limitation of this approach is that it tends to neglect the differences between classes: representations of certain classes can be more easily extracted and aligned between the source and target domains than others, limiting the adaptation over all classes. Here, we address this problem by introducing a Class-Conditional Domain Adaptation (CCDA) method. This incorporates a class-conditional multi-scale discriminator and class-conditional losses for both segmentation and adaptation. Together, they measure the segmentation, shift the domain in a class-conditional manner, and equalize the loss over classes. Experimental results demonstrate that the performance of our CCDA method matches, and in some cases, surpasses that of state-of-the-art methods.

Heritage applications of landscape design in environmental art based on image style migration

To create environmental art on the basis of preserving environmental and ecological resources, landscape design considers the peaceful coexistence of art and ecology. In picture generation and rendering for landscape design, image style migration techniques are frequently employed, however their quality is still lacking at this point. In order to enhance the migration effect, the study builds an image style migration model for landscape design, tries to employ a multi-scale discriminator, and introduces a content feature mapping module and a migration learning approach. According to the test results, adding the content feature mapping module enhanced the feature mapping effect of the photographs. The revised model's peak signal-to-noise ratio and similarity assessment indexes were 17.42 and 0.91, respectively, and the model's generated images were less distorted. While the Frey interval distance was only 16.32 and the generated images were strongly correlated, the model had good generalisation. The addition of migration learning somewhat improves all of the measures. In conclusion, the research findings are beneficial for the expression of environmental art in landscape design, and the model's picture creation assessment index and stability are both good.

Generative adversarial networks with multi-scale and attention mechanisms for underwater image enhancement

The images captured underwater are usually degraded due to the effects of light absorption and scattering. Degraded underwater images exhibit color distortion, low contrast, and blurred details, which in turn reduce the accuracy of marine biological monitoring and underwater object detection. To address this issue, a generative adversarial network with multi-scale and an attention mechanism is proposed to improve the quality of underwater images. To extract more effective features within the generative network, several modules are introduced: a multi-scale dilated convolution module, a novel attention module, and a residual module. These modules are utilized to design a generative network with a U-shaped structure. The multi-scale dilated convolution module is designed to extract features at multiple scales and expand the receptive field to capture more global information. The attention module directs the network’s focus towards important features, thereby reducing the interference from redundant feature information. To improve the discriminative power of the adversarial network, a multi-scale discriminator is designed. It has two output feature maps with different scales. Additionally, an improved loss function for the generative adversarial network is proposed. This improvement involves incorporating the total variation loss into the traditional loss function. The performance of different methods for enhancing underwater images is evaluated using the EUVP dataset and UIEB dataset. The experimental results demonstrate that the enhanced underwater images exhibit better quality and visual effects compared to other methods.

Improving Generative Adversarial Network-based Vocoding through Multi-scale Convolution

Vocoding is a sub-process of text-to-speech task, which aims at generating audios from intermediate representations between text and audio. Several recent works have shown that generative adversarial network– (GAN) based vocoders can generate audios with high quality. While GAN-based neural vocoders have shown higher efficiency in generating speed than autoregressive vocoders, the audio fidelity still cannot compete with ground-truth samples. One major cause of the degradation in audio quality and spectrogram vague comes from the average pooling layers in discriminator. As the multi-scale discriminator commonly used by recent GAN-based vocoders applies several average pooling layers to capture different-frequency bands, we believe it is crucial to prevent the high-frequency information from leakage in the average pooling process. This article proposes MSCGAN, which solves the above-mentioned problem and achieves higher-fidelity speech synthesis. We demonstrate that substituting the average pooling process with a multi-scale convolution architecture effectively retains high-frequency features and thus forces the generator to recover audio details in time and frequency domain. Compared with other state-of-the-art GAN-based vocoders, MSCGAN can produce competitive audio with a higher spectrogram clarity and mean opinion score score in subjective human evaluation.

Pose-invariant and occlusion-robust neonatal facial pain assessment

Neonatal Facial Pain Assessment (NFPA) is essential to improve neonatal pain management. Pose variation and occlusion, which can significantly alter the facial appearance, are two major and still unstudied barriers to NFPA. We bridge this gap in terms of method and dataset. Techniques to tackle both challenges in other tasks either expect pose/occlusion-invariant deep learning methods or first generate a normal version of the input image before feature extraction, combining these we argue that it is more effective to jointly perform adversarial learning and end-to-end classification for their mutual benefit. To this end, we propose a Pose-invariant Occlusion-robust Pain Assessment (POPA) framework, with two novelties. We incorporate adversarial learning-based disturbance mitigation for end-to-end pain-level classification and propose a novel composite loss function for facial representation learning; compared to the vanilla discriminator that implicitly determines occlusion and pose conditions, we propose a multi-scale discriminator that determines explicitly, while incorporating local discriminators to enhance the discrimination of key regions. For a comprehensive evaluation, we built the first neonatal pain dataset with disturbance annotation involving 1091 neonates and also applied the proposed POPA to the facial expression recognition task. Extensive qualitative and quantitative experiments prove the superiority of the POPA.

Remote Sensing Image Dehazing through an Unsupervised Generative Adversarial Network.

The degradation of visual quality in remote sensing images caused by haze presents significant challenges in interpreting and extracting essential information. To effectively mitigate the impact of haze on image quality, we propose an unsupervised generative adversarial network specifically designed for remote sensing image dehazing. This network includes two generators with identical structures and two discriminators with identical structures. One generator is focused on image dehazing, while the other generates images with added haze. The two discriminators are responsible for distinguishing whether an image is real or generated. The generator, employing an encoder-decoder architecture, is designed based on the proposed multi-scale feature-extraction modules and attention modules. The proposed multi-scale feature-extraction module, comprising three distinct branches, aims to extract features with varying receptive fields. Each branch comprises dilated convolutions and attention modules. The proposed attention module includes both channel and spatial attention components. It guides the feature-extraction network to emphasize haze and texture within the remote sensing image. For enhanced generator performance, a multi-scale discriminator is also designed with three branches. Furthermore, an improved loss function is introduced by incorporating color-constancy loss into the conventional loss framework. In comparison to state-of-the-art methods, the proposed approach achieves the highest peak signal-to-noise ratio and structural similarity index metrics. These results convincingly demonstrate the superior performance of the proposed method in effectively removing haze from remote sensing images.