Unsupervised Generative Adversarial Network Research Articles

Technical note: Minimizing CIED artifacts on a 0.35 T MRI-Linac using deep learning.

Artifacts from implantable cardioverter defibrillators (ICDs) are a challenge to magnetic resonance imaging (MRI)-guided radiotherapy (MRgRT). This study tested an unsupervised generative adversarial network to mitigate ICD artifacts in balanced steady-state free precession (bSSFP) cine MRIs and improve image quality and tracking performance for MRgRT. Fourteen healthy volunteers (Group A) were scanned on a 0.35 T MRI-Linac with and without an MR conditional ICD taped to their left pectoral to simulate an implanted ICD. bSSFP MRI data from 12 of the volunteers were used to train a CycleGAN model to reduce ICD artifacts. The data from the remaining two volunteers were used for testing. In addition, the dataset was reorganized three times using a Leave-One-Out scheme. Tracking metrics [Dice similarity coefficient (DSC), target registration error (TRE), and 95 percentile Hausdorff distance (95% HD)] were evaluated for whole-heart contours. Image quality metrics [normalized root mean square error (nRMSE), peak signal-to-noise ratio (PSNR), and multiscale structural similarity (MS-SSIM) scores] were evaluated. The technique was also tested qualitatively on three additional ICD datasets (Group B) including a patient with an implanted ICD. For the whole-heart contour with CycleGAN reconstruction: 1) Mean DSC rose from 0.910 to 0.935; 2) Mean TRE dropped from 4.488 to 2.877mm; and 3) Mean 95% HD dropped from 10.236 to 7.700mm. For the whole-body slice with CycleGAN reconstruction: 1) Mean nRMSE dropped from 0.644 to 0.420; 2) Mean MS-SSIM rose from 0.779 to 0.819; and 3) Mean PSNR rose from 18.744 to 22.368. The three Group B datasets evaluated qualitatively displayed a reduction in ICD artifacts in the heart. CycleGAN-generated reconstructions significantly improved both tracking and image quality metrics when used to mitigate artifacts from ICDs.

An Unsupervised Generative Adversarial Network System to Detect DDoS Attacks in SDN

An Unsupervised Generative Adversarial Network System to Detect DDoS Attacks in SDN

Isotropic multi-scale neuronal reconstruction from high-ratio expansion microscopy with contrastive unsupervised deep generative models

Background and objectiveCurrent methods for imaging reconstruction from high-ratio expansion microscopy (ExM) data are limited by anisotropic optical resolution and the requirement for extensive manual annotation, creating a significant bottleneck in the analysis of complex neuronal structures. MethodsWe devised an innovative approach called the IsoGAN model, which utilizes a contrastive unsupervised generative adversarial network to sidestep these constraints. This model leverages multi-scale and isotropic neuron/protein/blood vessel morphology data to generate high-fidelity 3D representations of these structures, eliminating the need for rigorous manual annotation and supervision. The IsoGAN model introduces simplified structures with idealized morphologies as shape priors to ensure high consistency in the generated neuronal profiles across all points in space and scalability for arbitrarily large volumes. ResultsThe efficacy of the IsoGAN model in accurately reconstructing complex neuronal structures was quantitatively assessed by examining the consistency between the axial and lateral views and identifying a reduction in erroneous imaging artifacts. The IsoGAN model accurately reconstructed complex neuronal structures, as evidenced by the consistency between the axial and lateral views and a reduction in erroneous imaging artifacts, and can be further applied to various biological samples. ConclusionWith its ability to generate detailed 3D neurons/proteins/blood vessel structures using significantly fewer axial view images, IsoGAN can streamline the process of imaging reconstruction while maintaining the necessary detail, offering a transformative solution to the existing limitations in high-throughput morphology analysis across different structures.

Underwater Image Enhancement based on Retinex Decomposition and Unsupervised Generative Adversarial Networks

Background:: Due to the difficulty of obtaining the real dataset of paired underwater images, it is urgent to build an unsupervised underwater image enhancement network. Objective:: To address the problem, a novel underwater image enhancement based on Retinex decomposition and Unsupervised Generative Adversarial Network (RUGAN) is proposed. Method:: A color correction module is proposed considering the different color distortions of underwater images. Further, considering the human visual perception mechanism, the RUGAN network, which is similar to U-Net, is constructed using the characteristics of underwater imaging and Retinex decomposition. Based on Retinex decomposition and the characteristics of underwater imaging, the RUGAN network similar to U-Net is constructed. The reflectance image and illumination image are obtained. The reflectance image with a better effect is taken as the enhancement result. Unlike the previous supervised methods, RUGAN adopts clear air images and distorted underwater images as training. RUGAN adopts the underwater image of the color correction module as pseudo-ground truth to achieve an unsupervised effect. Results:: The superiority of RUGAN network is further supported by extensive experiments that compared it with more methods. conclusion: The proposed RUGAN achieves better results both subjectively and objectively.

N-LoLiGan: Unsupervised low-light enhancement GAN with an N-Net for low-light tunnel images

Because of low luminance in tunnels, target features in images are not salient, which makes the detection of tunnel surface defects challenging. Most deep-learning-based low-light enhancement methods suffer from contrast distortion and insufficient brightness in extremely dark scenes, as well as artifacts and overexposure in tunnel scenes with large light and dark differences. To alleviate these problems, we propose a novel unsupervised generative adversarial network (GAN), called the N-shape low-light enhancement GAN (N-LoLiGan), which can be trained using unpaired low-/normal-light images and is proven to generalize sufficiently on tunnel images with complex lighting. Experimental results demonstrate that the proposed N-LoLiGan outperforms several state-of-the-art low-light image enhancement methods in terms of visual quality and two no-reference image quality metrics. Additionally, when the proposed method is used as a preprocessing step for the tunnel multi-target detection task, it yields a higher detection rate (88.9% and 97.23%) than other methods.

Unsupervised GAN epoch selection for biomedical data synthesis

Abstract Supervised Neural Networks are used for segmentation in many biological and biomedical applications. To omit the time-consuming and tiring process of manual labeling, unsupervised Generative Adversarial Networks (GANs) can be used to synthesize labeled data. However, the training of GANs requires extensive computation and is often unstable. Due to the lack of established stopping criteria, GANs are usually trained multiple times for a heuristically fixed number of epochs. Early stopping and epoch selection can lead to better synthetic datasets resulting in higher downstream segmentation quality on biological or medical data. This article examines whether the Frechet Inception Distance (FID), the Kernel Inception Distance (KID), or the WeightWatcher tool can be used for early stopping or epoch selection of unsupervised GANs. The experiments show that the last trained GAN epoch is not necessarily the best one to synthesize downstream segmentation data. On complex datasets, FID and KID correlate with the downstream segmentation quality, and both can be used for epoch selection.

Remote Sensing Image Dehazing through an Unsupervised Generative Adversarial Network.

The degradation of visual quality in remote sensing images caused by haze presents significant challenges in interpreting and extracting essential information. To effectively mitigate the impact of haze on image quality, we propose an unsupervised generative adversarial network specifically designed for remote sensing image dehazing. This network includes two generators with identical structures and two discriminators with identical structures. One generator is focused on image dehazing, while the other generates images with added haze. The two discriminators are responsible for distinguishing whether an image is real or generated. The generator, employing an encoder-decoder architecture, is designed based on the proposed multi-scale feature-extraction modules and attention modules. The proposed multi-scale feature-extraction module, comprising three distinct branches, aims to extract features with varying receptive fields. Each branch comprises dilated convolutions and attention modules. The proposed attention module includes both channel and spatial attention components. It guides the feature-extraction network to emphasize haze and texture within the remote sensing image. For enhanced generator performance, a multi-scale discriminator is also designed with three branches. Furthermore, an improved loss function is introduced by incorporating color-constancy loss into the conventional loss framework. In comparison to state-of-the-art methods, the proposed approach achieves the highest peak signal-to-noise ratio and structural similarity index metrics. These results convincingly demonstrate the superior performance of the proposed method in effectively removing haze from remote sensing images.

Cyclic Generative Attention-Adversarial Network for Low-Light Image Enhancement.

Images captured under complex conditions frequently have low quality, and image performance obtained under low-light conditions is poor and does not satisfy subsequent engineering processing. The goal of low-light image enhancement is to restore low-light images to normal illumination levels. Although many methods have emerged in this field, they are inadequate for dealing with noise, color deviation, and exposure issues. To address these issues, we present CGAAN, a new unsupervised generative adversarial network that combines a new attention module and a new normalization function based on cycle generative adversarial networks and employs a global-local discriminator trained with unpaired low-light and normal-light images and stylized region loss. Our attention generates feature maps via global and average pooling, and the weights of different feature maps are calculated by multiplying learnable parameters and feature maps in the appropriate order. These weights indicate the significance of corresponding features. Specifically, our attention is a feature map attention mechanism that improves the network's feature-extraction ability by distinguishing the normal light domain from the low-light domain to obtain an attention map to solve the color bias and exposure problems. The style region loss guides the network to more effectively eliminate the effects of noise. The new normalization function we present preserves more semantic information while normalizing the image, which can guide the model to recover more details and improve image quality even further. The experimental results demonstrate that the proposed method can produce good results that are useful for practical applications.

Automated Classical Cipher Emulation Attacks via Unified Unsupervised Generative Adversarial Networks

Cryptanalysis has been studied and gradually improved with the evolution of cryptosystems over past decades. Recently, deep learning (DL) has started to be used in cryptanalysis to attack digital cryptosystems. As computing power keeps growing, deploying DL-based cryptanalysis becomes feasible in practice. However, since these studies can analyze only one cipher type for one DL model learning, it takes a lot of time to analyze multi ciphers. In this paper, we propose a unified cipher generative adversarial network (UC-GAN), which can perform ciphertext-to-plaintext translations among multiple domains (ciphers) using only a single DL model. In particular, the proposed model is based on unified unsupervised DL for the analysis of classical substitutional ciphers. Simulation results have indicated the feasibility and good performance of the proposed approach. In addition, we compared our experimental results with the findings of conditional GAN, where plaintext and ciphertext pairs in only the single domain are given as training data, and with CipherGAN, which is cipher mapping between unpaired ciphertext and plaintext in the single domain, respectively. The proposed model showed more than 97% accuracy by learning only data without prior knowledge of three substitutional ciphers. These findings could open a new possibility for simultaneously cracking various block ciphers, which has a great impact on the field of cryptography. To the best of our knowledge, this is the first study of the cryptanalysis of multiple cipher algorithms using only a single DL model

An alternately optimized generative adversarial network with texture and content constraints for deformable registration of 3D ultrasound images

Objective. 3D ultrasound non-rigid registration is significant for intraoperative motion compensation. Nevertheless, distorted textures in the registered image due to the poor image quality and low signal-to-noise ratio of ultrasound images reduce the accuracy and efficiency of the existing methods. Approach. A novel 3D ultrasound non-rigid registration objective function with texture and content constraints in both image space and multiscale feature space based on an unsupervised generative adversarial network based registration framework is proposed to eliminate distorted textures. A similarity metric in the image space is formulated based on combining self-structural constraint with intensity to strengthen the robustness to abnormal intensity change compared with common intensity-based metrics. The proposed framework takes two discriminators as feature extractors to formulate the texture and content similarity between the registered image and the fixed image in the multiscale feature space respectively. A distinctive alternating training strategy is established to jointly optimize the combination of various similarity loss functions to overcome the difficulty and instability of training convergence and balance the training of generator and discriminators. Main results. Compared with five registration methods, the proposed method is evaluated both with small and large deformations, and achieves the best registration accuracy with average target registration error of 1.089 mm and 2.139 mm in cases of small and large deformations, respectively. The performance on peak signal to noise ratio (PSNR) and structural similarity (SSIM) also proves the effective constraints on distorted textures of the proposed method (PSNR is 31.693 dB and SSIM is 0.9 in the case of small deformation; PSNR is 28.177 dB and SSIM is 0.853 in the case of large deformation). Significance. The proposed 3D ultrasound non-rigid registration method based on texture and content constraints with the distinctive alternating training strategy can eliminate the distorted textures with improving the registration accuracy.

Reduction of rain effect on wave height estimation from marine X-band radar images using unsupervised generative adversarial networks

ABSTRACT An intelligent single radar image de-raining method based on unsupervised self-attention generative adversarial networks is proposed to improve the accuracy of wave height parameter inversion results. The method builds a trainable end-to-end de-raining model with an unsupervised cycle-consistent adversarial network as an AI framework, which does not require pairs of rain-contaminated and corresponding ground-truth rain-free images for training. The model is trained by feeding rain-contaminated and clean radar images in an unpaired manner, and the atmospheric scattering model parameters are not required as a prior condition. Additionally, a self-attention mechanism is introduced into the model, allowing it to focus on rain clutter when processing radar images. This combines global and local rain clutter context information to output more accurate and clear de-raining radar images. The proposed method is validated by applying it to actual field test data, which shows that compared with the wave height derived from the original rain-contaminated data, the root-mean-square error is reduced by 0.11 m and the correlation coefficient of the wave height is increased by 14% using the de-raining method. These results demonstrate that the method effectively reduces the impact of rain on the accuracy of wave height parameter estimation from marine X-band radar images.

Adversarial Fault Detector Guided by One-Class Learning for a Multistage Centrifugal Pump

The data unavailability of critical machinery is an open issue in the field of condition-based maintenance research. Acquiring signals in all possible health conditions is impractical in most equipment. This lack of data affects the ability to extract informative features to build effective fault detectors; therefore, the development of proposals to deal with these conditions is necessary. To address this issue, we introduce a systematic methodology to build fault detection models from vibration signals for cyclo-stationary machines under limited data availability under faulty conditions. In the first step, vibration signals are modeled by unsupervised generative adversarial network (GAN)-based approach. Then, the best critic model for the GAN is determined for the feature extraction task guided by a one-class classifier. Finally, a fault detector is optimized to determine the health condition of the machinery. We propose an interpretation of the one-class support vector machine (SVM) hyperparameters for the feature space evaluation. The experiments carried out in the proposal were applied on a multistage centrifugal pump for single and multifault scenarios, which show a resulting feature space simpler than other methods reported in the literature, and outperform them in the fault detection task.

Two-Step Unsupervised Approach for Sand-Dust Image Enhancement

In sand-dust environments, light is scattered and absorbed, and sand-dust images thus suffer from severe image degradation problems, such as color shifts, low contrast, and blurred details. To address these problems, we propose a two-step unsupervised sand-dust image enhancement algorithm. In the first step, a convenient and competent color correction method is put forward to solve the color shift problem. Considering the wavelength attenuation features of sand-dust images, a linear stretching and blue channel compensation method is designed, and an adaptive color shift correction factor is developed to remove the color shift. In the second step, to enhance the clarity and details of the images, an unsupervised generative adversarial network is proposed, which does not require pairs of data for training. To reduce detail loss, the detail enhancement branch is designed, and the generator considers to more details through the constructed coarse-grained and fine-grained discriminators. The introduced multiscale perceptual loss promotes the image fidelity well. Experiments show that the proposed method achieves better color correction, enhances image details and clarity, has a better subjective effect, and outperforms existing sand-dust image enhancement methods both quantitatively and qualitatively. Similarly, our method promotes the application capability of the target detection algorithm and also has a good enhancement effect on underwater images and haze images.

Multivariate Time Series Anomaly Detection With Generative Adversarial Networks Based on Active Distortion Transformer

Detecting anomalies for multivariate time series is of great importance in modern industrial applications. However, due to the complex temporal dynamics in modern systems, finding a distinguishable judge criterion is hard, which makes accurate anomaly detection still a challenging task. In order to better capture the anomalous features and design a more informative judge criterion, this paper presents an unsupervised generative adversarial network (GAN) for multivariate time series anomaly detection, which highlights a novel Active Distortion Transformer (ADT) block. Different from the vanilla Transformer, the ADT block can make good use of the prior knowledge of time sequences’ overall associations by actively conducting distortion during the reconstruction of input sequences. Benefiting from the ADT block, the network simultaneously utilizes the sequence associations and reconstruction error to recognize anomalies. In the online detection phase, anomalous data points tend to be less correlated with the overall sequence and have greater reconstruction errors than normal ones, so that an irrelevance score and a reconstruction error score can be obtained. We combine the two scores to generate a more powerful anomaly score as the judge criterion. Extensive experiments are conducted on four publicly available sensor datasets, and we also make comparisons with the recent baselines. Results show that our model outperforms the recent state-of-the-art methods, demonstrating the effectiveness of our method.

Infrared and Visible Image Homography Estimation Using Multiscale Generative Adversarial Network

In computer vision, the homography estimation of infrared and visible multi-source images based on deep learning is a current research hotspot. Existing homography estimation methods ignore the feature differences of multi-source images, which leads to poor homography performance in infrared and visible image scenes. To address this issue, we designed an infrared and visible image homography estimation method using a Multi-scale Generative Adversarial Network, called HomoMGAN. First, we designed two shallow feature extraction networks to extract fine features of infrared and visible images, respectively, which extract important features in source images from two dimensions: color channel and imaging space. Second, we proposed an unsupervised generative adversarial network to predict the homography matrix directly. In our adversarial network, the generator captures meaningful features for homography estimation at different scales by using an encoder–decoder structure and further predicts the homography matrix. The discriminator recognizes the feature difference between the warped and target image. Through the adversarial game between the generator and the discriminator, the fine features of the warped image in the homography estimation process are closer to the fine features of the target image. Finally, we conduct extensive experiments in the synthetic benchmark dataset to verify the effectiveness of HomoMGAN and its components. We conduct extensive experiments and the results show that HomoMGAN outperforms existing state-of-the-art methods in the synthetic benchmark datasets both qualitatively and quantitatively.

A generative deep learning framework for inverse design of compositionally complex bulk metallic glasses

The design of bulk metallic glasses (BMGs) via machine learning (ML) has been a topic of active research recently. However, the prior ML models were mostly built upon supervised learning algorithms with human inputs to navigate the high dimensional compositional space, which becomes inefficient with the increasing compositional complexity in BMGs. Here, we develop a generative deep-learning framework to directly generate compositionally complex BMGs, such as high entropy BMGs. Our framework is built on the unsupervised Generative Adversarial Network (GAN) algorithm for data generation and the supervised Boosted Trees algorithm for data evaluation. We studied systematically the confounding effect of various data descriptors and the literature data on the effectiveness of our framework both numerically and experimentally. Most importantly, we demonstrate that our generative deep learning framework is capable of producing composition-property mappings, therefore paving the way for the inverse design of BMGs.

An optimized GAN method based on the Que-Attn and contrastive learning for underwater image enhancement.

Research on underwater image processing has increased significantly in the past decade due to the precious resources that exist underwater. However, it is still a challenging problem to restore degraded underwater images. Existing prior-based methods show limited performance in many cases due to their reliance on hand-crafted features. Therefore, in this paper, we propose an effective unsupervised generative adversarial network(GAN) for underwater image restoration. Specifically, we embed the idea of contrastive learning into the model. The method encourages two elements (corresponding patches) to map the similar points in the learned feature space relative to other elements (other patches) in the data set, and maximizes the mutual information between input and output through PatchNCE loss. We design a query attention (Que-Attn) module, which compares feature distances in the source domain, and gives an attention matrix and probability distribution for each row. We then select queries based on their importance measure calculated from the distribution. We also verify its generalization performance on several benchmark datasets. Experiments and comparison with the state-of-the-art methods show that our model outperforms others.

GN-CNN: A Point Cloud Analysis Method for Metaverse Applications

Metaverse applications often require many new 3D point cloud models that are unlabeled and that have never been seen before; this limited information results in difficulties for data-driven model analyses. In this paper, we propose a novel data-driven 3D point cloud analysis network GN-CNN that is suitable for such scenarios. We tackle the difficulties with a few-shot learning (FSL) approach by proposing an unsupervised generative adversarial network GN-GAN to generate prior knowledge and perform warm start pre-training for GN-CNN. Furthermore, the 3D models in the Metaverse are mostly acquired with a focus on the models’ visual appearances instead of the exact positions. Thus, conceptually, we also propose to augment the information by unleashing and incorporating local variance information, which conveys the appearance of the model. This is realized by introducing a graph convolution-enhanced combined multilayer perceptron operation (CMLP), namely GCMLP, to capture the local geometric relationship as well as a local normal-aware GeoConv, namely GNConv. The GN-GAN adopts an encoder–decoder structure and the GCMLP is used as the core operation of the encoder. It can perform the reconstruction task. The GNConv is used as the convolution-like operation in GN-CNN. The classification performance of GN-CNN is evaluated on ModelNet10 with an overall accuracy of 95.9%. Its few-shot learning performance is evaluated on ModelNet40, when the training set size is reduced to 30%, the overall classification accuracy can reach 91.8%, which is 2.5% higher than Geo-CNN. Experiments show that the proposed method could improve the accuracy in 3D point cloud classification tasks and under few-shot learning scenarios, compared with existing methods such as PointNet, PointNet++, DGCNN, and Geo-CNN, making it a beneficial method for Metaverse applications.

Pansharpening Using Unsupervised Generative Adversarial Networks With Recursive Mixed-Scale Feature Fusion

Panchromatic sharpening (pansharpening) is an important technology for improving the spatial resolution of multispectral (MS) images. The majority of the models are implemented at the reduced resolution, leading to unfavorable results at the full resolution. Moreover, the complicated relationship between MS and panchromatic (PAN) images is often ignored in detail injection. For the mentioned problems, unsupervised generative adversarial networks with recursive mixed-scale feature fusion for pansharpening (RMFF-UPGAN) are modeled to boost the spatial resolution and preserve the spectral information. RMFF-UPGAN comprises a generator and two U-shaped discriminators. A dual-stream trapezoidal branch is designed in the generator to obtain multiscale information. Further, a recursive mixed-scale feature fusion subnetwork is designed. Perform a prior fusion on the extracted MS and PAN features of the same scale. A mixed-scale fusion is conducted on the prior fusion results of the fine-scale and coarse-scale. The fusion is executed sequentially in the above manner building a recursive mixed-scale fusion structure and finally generating key information. A compensation information mechanism is also designed for the reconstruction of key information to compensate for information. A nonlinear rectification block for the reconstructed information is developed to overcome the distortion induced by neglecting the complicated relationship between MS and PAN images. Two U-shaped discriminators are designed and a new composite loss function is defined. The presented model is validated using two satellite data and the outcomes reveal better than the prevalent approaches regarding both visual assessment and objective indicators.

One-Sided Unsupervised Image Dehazing Network Based on Feature Fusion and Multi-Scale Skip Connection

Haze and mist caused by air quality, weather, and other factors can reduce the clarity and contrast of images captured by cameras, which limits the applications of automatic driving, satellite remote sensing, traffic monitoring, etc. Therefore, the study of image dehazing is of great significance. Most existing unsupervised image-dehazing algorithms rely on a priori knowledge and simplified atmospheric scattering models, but the physical causes of haze in the real world are complex, resulting in inaccurate atmospheric scattering models that affect the dehazing effect. Unsupervised generative adversarial networks can be used for image-dehazing algorithm research; however, due to the information inequality between haze and haze-free images, the existing bi-directional mapping domain translation model often used in unsupervised generative adversarial networks is not suitable for image-dehazing tasks, and it also does not make good use of extracted features, which results in distortion, loss of image details, and poor retention of image features in the haze-free images. To address these problems, this paper proposes an end-to-end one-sided unsupervised image-dehazing network based on a generative adversarial network that directly learns the mapping between haze and haze-free images. The proposed feature-fusion module and multi-scale skip connection based on residual network consider the loss of feature information caused by convolution operation and the fusion of different scale features, and achieve adaptive fusion between low-level features and high-level features, to better preserve the features of the original image. Meanwhile, multiple loss functions are used to train the network, where the adversarial loss ensures that the network generates more realistic images and the contrastive loss ensures a meaningful one-sided mapping from the haze image to the haze-free image, resulting in haze-free images with good quantitative metrics and visual effects. The experiments demonstrate that, compared with existing dehazing algorithms, our method achieved better quantitative metrics and better visual effects on both synthetic haze image datasets and real-world haze image datasets.