Basic Generative Adversarial Network Research Articles

Image-to-Image Translation with CNN Based Perceptual Similarity Metrics

Image-to-image translation is the process of transforming images from different domains. Generative Adversarial Networks (GANs), and Convolutional Neural Networks (CNNs) are widely used in image translation. This study aims to find the most effective loss function for GAN architectures and synthesize better images. For this, experimental results were obtained by changing the loss functions on the Pix2Pix method, one of the basic GAN architectures. The exist loss function used in the Pix2Pix method is the Mean Absolute Error (MAE). It is called the L_1metric. In this study, the effect of convolutional-based perceptual similarity CONTENT, LPIPS, and DISTS metrics on image-to-image translation was applied on the loss function in Pix2Pix architecture. In addition, the effects on image-to-image translation were analyzed using perceptual similarity metrics ( L_1_CONTENT, L_1_LPIPS, and L_1_DISTS) with the original L_1 loss at a rate of 50%. Performance analyzes of the methods were performed with the Cityscapes, Denim2Mustache, Maps, and Papsmear datasets. Visual results were analyzed with conventional (FSIM, HaarPSI, MS-SSIM, PSNR, SSIM, VIFp and VSI) and up-to-date (FID and KID) image comparison metrics. As a result, it has been observed that better results are obtained when convolutional-based methods are used instead of conventional methods for the loss function of GAN architectures. It has been observed that LPIPS and DISTS methods can be used in the loss function of GAN architectures in the future

Stock market forecasting using DRAGAN and feature matching

Applying machine learning methods to forecast stock prices has been a topic of interest in recent years. However, a few studies have been reported based on generative adversarial networks (GANs) in this area, but their results are promising. While GANs are powerful generative models successfully applied in different areas, they suffer from inherent challenges such as training instability and mode collapse. Another primary concern is capturing correlations in stock prices. Therefore, the main challenges fall into two categories: capturing correlations and addressing the inherent problems of GANs. In this paper, we introduce a novel framework based on DRAGAN11Deep Regret Analytic Generative Adversarial Network. and feature matching for stock price forecasting, which improves training stability and alleviates mode collapse. We employ windowing to acquire temporal correlations by the generator and exploit conditioning on discriminator inputs to capture temporal correlations and correlations between prices and features. Experimental results on data from several stocks indicate that proposed method outperforms long short-term memory (LSTM) as a baseline method, as well as basic GANs and WGAN-GP22Wasserstein GAN – Gradient Penalty. as two different variants of GANs.

Evaluating Text-to-Image GANs Performance: A Comparative Analysis of Evaluation Metrics

Generative Adversarial Networks (GANs) have emerged as powerful techniques for generating high-quality images in various domains but assessing how realistic the generated images are is a challenging task. To address this issue, researchers have proposed a variety of evaluation metrics for GANs, each with its own strengths and limitations. This paper presents a comprehensive analysis of popular GAN evaluation metrics, including FID, Mode Score, Inception Score, MMD, PSNR, and SSIM. The strengths, weaknesses, and calculation processes of these metrics are discussed, focusing on assessing image fidelity and diversity. Two approaches, pixel distance, and feature distance, are employed to measure image similarity, while the importance of evaluating individual objects using input captions is emphasized. Experimental results on a basic GAN trained on the MNIST dataset demonstrate improvement in various metrics across different epochs. The FID score decreases from 497.54594 at Epoch 0 to 136.91156 at Epoch 100, indicating improved differentiation between real and generated images. In addition, the Inception Score increases from 1.1533 to 1.6408, reflecting enhanced image quality and diversity. These findings highlight the effectiveness of the GAN model in generating more realistic and diverse images with training progression. However, when it comes to evaluating GANs on complex datasets, challenges arise, highlighting the need to combine evaluation metrics with visual inspection and subjective measures of image quality. By adopting a comprehensive evaluation approach, researchers can gain a deeper understanding of GAN performance and guide the development of advanced models.

Researches Advanced in Finance based on Generative Adversarial Networks

Topics about how to apply Generative Adversarial Networks (GANs) to finance industry and research to promote financial market prosperity and progress with deep learning technology never fail to arouse public attention. For instance, financial academic realm such as portfolio optimization, trade execution strategies and financial information progressing have made use of machine learning. After a large number of literature retrieval and attempts to classify from different aspects, we ultimately find out that GAN is one of the most famous and appropriate technologies widely adopted in academic research and commercial applications. Aiming at assisting reader with having insight into how GAN improved financial work’s efficiency, in this paper, we propose three research perspectives of GAN in financial work: Stock Market Prediction, Fault Detection, and Time Series. By organizing from an innovative and macro perspective, we present and classify the application methods of GAN and the improvement of the basic model of GAN in order to adapt to the corresponding fields under the corresponding research directions. In general, the following improved models based on the basic GAN model for the corresponding financial perspective are involved: GAN of three-layer dense network, GAN with the MLP and LSTM separately as the discriminator and the generator, GAN-FD stock projection model, BERT and GAN in Stock Prediction; SSGANs, DAEGAN in Fault Detection; SeqGAN, Quant GAN, RNN-GAN, PSA-GAN in Time Series.

Games of GANs: game-theoretical models for generative adversarial networks

Generative Adversarial Networks (GANs) have recently attracted considerable attention in the AI community due to their ability to generate high-quality data of significant statistical resemblance to real data. Fundamentally, GAN is a game between two neural networks trained in an adversarial manner to reach a zero-sum Nash equilibrium profile. Despite the improvement accomplished in GANs in the last few years, several issues remain to be solved. This paper reviews the literature on the game-theoretic aspects of GANs and addresses how game theory models can address specific challenges of generative models and improve the GAN’s performance. We first present some preliminaries, including the basic GAN model and some game theory background. We then present a taxonomy to classify state-of-the-art solutions into three main categories: modified game models, modified architectures, and modified learning methods. The classification is based on modifications made to the basic GAN model by proposed game-theoretic approaches in the literature. We then explore the objectives of each category and discuss recent works in each class. Finally, we discuss the remaining challenges in this field and present future research directions.

A survey of generative adversarial networks and their application in text-to-image synthesis

<abstract><p>With the continuous development of science and technology (especially computational devices with powerful computing capabilities), the image generation technology based on deep learning has also made significant achievements. Most cross-modal technologies based on deep learning can generate information from text into images, which has become a hot topic of current research. Text-to-image (T2I) synthesis technology has applications in multiple fields of computer vision, such as image enhancement, artificial intelligence painting, games and virtual reality. The T2I generation technology using generative adversarial networks can generate more realistic and diverse images, but there are also some shortcomings and challenges, such as difficulty in generating complex backgrounds. This review will be introduced in the following order. First, we introduce the basic principles and architecture of basic and classic generative adversarial networks (GANs). Second, this review categorizes T2I synthesis methods into four main categories. There are methods based on semantic enhancement, methods based on progressive structure, methods based on attention and methods based on introducing additional signals. We have chosen some of the classic and latest T2I methods for introduction and explain their main advantages and shortcomings. Third, we explain the basic dataset and evaluation indicators in the T2I field. Finally, prospects for future research directions are discussed. This review provides a systematic introduction to the basic GAN method and the T2I method based on it, which can serve as a reference for researchers.</p></abstract>

Distribution Bias Aware Collaborative Generative Adversarial Network for Imbalanced Deep Learning in Industrial IoT

The impact of Internet of Things (IoT) has become increasingly significant in smart manufacturing, while deep generative model (DGM) is viewed as a promising learning technique to work with large amount of continuously generated industrial Big Data in facilitating modern industrial applications. However, it is still challenging to handle the imbalanced data when using conventional Generative Adversarial Network (GAN) based learning strategies. In this article, we propose a distribution bias aware collaborative GAN (DB-CGAN) model for imbalanced deep learning in industrial IoT, especially to solve limitations caused by distribution bias issue between the generated data and original data, via a more robust data augmentation. An integrated data augmentation framework is constructed by introducing a complementary classifier into the basic GAN model. Specifically, a conditional generator with random labels is designed and trained adversarially with the classifier to effectively enhance augmentation of the number of data samples in minority classes, while a weight sharing scheme is newly designed between two separated feature extractors, enabling the collaborative adversarial training among generator, discriminator, and classifier. An augmentation algorithm is then developed for intelligent anomaly detection in imbalanced learning, which can significantly improve the classification accuracy based on the correction of distribution bias using the rebalanced data. Compared with five baseline methods, experiment evaluations based on two real-world imbalanced datasets demonstrate the outstanding performance of our proposed model in tackling the distribution bias issue for multiclass classification in imbalanced learning for industrial IoT applications.

Virtual Data Generation for Marine Turbochargers Using GANs

A large amount and diversity of data is required for training an effective and robust deep learning or machine learning model. However, obtaining sufficient and various real-time data is difficult for bulky equipment such as marine turbochargers in practical working conditions, since exhaustive acquisition and annotation is hard and time-consuming. In this paper, the proposed methods use Generative Adversarial Networks (GANs) to generate virtual numerical data from limited number of real data. Two GAN architectures, which are basic GAN and conditional GAN, are utilized to develop the methods. The resultant virtual data can be either unlabelled or labelled. It shows that GANs can generate numerical data which is similar enough and not significantly different from the real data. Furthermore, GAN-based domain adaptation approach will be developed to generate synthetic data for predictive maintenance of a turbocharger, given by some healthy data of itself and laboratory testing data of other turbochargers.

Basic GAN Models and the Application In Medical Image Field

Generative adversarial networks (GANs) have shown its prominent performance to the world and be exploited in many areas. For example, generating human face image, translating image between different domain, generating supplementary data for tasks which lacks training and labeled data. Specially, GANs also shows its potential in medical image analysis. It can tackle the problems in medical image analysis such as medical image translation, segmentation, reconstruction, detection and image classification. Being capable of generating images at amazing level of realisim, GANs gives the opportunity to resolve the problem that labeled data for those rare diseases cannot meet the need for medical field. In this paper, we give an overview of GANs and its application in medical image analysis. Defects and advantages of those GANs methods are also thoroughly reviewed. We also discuss its potential improvement in future. We review those most frequently methods published until now. And essential details about papers we discussed in this paper and access to that paper is attached at the endpage.

Study of Generative Adversarial Networks for Acoustic Signal Enhancement: A Review

Acoustic signals enhancement is an important research topic. It has many applications like cochlear implants, speech and speaker recognition, hearing aids, mobile phones etc. The signals processed by these system are always susceptible to noises. Hence, algorithms are required to extract clean signal from noisy ones. Nowadays , deep neural network are the most sought after tool for signal enhancement. Generative Adversarial Network(GAN) is also one of the recent approaches applied to signal enhancement domain. More work is performed by GANs in image and video processing. To the best of my knowledge no review work on the usage of GANs for acoustic signal enhancement have been done. This paper is a review on the use of GANs for acoustical signals enhancement where speech signal is used as acoustic signal. The paper provides in a summarized manner about the basic GAN architectures and its limitations, feature sets used as input to GAN, limitations, performance evaluation measures and future directions.

Scope of generative adversarial networks (GANs) in image processing

Generative Adversarial Network is the topic of interest in today’s research in the field of image processing and computer vision. A basic GAN model was introduced by Ian Goodfellow et al. in 2014. After that advancement in the field of research in GAN models has been application specific. In computer vision and image to image translation GANs are playing very effective role either in the case of face detection and recognition or in image resolution enhancement and image augmentation. This paper represents a concise overview of various GAN models along with their features and applications. Pix2Pix and conditional GAN models work upon paired datasets while other models like cycle GAN, discover GAN, dual GAN, info GAN, deep convolutional GAN etc. work upon unpaired datasets. Various image datasets which are commonly used for training of generator and discriminator networks are also discussed in this paper. Since partial mode collapse is a common problem to occur during training process for all models, therefore various normalization techniques are also preferred during the training of generator and discriminator networks. As the advancements in GAN models are increasing at a very fast rate, soon these models will also be preferred in commercial applications.

Modeling of subsurface sedimentary facies using Self-Attention Generative Adversarial Networks (SAGANs)

Understanding subsurface distribution features is crucial for reliable sedimentary facies modeling. Obtaining the distribution feature of complex subsurface sedimentary facies is challenging, especially non-stationary sedimentary facies which are location-specific, well-ordered, facies sequence (e.g., deltas). The application of machine learning algorithms has potential to assist with this imaging problem, particularly using the Generative Adversarial Networks (GANs) method. Recently, GANs have proven to be outstanding for unsupervised learning on complex distributions of training images. However, in most GAN-based model convolution processes, the information in a local area is computationally invalid for reproducing the global features of training images. To remedy this, we introduce an advanced Self-Attention Generative Adversarial Network (SAGAN) for subsurface geological facies modeling. Compared with the basic GANs, SAGANs introduce a self-attention mechanism to attain details from a long distance in the image, reproducing global features of training images. SAGAN case studies involve stationary channels and non-stationary delta facies. We use probability maps, variograms, connectivity functions, and visualization results to evaluate and compare our simulation realizations. For channel cases, SAGANs’ realizations can reproduce different distributions of channels and point bars in different river systems. For the delta case, the SAGAN method shows a better ability to reproduce delta non-stationary characteristics than the MPS and basic GAN methods. All results are of high quality and diversity, reproduced the known geological sedimentary patterns, and compared with the basic GANs, SAGANs can better reproduce the global features of non-stationary training images. It is demonstrated that our first proposed SAGANs for geological facies modeling represent a powerful method for reproducing depositional facies distribution pattern.

Gradient-based conditional generative adversarial network for non-uniform blind deblurring via DenseResNet

Blind image deblurring aims to recover the sharp image from a blurry image. The problem is seriously ill-conditioned and many existing algorithms based on kernel estimation require heuristic parameter adjustments and high computational cost, and cannot perform well on non-uniform motion blurs. To address this issue, image deblurring is viewed as an image translation problem in this paper. The authors solve it based on a conditional generative adversarial network (GAN), where the sharp image is restored by an end-to-end trainable neural network. Different from the generative network in basic conditional GAN, the proposed generator is based on dense blocks and residual network (DenseResNet), aiming to mitigate the problems of overfitting and vanishing gradient, and strengthen the blur feature propagation. To generate clear structure, the basic conditional GAN formulation is further revised by introducing joint VGG features and L1-based gradient loss. Extensive experimental results demonstrate the superior performance of the proposed method.

A feature-supervised generative adversarial network for environmental monitoring during hazy days

The adverse haze weather condition has brought considerable difficulties in vision-based environmental applications. While, until now, most of the existing environmental monitoring studies are under ordinary conditions, and the studies of complex haze weather conditions have been ignored. Thence, this paper proposes a feature-supervised learning network based on generative adversarial networks (GAN) for environmental monitoring during hazy days. Its main idea is to train the model under the supervision of feature maps from the ground truth. Four key technical contributions are made in the paper. First, pairs of hazy and clean images are used as inputs to supervise the encoding process and obtain high-quality feature maps. Second, the basic GAN formulation is modified by introducing perception loss, style loss, and feature regularization loss to generate better results. Third, multi-scale images are applied as the input to enhance the performance of discriminator. Finally, a hazy remote sensing dataset is created for testing our dehazing method and environmental detection. Extensive experimental results show that the proposed method has achieved better performance than current state-of-the-art methods on both synthetic datasets and real-world remote sensing images.

Gaussian Discriminative Analysis aided GAN for imbalanced big data augmentation and fault classification

With data in industrial processes being larger in scale and easier to access, data-driven technologies have become more prevalent in process monitoring. Fault classification is an indispensable part of process monitoring, while machine learning is an effective tool for fault classification. In most practical cases, however, the number of fault data is far smaller than normal data, and this imbalance of dataset would lead to the significant decline in performance of common classifier learning algorithms. To this issue, we propose a data augmentation method, which is based on Generative Adversarial Networks(GAN) and aided by Gaussian Discriminant Analysis(GDA), for enhancement of fault classification accuracy. To validate the effectiveness of this method for imbalanced fault classification, on toy data and the Tennessee Eastman (TE) benchmark process, common oversampling method and the basic GAN are compared to our method, with different classification algorithms. Besides, proposed method is deployed and parallelly trained on Tensorflow platform, which is suitable for applications like data augmentation and imbalanced fault classification in industrial big data environments.

Improvement of Learning Stability of Generative Adversarial Network Using Variational Learning

In this paper, we propose a new network model using variational learning to improve the learning stability of generative adversarial networks (GAN). The proposed method can be easily applied to improve the learning stability of GAN-based models that were developed for various purposes, given that the variational autoencoder (VAE) is used as a secondary network while the basic GAN structure is maintained. When the gradient of the generator vanishes in the learning process of GAN, the proposed method receives gradient information from the decoder of the VAE that maintains gradient stably, so that the learning processes of the generator and discriminator are not halted. The experimental results of the MNIST and the CelebA datasets verify that the proposed method improves the learning stability of the networks by overcoming the vanishing gradient problem of the generator, and maintains the excellent data quality of the conventional GAN-based generative models.

Adversarial-learning-based image-to-image transformation: A survey

Recently, the generative adversarial network (GAN) has attracted wide attention for various computer vision tasks. GAN provides a novel concept for image-to-image transformation by means of adversarial learning. In recent years, numerous adversarial-learning-based methods have been proposed, and impressive results have been achieved. Related reviews have mainly focused on the basic GAN model and its general variants; in contrast, this survey aims to provide an overview of adversarial-learning-based methods by focusing on the image-to-image transformation scenario. First, a brief review of basic GAN is presented; next, the related approaches are roughly divided into adversarial style transfer and adversarial image restoration, e.g., super-resolution, image inpainting, and de-raining. The network architectures of generative models and loss functions are introduced and discussed in detail. Finally, we conclude the survey with an analysis of the trends and challenges.

Generative Adversarial Networks and Its Applications in Biomedical Informatics.

The basic Generative Adversarial Networks (GAN) model is composed of the input vector, generator, and discriminator. Among them, the generator and discriminator are implicit function expressions, usually implemented by deep neural networks. GAN can learn the generative model of any data distribution through adversarial methods with excellent performance. It has been widely applied to different areas since it was proposed in 2014. In this review, we introduced the origin, specific working principle, and development history of GAN, various applications of GAN in digital image processing, Cycle-GAN, and its application in medical imaging analysis, as well as the latest applications of GAN in medical informatics and bioinformatics.

Data augmentation using generative adversarial networks for robust speech recognition

For noise robust speech recognition, data mismatch between training and testing is a significant challenge. Data augmentation is an effective way to enlarge the size and diversity of training data and solve this problem. Different from the traditional approaches by directly adding noise to the original waveform, in this work we utilize generative adversarial networks (GAN) for data generation to improve speech recognition under noise conditions. In this paper we investigate different configurations of GANs. Firstly the basic GAN is applied: the generated speech samples are based on spectrum feature level and produced frame by frame without dependence among them, and there is no true labels. Thus, an unsupervised learning framework is proposed to utilize these untranscribed data for acoustic modeling. Then, in order to better guide the data generation, condition information is introduced into GAN structures, and the conditional GAN is utilized: two different conditions are explored, including the acoustic state of each speech frame and the original paired clean speech of each speech frame. With the incorporation of specific condition information into data generation, these conditional GANs can provide true labels directly, which can be used for later acoustic modeling. During the acoustic model training, these true labels are combined with the soft labels which make the model better. The proposed GAN-based data augmentation approaches are evaluated on two different noisy tasks: Aurora4 (simulated data with additive noise and channel distortion) and the AMI meeting transcription task (real data with significant reverberation). The experiments show that the new data augmentation approaches can obtain the performance improvement under all noisy conditions, which including additive noise, channel distortion and reverberation. With these augmented data by basic GAN / conditional GAN, a relative 6% to 14% WER reduction can be obtained upon an advanced acoustic model.

A review of generative adversarial networks and its application in cybersecurity

This paper reviews Generative Adversarial Networks (GANs) in detail by discussing the strength of the GAN when compared to other generative models, how GANs works and some of the notable problems with training, tuning and evaluating GANs. The paper also briefly reviews notable GAN architectures like the Deep Convolutional Generative Adversarial Network (DCGAN), and Wasserstein GAN, with the aim of showing how design specifications in these architectures help solve some of the problems with the basic GAN model. All this is done with a view of discussing the application of GANs in cybersecurity studies. Here, the paper reviews notable cybersecurity studies where the GAN plays a key role in the design of a security system or adversarial system. In general, from the review, one can observe two major approaches these cybersecurity studies follow. In the first approach, the GAN is used to improve generalization to unforeseen adversarial attacks, by generating novel samples that resembles adversarial data which can then serve as training data for other machine learning models. In the second approach, the GAN is trained on data that contains authorized features with the goal of generating realistic adversarial data that can thus fool a security system. These two approaches currently guide the scope of modern cybersecurity studies with generative adversarial networks.