Generative Adversarial Networks Research Articles

SYN-GAN: A robust intrusion detection system using GAN-based synthetic data for IoT security

As technological communication progresses, diverse datasets are exchanged across distributed environments using the Internet of Things (IoT). However, the IoT environment is vulnerable to attacking and breaching data privacy or making a robust system worse by providing attack data. To address potential risks of attacks, researchers have been conducting experiments on network intrusion detection systems (NIDS) to mitigate threats effectively. The issue of data imbalance and associated data collection costs persists, hindering the ability of machine learning (ML) models to learn malicious behaviour effectively and consequently impacting the accuracy of network threat detection. Addressing these issues, our study explores the potential of using 100% synthetic data generated via Generative Adversarial Networks (GAN) for training ML models in Network Intrusion Detection Systems (NIDS). This approach reduces the dependency on real-world data significantly, paving the way for a more flexible and ethically convenient model-building process. For the UNSW-NB15 dataset, we achieved an accuracy of 90%, a precision of 91%, a recall of 90%, and an F1 score of 89%. For the NSL-KDD dataset, our results showed an accuracy of 84%, a precision of 85%, a recall of 84%, and an F1 score of 84%. For the BoT-IoT dataset, we attained perfect scores of 100% across all metrics. These outcomes indicate that the values obtained from our analysis demonstrate high performance, yielding comparative or superior results to previous studies. Therefore, our study successfully replicates real-world network intrusion detection data, showing new opportunities for the use of generative data in cyber security.

Energy Load Forecasting Techniques in Smart Grids: A Cross-Country Comparative Analysis

Energy management systems allow the Smart Grids industry to track, improve, and regulate energy use. Particularly, demand-side management is regarded as a crucial component of the entire Smart Grids system. Therefore, by aligning utility offers with customer demand, anticipating future energy demands is essential for regulating consumption. An updated examination of several forecasting techniques for projecting energy short-term load forecasts is provided in this article. Each class of algorithms, including statistical techniques, Machine Learning, Deep Learning, and hybrid combinations, are comparatively evaluated and critically analyzed, based on three real consumption datasets from Spain, Germany, and the United States of America. To increase the size of tiny training datasets, this paper also proposes a data augmentation technique based on Generative Adversarial Networks. The results show that the Deep Learning-hybrid model is more accurate than traditional statistical methods and basic Machine Learning procedures. In the same direction, it is demonstrated that more comprehensive datasets assisted by complementary data, such as energy generation and weather, may significantly boost the accuracy of the models. Additionally, it is also demonstrated that Generative Adversarial Networks-based data augmentation may greatly improve algorithm accuracy.

Age-Invariant Face Recognition Using FaceNet and Multi-task Cascaded Convolutional Networks (MTCNN)

In an era where face recognition technology serves as a cornerstone for security, surveillance, and human-computer interaction, the persistence of age-related variations in facial appearance presents a significant challenge. This research introduces an innovative solution that amalgamates the capabilities of the FaceNet deep learning model with Multi-task Cascaded Convolutional Networks (MTCNN) to achieve precise and robust face recognition across diverse age groups. Leveraging FaceNet's prowess in extracting distinctive features from facial images and mapping them to a high-dimensional feature space for efficient face matching, our system incorporates MTCNN as a pre-processing step to accurately detect and align faces, effectively mitigating age-related changes in facial geometry. Crucially, this novel approach obviates the need for age-specific databases and age group categorization, rendering it a versatile and practical solution for age-invariant face recognition. Rigorous experimentation on benchmark datasets underscores the system's resilience and accuracy across a broad spectrum of age groups, culminating in state-of-the-art results in age-invariant face recognition. The implications of this proposed approach are profound, promising to fortify security measures and enhance user experiences in applications such as access control, personal identification, and customer service, while ensuring dependable and accurate face recognition, irrespective of an individual's age. Key Words: Face recognition, face aging, generative adversarial networks

Pointer generation and main scale detection for occluded meter reading based on generative adversarial network

The meter reading with machine vision greatly improves the efficiency of industrial monitoring. However, the pointer and scales of the meter can be occluded by rain or dirt, which greatly reduces the accuracy of the meter reading recognition. To solve this problem, we propose a generative adversarial network (PMS-GAN) with pointer generation and main scale detection for occluded meter reading. Specifically, dilated convolution block is designed to correlate separated pointer features. Then multi-scale feature fusion mechanism is proposed to guarantee the precision of pointer generation and main scale detection with guidance of semantic information. Moreover, feature enhancement mechanism is proposed to construct the long-range relationship for generating pointer under high occlusion. Finally, the reading is accomplished by calculating local angle with generated pointer and detected main scales. Experiments show that PMS-GAN can generate more intact pointer and detect main scales to guarantee the success and accuracy of occluded meter reading.

GAN-Based Anomaly Detection Tailored for Classifiers

Pattern recognition systems always misclassify anomalies, which can be dangerous for uninformed users. Therefore, anomalies must be filtered out from each classification. The main challenge for the anomaly filter design is the huge number of possible anomaly samples compared with the number of samples in the training set. Tailoring the filter for the given classifier is just the first step in this reduction. Paper tests the hypothesis that the filter trained in avoiding “near” anomalies will also refuse the “far” anomalies, and the anomaly detector is then just a classifier distinguishing between “far real” and “near anomaly” samples. As a “far real” samples generator was used, a Generative Adversarial Network (GAN) fake generator that transforms normally distributed random seeds into fakes similar to the training samples. The paper proves the assumption that seeds unused in fake training will generate anomalies. These seeds are distinguished according to their Chebyshev norms. While the fakes have seeds within the hypersphere with a given radius, the near anomalies have seeds within the sphere near cover. Experiments with various anomaly test sets have shown that GAN-based anomaly detectors create a reliable anti-anomaly shield using the abovementioned assumptions. The proposed anomaly detector is tailored to the given classifier, but its limitation is due to the need for the availability of the database on which the classifier was trained.

A multiscale and multicriteria generative adversarial network to synthesize 1-dimensional turbulent fields

This article introduces a new neural network stochastic model to generate a 1-dimensional stochastic field with turbulent velocity statistics. Both the model architecture and training procedure ground on the Kolmogorov and Obukhov statistical theories of fully developed turbulence, so guaranteeing descriptions of (1) energy distribution, (2) energy cascade and (3) intermittency across scales in agreement with experimental observations. The model is a generative adversarial network (GAN) with multiple multiscale optimization criteria. First, we use three physics-based criteria: the variance, skewness and flatness of the increments of the generated field, that retrieve respectively the turbulent energy distribution, energy cascade and intermittency across scales. Second, the GAN criterion, based on reproducing statistical distributions, is used on segments of different length of the generated field. Furthermore, to mimic multiscale decompositions frequently used in turbulence’s studies, the model architecture is fully convolutional with kernel sizes varying along the multiple layers of the model. To train our model, we use turbulent velocity signals from grid turbulence at Modane wind tunnel.

A sea–land clutter classification framework for over-the-horizon radar based on weighted loss semi-supervised generative adversarial network

Deep convolutional neural network has made great achievements in sea–land clutter classification for over-the-horizon radar (OTHR). The premise is that a large number of labeled training samples must be provided for a sea–land clutter classifier. In practical engineering applications, it is relatively easy to obtain label-free sea–land clutter samples. However, the labeling process is extremely cumbersome and requires expertise in the field of OTHR. To solve this problem, we propose an improved generative adversarial network, namely weighted loss semi-supervised generative adversarial network (WL-SSGAN). Specifically, we propose a joint feature matching loss by weighting the middle layer features of the discriminator of semi-supervised generative adversarial network (SSGAN). Furthermore, we propose the weighted loss of WL-SSGAN by linearly weighting the standard adversarial loss of SSGAN and the joint feature matching loss. The classification performance of WL-SSGAN is evaluated on sea–land clutter datasets. The experimental results show that WL-SSGAN can improve the performance of the fully supervised classifier with only a small number of labeled samples by utilizing a large number of unlabeled sea–land clutter samples. Further, the proposed weighted loss is superior to both the adversarial loss and the feature matching loss. Additionally, we compare WL-SSGAN with conventional semi-supervised classification methods and demonstrate that WL-SSGAN achieves the highest classification accuracy.

Enhancing X-ray Security Image Synthesis: Advanced Generative Models and Innovative Data Augmentation Techniques

This study addresses the field of X-ray security screening and focuses on synthesising realistic X-ray images using advanced generative models. Insufficient training data in this area pose a major challenge, which we address through innovative data augmentation techniques. We utilise the power of generative adversarial networks (GANs) and conditional GANs (cGANs), in particular the Pix2Pix and Pix2PixHD models, to investigate the generation of X-ray images from various inputs such as masks and edges. Our experiments conducted on a Korean dataset containing dangerous objects relevant to security screening show the effectiveness of these models in improving the quality and realism of image synthesis. Quantitative evaluations based on metrics such as PSNR, SSIM, LPIPS, FID, and FSIM, with scores of 19.93, 0.71, 0.12, 29.36, and 0.54, respectively, show the superiority of our strategy, especially when integrated with hybrid inputs containing both edges and masks. Overall, our results highlight the potential of advanced generative models to overcome the challenges of data scarcity in X-ray security screening and pave the way for more efficient and accurate inspection systems.

Investigating the causes of permeability anisotropy in heterogeneous conglomeratic sandstone using multiscale digital rock

Heterogeneous conglomeratic sandstone exhibits anisotropic physical properties, rendering a comprehensive analysis of its physical processes challenging with experimental measurements. Digital rock technology provides a visual and intuitive analysis of the microphysical processes in rocks, thereby aiding in scientific inquiry. Nevertheless, the multiscale characteristics of conglomeratic sandstone cannot be fully captured by a single-scale digital rock, thus limiting its ability to characterize the pore structure. Our work introduces a proposed workflow that uses multiscale digital rock fusion to investigate permeability anisotropy in heterogeneous rock. We use a cycle-consistent generative adversarial network to fuse computed tomography scan data of different resolutions, creating a large-scale and high-precision digital rock that comprehensively represents the conglomeratic sandstone pore structure. Subsequently, the digital rock is partitioned into multiple blocks, and the permeability of each block is simulated using a pore network. Finally, the total permeability of the sample is calculated by conducting an upscaling numerical simulation using the Darcy-Stokes equation. This process facilitates the analysis of the pore structure in conglomeratic sandstone and provides a step-by-step solution for permeability. From a multiscale perspective, this approach reveals that the anisotropy of permeability in conglomeratic sandstone stems from the layered distribution of grain sizes and differences in grain arrangement across different directions.

Machine learning methods for background potential estimation in 2DEGs

Two-dimensional electron gases (2DEGs) can show exceptional carrier mobility, making them promising candidates for future quantum technologies. However, impurities and defects can significantly degrade their performance, impacting transport, conductivity, and coherence times. We leverage scanning gate microscopy (SGM) and machine learning approaches to extract the potential landscape of 2DEGs from SGM data. We compare three techniques: image-to-image translation with generative adversarial networks (GANs), cellular neural networks (CNNs), and an evolutionary search algorithm. Notably, the evolutionary approach outperforms both alternatives in defect identification and analysis. This work clarifies the interaction between defects and 2DEG properties, demonstrating the potential of machine learning for understanding and manipulating quantum materials, facilitating advancements in quantum computing and nanoelectronics.

Continuous Glucose, Insulin and Lifestyle Data Augmentation in Artificial Pancreas Using Adaptive Generative and Discriminative Models.

Artificial pancreas requires data from multiple sources for accurate insulin dose estimation. These include data from continuous glucose sensors, past insulin dosage information, meal quantity and time and physical activity data. The effectiveness of closed-loop diabetes management systems might be hampered by the absence of these data caused by device error or lack of compliance by patients. In this study, we demonstrate the effect of output sequence length-driven generative and discriminative model selection in high quality data generation and augmentation. This novel generative adversarial network (GAN) based architecture automatically selects the generator and discriminator architecture based on the desired output sequence length. The proposed model is able to generate glucose, physical activity, meal information data for individual patients. The discriminative scores for Ohio T1DM (2018) dataset were 0.17 ±0.03 (Inputs: CGM, CHO, Insulin) and 0.15 ±0.02 (Inputs: CGM, CHO, Insulin, Heart Rate, Steps) and for Ohio T1D (2020) dataset was 0.16 ±0.02 (Inputs: CGM, CHO, Insulin) and 0.15 ±0.02 (Inputs: CGM, CHO, Insulin, acceleration). A mixture of generated and real data was used to test predictive scores for glucose forecasting models. The best RMSE and MARD achieved for OhioT1DM patients were 17.19 ±3.22 and 7.14 ±1.76 for PH=30min with CGM, CHO, Insulin, heartrate and steps as inputs. Similarly, the RMSE and MARD for real+synthetic data were 15.63 ±2.57 and 5.86 ±1.69 respectively. Compared to existing generative models, we demonstrate that sequence length based architecture selection leads to better synthetic data generation for multiple output sequences (CGM, CHO, Insulin) and forecasting accuracy.

Maize leaf disease image enhancement algorithm using TFEGAN

Maize leaf diseases have a profound impact on maize crop yield. However, due to practical constraints, gathering a substantial number of maize disease images is time-consuming and labor-intensive. Furthermore, low-light conditions and noise interference in maize leaf disease images present significant challenges for image generation. Therefore, this paper proposes TFEGAN, a novel algorithm designed to generate maize leaf disease images. TFEGAN leverages a Two-Way Extraction Module (TWEM) to enhance the perceptual domain of the model and effectively reuse maize leaf disease features. The algorithm employs two well-trained Generative Adversarial Networks (GANs) to enhance low-light and noisy maize leaf disease images. The low-light enhancement component utilizes a double discriminator and self-regularization to balance global and local enhancement while constraining the distance between the input image and the denoising component. Moreover, the denoising part employs a strong coordinated attention mechanism and an environmental encoder to learn noise regions and their surrounding structures, thus generating better local images and enabling more accurate evaluation by the discriminant network. The generators of both networks are encapsulated by two discriminators and combined with a bi-directional feature extraction structure to generate normal maize leaf disease images. Experimental evaluations demonstrate the effectiveness of TFEGAN in improving the quality of maize leaf disease images under low-light and noisy conditions, showcasing its potential for practical applications. Overall, this paper presents a promising solution to the challenges of low-light and noise interference in maize leaf disease images, utilizing bidirectional feature extraction and GANs.

Brain2GAN: Feature-disentangled neural encoding and decoding of visual perception in the primate brain.

A challenging goal of neural coding is to characterize the neural representations underlying visual perception. To this end, multi-unit activity (MUA) of macaque visual cortex was recorded in a passive fixation task upon presentation of faces and natural images. We analyzed the relationship between MUA and latent representations of state-of-the-art deep generative models, including the conventional and feature-disentangled representations of generative adversarial networks (GANs) (i.e., z- and w-latents of StyleGAN, respectively) and language-contrastive representations of latent diffusion networks (i.e., CLIP-latents of Stable Diffusion). A mass univariate neural encoding analysis of the latent representations showed that feature-disentangled w representations outperform both z and CLIP representations in explaining neural responses. Further, w-latent features were found to be positioned at the higher end of the complexity gradient which indicates that they capture visual information relevant to high-level neural activity. Subsequently, a multivariate neural decoding analysis of the feature-disentangled representations resulted in state-of-the-art spatiotemporal reconstructions of visual perception. Taken together, our results not only highlight the important role of feature-disentanglement in shaping high-level neural representations underlying visual perception but also serve as an important benchmark for the future of neural coding.

An improved pix2pix generative adversarial networks for sand-dust image enhancement

An improved pix2pix generative adversarial networks for sand-dust image enhancement

Integrating Pix2Pix and computational fluid dynamics for enhanced indoor airflow prediction: A case study with wing-walls

Computational Fluid Dynamics (CFD) is often used for analyzing natural ventilation but is limited in early design stages due to its high computational demands and the need for detailed inputs. To overcome such limitations of CFD simulation, this paper proposes a method using a conditional Generative Adversarial Network (cGAN) based image-to-image translation (Pix2Pix) to predict the indoor airflow condition. Compared to other traditional machine learning models that predict averaged values like indoor air velocity, Pix2Pix can predict contour plot of indoor airflow for the given building floor plan image, which provides more intuitive feedback to designers. The proposed method was tested to understand indoor air movement caused by wing-walls attached to the windows. The model was trained using 1153 pairs of floor plan images, incorporating variations in window widths, wing-wall depth, wing-wall angle, wind speed, and wind direction. It achieved a prediction accuracy of 94 % and produced results in less than a second. Moreover, the model showed relatively better performance with the changes in windows and wing-walls properties rather than wind properties. This high performance indicates that Pix2Pix can serve as an efficient proxy model of conventional CFD simulations, helping designers optimize ventilation in building designs at an early stage without the need for complex inputs.

SURVEY ON STABLE DIFFUSION TEXT TO IMAGE USING AI

The Fusion Nexus Text-to-Image Synthesis Initiative integrates cutting-edge Generative Adversarial Networks (GANs) with Natural Language Processing (NLP) techniques to narrow the semantic divide between textual input and visual output. Built upon the robust Stable Diffusion training paradigm, this initiative is engineered to produce immersive, true-to-life images based on descriptive text prompts. While GANs have exhibited potential in image generation, issues like mode collapse and training instability have impeded their effectiveness. The Fusion Nexus initiative circumvents these challenges by leveraging the Stable Diffusion framework, which furnishes a stable and reliable training methodology for GANs. By amalgamating recent advancements in deep learning, this project spearheads a novel approach to text-to-image synthesis. Its primary aim is to craft cohesive and highly realistic visual representations from textual descriptions, thereby bridging the gap between linguistic expression and visual perception. This ambitious undertaking marks a significant stride at the convergence of GANs and NLP, presenting a promising solution to the intricate task of text-to-image generation. Keywords: Adversarial, Diffusion, Framework, Fusion, GANs, Generation, Generation, Image, Language, Natural, NLP, Processing, Robust, Stable, Text-to-image, Training, Visuals.

The Style Transformation of Gu-Zheng Music Based on the Improved Model of Cyclegan

In recent years, generative adversarial networks have excelled in the field of image style migration, however their performance in the field of music has been mediocre. Existing music style migration does not work well for style migration of gu-zheng music. In order to solve these problems, we first extract the features of gu-zheng music and the Mel-spectrum features, then use CycleGAN to do style transformation on the combined features and Mel-spectrum features, and then use WaveNet vocoder to decode the migrated spectrograms, and finally achieve the style migration with gu-zheng music. The proposed model was evaluated on the publicly available dataset FMA, and the average style migration rate of the compliant music reached 94.07%. Compared to other algorithms, the music produced by this method outperformed other algorithms in terms of style migration rate and audio quality.

GANSharp: High-definition image reconstruction using generative adversarial networks

In the realm of digital imaging, enhancing low-resolution images to high-definition quality is a pivotal challenge, particularly crucial for applications in medical imaging, security, and remote sensing. Traditional methods, primarily relying on basic interpolation techniques, often result in images that lack detail and fidelity. GANSharp introduces an innovative GAN-based framework that substantially improves the generator network, incorporating adversarial and perceptual loss functions for enhanced image reconstruction. The core issue addressed is the loss of critical information during down-sampling processes. To counteract this, we proposed a GAN-based method leveraging deep learning algorithms, trained using sets of both low- and high-resolution images. Our approach, which focuses on expanding the generator network’s size and depth and integrating adversarial and perceptual loss, was thoroughly evaluated on various benchmark datasets. The experimental results showed remarkable outcomes. On the Set5 dataset, our method achieved a PSNR of 34.18 dB and a SSIM of 0.956. Comparatively, on the Set14 dataset, it yielded a PSNR of 31.16 dB and an SSIM of 0.920, and on the B100 dataset, it achieved a PSNR of 30.51 dB and an SSIM of 0.912. These results were superior or comparable to those of existing advanced algorithms, demonstrating the proposed method’s potential in generating high-quality, high-resolution images. Our research underscores the potency of GANs in image super-resolution, making it a promising tool for applications spanning medical diagnostics, security systems, and remote sensing. Future exploration could extend to the utilization of alternative loss functions and novel training techniques, aiming to further refine the efficacy of GAN-based image restoration algorithms.

Transductive zero-shot learning with generative model-driven structure alignment

Zero-shot learning (ZSL) facilitates the transfer of knowledge from seen to unseen categories through high-dimensional vectors that capture both known and unknown class names. However it encounters challenges with domain shift arising from a lack of sufficient labeled data. Although transductive zero-shot learning (TZSL) addresses this bias by including samples from unseen classes, it still faces obstacles in enhancing TZSL performance. In this study, We introduce the Structure Alignment Variational Autoencoder Generative Adversarial Network (SA-VAEGAN), a novel approach that enhances the alignment between visual and auxiliary spaces. We delved into the underlying causes of domain shift and introduced a structural alignment (SA) strategy to tackle these challenges. The SA model thoroughly accounts for both inter-class and intra-class dynamics, designed to leverage the model’s comprehension of high-level semantic relations to disambiguate confusion among similar classes and mitigate intra-class confusion by penalizing atypical visual samples within classes. Assessed across four benchmark datasets, SA-VAEGAN has established a new performance standard, underscoring its efficiency in addressing the domain shift challenge within TZSL tasks, and achieving high accuracy.

Similarity indicator and CG-CGAN prediction model for remaining useful life of rolling bearings

To tackle the challenges of performing early fault warning and improving the prediction accuracy for the remaining useful life (RUL) of rolling bearings, this paper proposes a similarity health indicator and a predictive model of CG-conditional generative adversarial network (CGAN), which relies on a CGAN that combines one-dimensional convolutional neural network (CNN) with a bidirectional gate recurrent unit (Bi-GRU). This framework provides a comprehensive theoretical foundation for RUL prediction of rolling bearings. The similarity health indicator allows for early fault warning of rolling bearings without expert knowledge. Within the CGAN framework, the inclusion of constraints guides the generation of samples in a more targeted manner. Additionally, the proposed CG-CGAN model incorporates Bi-GRU to consider both forward and backward information, thus improving the precision of RUL forecasting. Firstly, the similarity indicator between the vibration signals of the rolling bearing over its full life span and the standard vibration signals (healthy status) is calculated. This indicator helps to determine the early deterioration points of the rolling bearings. Secondly, the feature matrix composed of traditional health indicators and similarity health indicator, is utilized to train and test the proposed CG-CGAN model for RUL prediction. Finally, to corroborate the efficacy of the proposed method, two sets of real experiment data of rolling bearing accelerated life from the Intelligent Maintenance Systems (IMS) are utilized. Experimental findings substantiate that the proposed similarity health indicator offers early fault alerts and precisely delineates the performance diminution of the rolling bearing. Furthermore, the put-forward CG-CGAN model achieves high-precision RUL prediction of rolling bearing.