Generative Adversarial Network Model Research Articles

A Cross-Modal Generative Adversarial Network for Scenarios Generation of Renewable Energy

Scenarios data of renewable energy resources plays an essential role in the study of mitigating the risk in the power system due to their intermittent nature. Existing researches have mainly focused on how to generate high-quality time series based scenarios with corrupted or missing observed data. However, the multimodality of renewable energy resources is largely ignored from the perspective of the spatial-temporal correlation which can significantly strengthen the renewable energy prediction and optimization. To make full use of these multi-modal data and further improve the data quality of scenarios, a cross-modal Generative Adversarial Network (cGAN) model is proposed with the setting of two spatial-temporal transformer models. In cGAN, the task of scenarios generation is formulated as a probability approximation problem. Then, a cross-modal scenarios generation framework is constructed to process data from multi-modal observations. Theoretically, it can generate infinite number of uncertain scenarios data via the repeated random noise sampling technique. Extensive experiments are carried out at a NREL photovoltaic power dataset and a real-world wind power dataset, respectively. The results validate the state-of-the-art performance of cGAN, even though the validation dataset is missing at random. Moreover, the results also show that the setting of spatial-temporal transformer clearly explains the contributions of different modal data, and stabilize the training process of cGAN. Finally, a stochastic day-head economic dispatch is also studied to show the practical value of cGAN.

Data-driven framework for prediction and optimization of gas turbine blade film cooling

Film cooling is a crucial technique for protecting critical components of gas turbines from excessive temperatures. Multiparameter film cooling optimization is still relatively time-consuming owing to the substantial computational demands of computational fluid dynamics (CFD) methods. To reduce the computational cost, the present study develops a data-driven framework for predicting and optimizing the film cooling effectiveness of high-pressure turbines based on deep learning. Multiple rows of cooling holes located on the pressure surface of the turbine blade are optimized, with the coolant hole diameter, the incline angle, and the compound angle as design parameters. A conditional generative adversarial network model combining a gated recurrent unit and a convolutional neural network is designed to establish the complex nonlinear regression between the design parameters and the film cooling effectiveness. The surrogate model is trained and tested using independent CFD results. A sparrow search algorithm and the well-trained surrogate model are combined to acquire the optimal film cooling parameters. The proposed framework is found to improve multi-row film cooling effectiveness by 21.2% at an acceptable computational cost.

Hyperspectral Image Classification Based on Mutually Guided Image Filtering

Hyperspectral remote sensing images (HSIs) have both spectral and spatial characteristics. The adept exploitation of these attributes is central to enhancing the classification accuracy of HSIs. In order to effectively utilize spatial and spectral features to classify HSIs, this paper proposes a method for the spatial feature extraction of HSIs based on a mutually guided image filter (muGIF) and combined with the band-distance-grouped principal component. Firstly, aiming at the problem that previously guided image filtering cannot effectively deal with the inconsistent information structure between the guided and target information, a method for extracting spatial features using muGIF is proposed. Then, aiming at the problem of the information loss caused by a single principal component as a guided image in the traditional GIF-based spatial–spectral classification, a spatial feature-extraction framework based on the band-distance-grouped principal component is proposed. The method groups the bands according to the band distance and extracts the principal components of each set of band subsets as the guide map of the current band subset to filter the HSIs. A deep convolutional neural network model and a generative adversarial network model for the filtered HSIs are constructed and then trained using samples for HSIs’ spatial–spectral classification. Experiments show that compared with the traditional methods and several popular spatial–spectral HSI classification methods based on a filter, the proposed methods based on muGIF can effectively extract the spatial–spectral features and improve the classification accuracy of HSIs.

A multi-task spatio-temporal generative adversarial network for prediction of travel time reliability in peak hour periods

Travel time reliability (TTR) serves as a crucial indicator for evaluating the efficiency and service quality of a road traffic network. This paper proposes a multi-task spatio-temporal generative adversarial network (MTST-GAN) model that simultaneously predicts the TTR in morning and evening peak hour periods. The model incorporates multi-graph convolutional networks to extract spatial correlations from travel time data, while long short-term memory neural networks are employed to consider temporal correlations. Additionally, self-attention mechanisms are applied to the proposed MTST-GAN model to further capture spatial and temporal features. A feature fusion bridge is constructed to integrate the spatial and temporal features learned by each task. Through a numerical experiment conducted on a road network in a Chinese city, our findings demonstrate that the proposed model outperforms several state-of-the-art approaches in terms of Jensen-Shannon divergence, mean, standard deviation, and buffer time indices. Finally, we provide conclusions and suggest areas for further research.

Deep learning‐ and image processing‐based methods for automatic estimation of leaf herbivore damage

Abstract Quantifying the intensity of leaf herbivory pressure is crucial for understanding the interaction between plants and herbivores in both applied and basic science. Visual estimates and digital analysis have been commonly used to estimate leaf herbivore damage but are time‐consuming which limits the amount of data that can be collected and prevent answering big picture questions that require large‐scale sampling of herbivory pressure. Recent developments in deep learning have provided a potential tool for automatic collection of ecological data from various sources. However, most applications have focused on identification and counting, and there is a lack of deep learning tools for quantitative estimation of leaf herbivore damage. Here, we trained generative adversarial networks (GANs) to predict the intact status of damaged leaves and applied image processing technique to estimate the area and percentage of leaf damage. We first described procedures for collecting leaf images, training GAN models, predicting intact leaves and calculating leaf area, with a Python package provided to enable hands‐on application of these procedures. Then, we collected a large leaf data set to train a universal deep learning model and developed an online app HerbiEstim to allow direct use of pretrained models to estimate herbivory damage of leaves. We tested these methods using both simulated and real leaf damage data. The procedures provided in our study greatly improved the efficiency of leaf herbivore damage estimation. Our test demonstrated that the reconstruction of damaged leaf image resembled the ground‐truth image with a similarity of 98.8%. The estimation of leaf herbivore damage exhibited a high accuracy with an averaged root mean square error of 1.6% and had a general applicability to different plant taxa and leaf shapes. Overall, our work demonstrated the feasibility of applying deep learning techniques to quantify leaf herbivory intensity. The use of GANs allows automatic estimation of leaf damage, representing a major advantage of the method. The Python package and the online app with pre‐trained models will facilitate the use of our method for the analysis of large data sets of plant–herbivore interactions.

Application of electrical impedance tomography imaging technology combined with generative adversarial network in pulmonary ventilation monitoring

Electrical impedance tomography (EIT) plays a crucial role in the monitoring of pulmonary ventilation and regional pulmonary function test. However, the inherent ill-posed nature of EIT algorithms results in significant deviations in the reconstructed conductivity obtained from voltage data contaminated with noise, making it challenging to obtain accurate distribution images of conductivity change as well as clear boundary contours. In order to enhance the image quality of EIT in lung ventilation monitoring, a novel approach integrating the EIT with deep learning algorithm was proposed. Firstly, an optimized operator was introduced to enhance the Kalman filter algorithm, and Tikhonov regularization was incorporated into the state-space expression of the algorithm to obtain the initial lung image reconstructed. Following that, the imaging outcomes were fed into a generative adversarial network model in order to reconstruct accurate lung contours. The simulation experiment results indicate that the proposed method produces pulmonary images with clear boundaries, demonstrating increased robustness against noise interference. This methodology effectively achieves a satisfactory level of visualization and holds potential significance as a reference for the diagnostic purposes of imaging modalities such as computed tomography.

Research on Parkinson's disease recognition algorithm based on sample enhancement

Parkinson's disease patients have early vocal cord damage, and their voiceprint characteristics differ significantly from those of healthy individuals, which can be used to identify Parkinson's disease. However, the samples of the voiceprint dataset of Parkinson's disease patients are insufficient, so this paper proposes a double self-attention deep convolutional generative adversarial network model for sample enhancement to generate high-resolution spectrograms, based on which deep learning is used to recognize Parkinson's disease. This model improves the texture clarity of samples by increasing network depth and combining gradient penalty and spectral normalization techniques, and a family of pure convolutional neural networks (ConvNeXt) classification network based on Transfer learning is constructed to extract voiceprint features and classify them, which improves the accuracy of Parkinson's disease recognition. The validation experiments of the effectiveness of this paper's algorithm are carried out on the Parkinson's disease speech dataset. Compared with the pre-sample enhancement, the clarity of the samples generated by the proposed model in this paper as well as the Fréchet inception distance (FID) are improved, and the network model in this paper is able to achieve an accuracy of 98.8%. The results of this paper show that the Parkinson's disease recognition algorithm based on double self-attention deep convolutional generative adversarial network sample enhancement can accurately distinguish between healthy individuals and Parkinson's disease patients, which helps to solve the problem of insufficient samples for early recognition of voiceprint data in Parkinson's disease. In summary, the method effectively improves the classification accuracy of small-sample Parkinson's disease speech dataset and provides an effective solution idea for early Parkinson's disease speech diagnosis.

Human-robot collaborative disassembly enabled by brainwaves and improved generative adversarial network

Human-robot collaboration (HRC) can greatly facilitate the disassembling processes of end-of-life (EoL) products. For the robot in HRC, an intuitive control function enabled by a brain-machine interface (BMI) and human brainwaves will be useful to support dynamic decision-making under various disassembly conditions. However, a major challenge in developing a BMI-enabled HRC disassembly system is that there are usually a limited number of brainwave signals available for system training. To address the challenge, a novel model, namely TCN-D2GAN, was designed in this study. The model was used to augment acquired brainwave signals and support the establishment of the BMI-enabled HRC disassembly system. The innovations of the TCN-D2GAN model come from the following aspects: (i) the strengths of a temporal convolutional network (TCN) and a dual discrimination generative adversarial network (D2GAN) were combined to accomplish high-performance signal augmentation, and (ii) new loss functions were defined in the model to avoid the use of hyperparameters, so the training effect of the model can be optimised. Based on two public electroencephalography (EEG) datasets, experiments for signal augmentation and system training were carried out. The TCN-D2GAN model was compared with several mainstream generative adversarial network (GAN) models. Results indicated that the TCN-D2GAN model exhibited better performance in terms of signal classification accuracy and training efficiency. Finally, disassembly experiments were presented to validate the applicability of the system.

CGAN-ECT: Reconstruction of Electrical Capacitance Tomography images from capacitance measurements using Conditional Generative Adversarial Networks

Due to the rapid growth of applications of Electrical Capacitance Tomography (ECT) across diverse industrial sectors, there is a pressing demand for swift and high-fidelity methodologies for image reconstruction from raw capacitance measurements. Deep learning, as an effective non-linear mapping tool for complicated functions, has been going viral in many fields including electrical tomography. This paper introduces a pioneering contribution by proposing a Conditional Generative Adversarial Network (CGAN) model explicitly tailored for the purpose of reconstructing ECT images from capacitance measurements. We propose a novel modulator to transform the raw input capacitance measurements into a 2D image. To the best of our knowledge, this is the first time to represent the capacitance measurements in an image form. We have created a new massive ECT dataset of synthetic image-measurements pairs for training and testing purposes of the proposed model. This extensive ECT dataset comprises 320,000 synthetic image-measurement pairs. It forms the basis for evaluating the feasibility and generalizability of the CGAN-ECT model, with assessments conducted on testing data, contaminated data, and flow patterns which are intentionally excluded from the model’s training phase. The evaluation results prove the efficacy of the proposed CGAN model in generating ECT images with heightened accuracy compared to traditional methods and alternative deep learning-based image reconstruction algorithms. The CGAN-ECT model has succeeded in a achieving an average image correlation coefficient exceeding 99.3%, coupled with an average relative image error of approximately 0.07.

General multi-attack detection for continuous-variable quantum key distribution with local local oscillator

Abstract Continuous-variable quantum key distribution with a local local oscillator(LLO CVQKD) has been extensively researched due to its simplicity and security. For the practical security of the LLO CVQKD system, there are two main attack modes referred to as reference pulse attack and polarization attack presently. However, there is currently no general defense strategy against such attacks and the security of the system needs further investigation. Here, we employ a deep learning framework called generative adversarial networks (GANs) to detect both attacks. We first analyze the data in different cases, derive a feature vector as input to a GAN model, and then show the training and testing process of the GAN model for attack classification. The proposed model has two parts, a discriminator and a generator, both of which employ a convolutional neural network (CNN) to improve accuracy. Simulation results show that the proposed scheme can detect and classify attacks without reducing the secret key rate and the maximum transmission distance. It only establishes a detection model by monitoring features of the pulse without adding additional devices.

CycleGAN-Based Data Augmentation for Subgrade Disease Detection in GPR Images with YOLOv5

Vehicle-mounted ground-penetrating radar (GPR) technology is an effective means of detecting railway subgrade diseases. However, existing methods of GPR data interpretation largely rely on manual identification, which is not only inefficient but also highly subjective. This paper proposes a semi-supervised deep learning method to identify railway subgrade diseases. This method addresses the sample imbalance problem in the defect dataset by utilizing a data augmentation method based on a generative adversarial network model. An initial network model for disease identification is obtained by training the YOLOv5 network with a small number of existing samples. The intelligently extended samples are then labeled to achieve a balance in the disease samples. The network is trained to improve the recognition accuracy of the intelligent model using a more complete dataset. The experimental results show that the accuracy of the proposed method can reach up to 94.53%, which is 23.85% higher than that of the supervised learning model without an extended dataset. This has strong industrial application value for railway subgrade disease detection as the potential learning ability of the model can be explored to a greater extent, thereby improving the recognition accuracy of subgrade diseases.

LPSRGAN: Generative adversarial networks for super-resolution of license plate image

This paper proposes a super-resolution algorithm for reconstructing license plate images based on generative adversarial networks (GAN) for improving the recognition rate of low-resolution license plate images. To this end, this paper first designs a new image degradation model, called the n-stage random combination degradation model (n-RCD), which extends the traditional degradation model horizontally and vertically to better simulate the features of low-resolution license plate images in natural scenes. Subsequently, to address the problem that existing GAN models cannot adapt to the complex degradation space in natural scenes, this paper proposes a new network structure called LPSRGAN, which optimizes the SRGAN model from the generator and discriminator to improve its super-resolution capability. In addition, this paper also proposes a perceptual optical character recognition (OCR) loss for license plate images, which defines the connectionist temporal classification (CTC) loss of the output of the OCR network with character label values to better preserve the character details and features of the license plate images. Finally, the experiments reported in this paper show that the proposed algorithm improves the recognition rate of the original low-resolution license plate image by 12.48%, and the recognition rate of the reconstructed image reaches 93.90%. Based on this, this paper also tests the algorithm on actual captured data, and the results show that it performs well in natural scenes, demonstrating the algorithm has broad application prospects.

IMDE-UGAN: Improved Memetic Direction Exploitation Optimized U-Net Generative Adversarial Network for Classification of Diabetic Retinopathy

The most important human organ is the eye which is essential for the processing and absorbing of external visual information. Diabetic retinopathy (DR) is the primary factor contributing to the global increase in blindness. To maintain eye health, people have been paying lots of attention to the problem. However, there are still lots of people who have eye problems all around the world. Thus, we proposed the Improved Memetic Direction Exploitation Optimized U-Net Generative Adversarial Network (IMDE-optimized U-Net GAN) model for DR grading. Also, for Multimodal Optimization Problems (MMOPs), a Niching Competition-based Memetic Direction Exploitation (NC-MDE) method is proposed with adaptive local search operations. Additionally, SPatially-ADaptivE (SPADE)-based conditional GANs are used to segment DR. The U-Net GAN model is used to categorize DR images. In the discriminator design, the standard classification network is modified to create the decoder-encoder network known as U-Net. The complete network error of the U-Net GAN architecture is modeled using the IMDE archives. The various performance metrics are used in the method for testing the proposed including accuracy, specificity, sensitivity, and ROC curve. As a result, the proposed method achieves 98.89% of accuracy, 98.54% for sensitivity, and specificity, it achieves 98.67%.

Enhancing diabetic retinopathy detection through preprocessing and feature extraction with MGA-CSG algorithm

Anticipatory monitoring of diabetic retinal (DR) disease is crucial in preventing vision loss and blindness, making it a leading cause of worldwide vision impairment. In this study, we propose a novel technique, the Modified Generative Adversarial-based Crossover Salp Grasshopper (MGA-CSG) approach, for early prediction and accurate classification of diabetic retinal diseases using fundus image datasets. The proposed MGA-CSG approach combines deep learning with an optimization algorithm to achieve superior classification accuracy. We start with pre-processing of data to attenuate image variation, convert intensities, denoise, and enhance contrast in the fundus images. Next, a Convolutional Neural Network (CNN) is used for feature extraction, capturing essential information from the images. To address the limited amount of feature vectors generated by the CNN and improve learning uncertainty, we employ a Generative Adversarial Network (GAN) model to augment the feature vectors. It generates additional images, allowing the classifier to train with fewer input samples, leading to improved classification performance. One of the key novelties of our approach lies in the utilization of both labeled and unlabeled data samples, enhancing the GAN training performance. We replace the binary classifier in the GAN with a multiclass classifier, enabling separate discrimination of retinal disease classes. The MGA-CSG approach leverages the crossover Grasshopper Optimizer Algorithm (GOA) and Salp Swarm Algorithm (SSA) to optimize the model and avoid falling into local optimal solutions. This optimization process further enhances the accuracy of diabetic retinal disease classification. Experimental results demonstrate the superiority of the proposed MGA-CSG approach, achieving high accuracy, precision, recall, specificity, F1-measure, and Kappa coefficient in predicting glaucoma, diabetic retinopathy, cataract, macular edema, and myopia. The classification achieved impressive accuracy, precision, recall, specificity, F1-measure, and Kappa coefficient of 98.8%, 98.7%, 96.5%, 97.1%, 98.2%, and 95%, respectively. Overall, our novel MGA-CSG approach offers an efficient and accurate method for early screening and classification of diabetic retinal diseases. The combination of CNN-based feature extraction, GAN-based augmentation, and MGA-CSG optimization contributes to the model's high performance. This approach holds significant potential in improving healthcare management and providing timely interventions for patients with retinal diseases.

Leveraging coverless image steganography to hide secret information by generating anime characters using GAN

There has been a growing need for secure communication in recent years, particularly in the digital realm. Researchers have developed various text-hiding methods to address this need, but these methods often suffer from limitations such as low embedding capacity or high embedding distortion. However, current methods for information hiding require a large image database or have a low capacity for hiding information, which makes them impractical. To address these issues, we propose a coverless information-hiding method that generates anime characters through generative adversarial networks (GANs). This method converts secret information into an attribute label set for anime characters and uses the label set to generate anime characters directly. The quality of the generated anime characters was improved by the super-resolution GAN (SRGAN) model. The resulting images were used to communicate secret information on the digital channel. Numerous experiments were conducted to evaluate the performance of the proposed framework, including hiding capacity, image clarity, robustness and security. Results showed that the proposed framework outperforms existing text-hiding methods in terms of hiding capacity and robustness while maintaining image clarity and computation time. In conclusion, our proposed framework provides a secure and efficient solution for text encryption and decryption in feasible computation time using anime characters generated by a GAN. The framework has a high embedding capacity and low embedding distortion, making it a promising solution for secure communication in the digital world.

Exploring techniques and overcoming hurdles in generative AI

The realm of artificial intelligence has witnessed significant advancements, with generative models standing at the forefront of this progress. Generative Artificial Intelligence concerns the development of algorithms and models equipped to generate novel content - be it images, text, or music. This paper delves into the primary techniques underpinning generative AI, including Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), and autoregressive models like Transformers. These methodologies have enabled a myriad of applications, from synthesizing images to facilitating data augmentation and style transfer. While the results from generative models have been profoundly impressive, they are not devoid of challenges. The surge in their capabilities has brought forth issues related to ethics, inherent biases, scalability, and the quest for more stable training methods. This paper aims to provide an insightful exploration of the pivotal methods defining generative AI while shedding light on the prevailing challenges and ethical implications intertwined with its growth.

Auto-segmentation of pelvic organs at risk on 0.35T MRI using 2D and 3D Generative Adversarial Network models

PurposeManual recontouring of targets and Organs At Risk (OARs) is a time-consuming and operator-dependent task. We explored the potential of Generative Adversarial Networks (GAN) to auto-segment the rectum, bladder and femoral heads on 0.35T MRIs to accelerate the online MRI-guided-Radiotherapy (MRIgRT) workflow. Methods3D planning MRIs from 60 prostate cancer patients treated with 0.35T MR-Linac were collected. A 3D GAN architecture and its equivalent 2D version were trained, validated and tested on 40, 10 and 10 patients respectively. The volumetric Dice Similarity Coefficient (DSC) and 95th percentile Hausdorff Distance (HD95th) were computed against expert drawn ground-truth delineations. The networks were also validated on an independent external dataset of 16 patients. ResultsIn the internal test set, the 3D and 2D GANs showed DSC/HD95th of 0.83/9.72 mm and 0.81/10.65 mm for the rectum, 0.92/5.91 mm and 0.85/15.72 mm for the bladder, and 0.94/3.62 mm and 0.90/9.49 mm for the femoral heads. In the external test set, the performance was 0.74/31.13 mm and 0.72/25.07 mm for the rectum, 0.92/9.46 mm and 0.88/11.28 mm for the bladder, and 0.89/7.00 mm and 0.88/10.06 mm for the femoral heads. The 3D and 2D GANs required on average 1.44 s and 6.59 s respectively to generate the OARs’ volumetric segmentation for a single patient. ConclusionsThe proposed 3D GAN auto-segments pelvic OARs with high accuracy on 0.35T, in both the internal and the external test sets, outperforming its 2D equivalent in both segmentation robustness and volume generation time.

Dunhuang murals image restoration method based on generative adversarial network

AbstractMurals are an important part of China’s cultural heritage. After more than a 1000 years of exposure to the sun and wind, most of these ancient murals have become mottled, with damage such as cracking, mold, and even large-scale detachment. It is an urgent work to restore these damaged murals. The technique of digital restoration of mural images refers to the reconstruction of structures and textures to virtually fill in the damaged areas of the image. Existing digital restoration methods have the problems of incomplete restoration and distortion of local details. In this paper, we propose a generative adversarial network model combining a parallel dual convolutional feature extraction depth generator and a ternary heterogeneous joint discriminator. The generator network is designed with the mechanism of parallel extraction of image features by vanilla convolution and dilated convolution, capturing multi-scale features simultaneously, and reasonable parameter settings reduce the loss of image information. A pixel-level discriminator is proposed to identify the pixel-level defects of the captured image, and its joint global discriminator and local discriminator discriminate the generated image at different levels and granularities. In this paper, we create the Dunhuang murals dataset and validate our method on this dataset, and the experimental results show that the method of this paper has an overall improvement in the evaluation metrics of PSNR and SSIM compared with the comparative methods. The restored resultant image is more in line with the subjective vision of human beings, which achieves the effective restoration of mural images.

DAGAN: A GAN Network for Image Denoising of Medical Images Using Deep Learning of Residual Attention Structures

Medical images are susceptible to noise and artifacts, so denoising becomes an essential pre-processing technique for further medical image processing stages. We propose a medical image denoising method based on dual-attention mechanism for generative adversarial networks (GANs). The method is based on a GAN model with fused residual structure and introduces a global skip-layer connection structure to balance the learning ability of the shallow and deep networks. The generative network uses a residual module containing channel and spatial attention for efficient extraction of CT image features. The mean square error loss and perceptual loss are introduced to construct a composite loss function to optimize the model loss function, which helps to improve the image generation effect of the model. Experimental results on the LUNA dataset and “the 2016 Low-Dose CT Grand Challenge” dataset show that DAGAN achieves the best results in root mean square error (RMSE), structural similarity (SSIM) and peak signal-to-noise ratio (PSNR) when compared to the state-of-the-art methods. In particular, PSNR reaches 31.2308 dB and 27.5265 dB, SSIM reaches 0.9115 and 0.7895, while RMSE is 0.0082 and 0.0112, respectively. This indicates that our method performs better than the state-of-the-art methods in the task of CT image denoising.

An application of generative AI for knitted textile design in fashion

In recent years, artificial intelligence (AI) in the form of generative deep learning models have proliferated as a tool to facilitate or exhibit creativity across various design fields. When it comes to fashion design, existing applications of AI have more heavily addressed general fashion design elements, such as style, silhouette, colour, and pattern, and paid less attention to the underlying textile attributes. To address this gap, this study explores the effects of applying a generative deep learning model specifically towards the textile component of the fashion design process, by utilizing a Generative Adversarial Network (GAN) model to generate new images of knitted textile designs, which were then assessed based on their aesthetic quality in a qualitative survey with over 200 respondents. The results suggest that the generative deep learning (GAN) based method has the ability to produce new textile designs with creative qualities and practical utility that facilitate the fashion design process.