Traditional Generative Adversarial Networks Research Articles

Unveiling the potential of progressive training diffusion model for defect image generation and recognition in industrial processes

Industrial defect image generation is a crucial technique for augmenting defect image data, catering to fulfill the exigencies of training data necessary for defect recognition methods. In recent years, generative adversarial networks have seen widespread use in generating defect images. However, constrained by the scarcity of training data and the intricate variability in various defect categories, the existing methods are susceptible to issues such as training instability, a dearth of diversity, and suboptimal image quality. To overcome these challenges, this paper proposes a defect image generation framework based on a progressive training diffusion model (PTDM). Firstly, this study adopts a denoising diffusion model, supplanting traditional generative adversarial networks, to mitigate issues related to training instability and the dearth of diversity observed in generated images. Secondly, a novel progressive training strategy based on the self-designed image quality evaluator is developed to efficiently generate numerous defect images while maintaining quality. Finally, extensive experiments on a steel surface defect image dataset were conducted to validate the performance of the proposed framework in defect image generation and recognition tasks.

A time-saving fault diagnosis using simplified fast GAN and triple-type data transfer learning

Existing intelligent fault diagnosis approaches demand substantial data for training diagnostic models. However, factors such as the inherent characteristics of bearings, operating conditions, and privacy security make collecting comprehensive fault-bearing data very difficult. Although generating synthetic data through generative adversarial networks (GANs) is feasible, the data generation of GANs is a time-consuming process. To address these challenges, a fault diagnosis framework based on GAN and deep transfer learning (DTL) is proposed, termed the simplified fast GAN triple-type data transfer learning (SFGAN-TDTL) method. Initially, an SFGAN is proposed as a replacement for traditional GANs. The time-frequency image data generated by SFGAN serve to augment the training dataset, offering faster and higher-quality data generation compared to traditional GANs. To further reduce time consumption for GAN-based methods, the TDTL method is proposed. Differing from DTL, which utilizes synthetic data to construct a pre-trained model and conducts targeted fine-tuning with real data, TDTL employs open-source data, synthetic data, and real data to fill the weights of the task-insensitive layer, task-sensitive layer, and fully connected layer, respectively. Numerical results demonstrate that SFGAN-TDTL maintains higher diagnostic accuracy while significantly reducing time consumption.

A sparse data gas sensor array feature mining method for rubber Mooney viscosity measurement

Mooney viscosity is an important index to reflect the performance and quality of rubber. At present, the rubber Mooney test has the problems of large time delay, destructive and unable to detect online, which restricts the development of rubber industry. In this paper, a gas sensor array-based online inspection method for rubber Mooney viscosity is proposed to improve the problems of measurement delay and raw material waste in the traditional method. A multiple generator time series generative adversarial network (MGTSGAN) structure is proposed to address the problem that the lack of sample data volume and uneven data distribution make it difficult to model. Transformer is introduced to solve the problem of traditional generative adversarial networks in dealing with long sequential dependencies. In the experimental part, a rubber Mooney viscosity detection device is built to verify the effectiveness of the proposed method. The performance of different generative models on two gas sensor datasets is compared to verify the advancement and generalization of the proposed method. The experimental results show that the correct rate of this paper’s method for rubber Mooney viscosity classification is higher than 96%. The correct rates are all improved after data enhancement, in which the MGTSGAN proposed in this paper obtains the highest correct rate of 98.35%. For the classification experiments on Gas sensor array under flow modulation dataset also achieved relatively good results. Among them, the highest correct classification rate of 95.63% is achieved after data enhancement using MGTSGAN.

Synergizing meat Science and AI: Enhancing long-chain saturated fatty acids prediction

In the field of the food industry, establishing a global predictive model for the content of long-chain saturated fatty acids (LC-SFAs) in red meat is of profound significance. However, this work requires the accumulation of a large number of diverse samples for model calibration. To address these formidable challenges, the Generative Inference Adversarial Autoencoder (GI-AAE) was adopted to enhance the model training process. Through multiple iterations, GI-AAE successfully generated simulated samples of higher quality than traditional generative adversarial networks and autoencoders. After incorporating these generated samples into the training dataset, multiple sets of models showed significant improvements. These results clearly demonstrated that our approach can effectively overcome the challenges posed by generative models in the regression domain, greatly enhancing the model's generalization ability. This advancement holds significant potential for applications in the food industry.

Data augmentation-based enhanced fingerprint recognition using deep convolutional generative adversarial network and diffusion models

The progress of fingerprint recognition applications encounters substantial hurdles due to privacy and security concerns, leading to limited fingerprint data availability and stringent data quality requirements. This article endeavors to tackle the challenges of data scarcity and data quality in fingerprint recognition by implementing data augmentation techniques. Specifically, this research employed two state-of-the-art generative models in the domain of deep learning, namely Deep Convolutional Generative Adversarial Network (DCGAN) and the Diffusion model, for fingerprint data augmentation. Generative Adversarial Network (GAN), as a popular generative model, effectively captures the features of sample images and learns the diversity of the sample images, thereby generating realistic and diverse images. DCGAN, as a variant model of traditional GAN, inherits the advantages of GAN while alleviating issues such as blurry images and mode collapse, resulting in improved performance. On the other hand, Diffusion, as one of the most popular generative models in recent years, exhibits outstanding image generation capabilities and surpasses traditional GAN in some image generation tasks. The experimental results demonstrate that both DCGAN and Diffusion can generate clear, high-quality fingerprint images, fulfilling the requirements of fingerprint data augmentation. Furthermore, through the comparison between DCGAN and Diffusion, it is concluded that the quality of fingerprint images generated by DCGAN is superior to the results of Diffusion, and DCGAN exhibits higher efficiency in both training and generating images compared to Diffusion.

Generating Artistic Portraits from Face Photos with Feature Disentanglement and Reconstruction

Generating artistic portraits from face photos presents a complex challenge that requires high-quality image synthesis and a deep understanding of artistic style and facial features. Traditional generative adversarial networks (GANs) have made significant strides in image synthesis; however, they encounter limitations in artistic portrait generation, particularly in the nuanced disentanglement and reconstruction of facial features and artistic styles. This paper introduces a novel approach that overcomes these limitations by employing feature disentanglement and reconstruction techniques, enabling the generation of artistic portraits that more faithfully retain the subject’s identity and expressiveness while incorporating diverse artistic styles. Our method integrates six key components: a U-Net-based image generator, an image discriminator, a feature-disentanglement module, a feature-reconstruction module, a U-Net-based information generator, and a cross-modal fusion module, working in concert to transform face photos into artistic portraits. Through extensive experiments on the APDrawing dataset, our approach demonstrated superior performance in visual quality, achieving a significant reduction in the Fréchet Inception Distance (FID) score to 61.23, highlighting its ability to generate more-realistic and -diverse artistic portraits compared to existing methods. Ablation studies further validated the effectiveness of each component in our method, underscoring the importance of feature disentanglement and reconstruction in enhancing the artistic quality of the generated portraits.

SSDNet: A SEMISUPERVISED DEEP GENERATIVE ADVERSARIAL NETWORK FOR ELECTROENCEPHALOGRAM-BASED EMOTION RECOGNITION

This research introduces a novel method for emotion recognition using Electroencephalography (EEG) signals, leveraging advancements in emotion computing and EEG signal processing. The proposed method utilizes a semisupervised deep convolutional generative adversarial network (SSDNet) as the central model. The model fully integrates the feature extraction methods of the generative adversarial network (GAN), deep convolutional GAN (DCGAN), spectrally normalized GAN (SSGAN) and the encoder, maximizing the advantages of each method to construct a more accurate emotion classification model. In our study, we introduce a flow-form-consistent merging pattern, which successfully addresses mismatches between the data by fusing the EEG data with the features. Implementing this merging pattern not only enhances the uniformity of the input but also decreases the computational load on the network, resulting in a more efficient model. By conducting experiments on the DEAP and SEED datasets, we evaluate the SSDNet model proposed in this paper in detail. The experimental results show that the accuracy of the proposed algorithm is improved by 6.4% and 8.3% on the DEAP and SEED datasets, respectively, compared to the traditional GAN. This significant improvement in performance validates the effectiveness and feasibility of the SSDNet model. The research contributions of this paper are threefold. First, we propose and implement an SSDNet model that integrates multiple feature extraction methods, providing a more accurate and comprehensive solution for emotion recognition tasks. Second, by introducing a flow-form-consistent merging pattern, we successfully address the problem of interdata mismatches and improve the generalization performance of the model. Finally, we experimentally demonstrate that the method in this paper achieves a significant improvement in accuracy over the traditional GANs on the DEAP and SEED datasets, providing an innovative solution in the field of EEG-based emotion recognition.

A novel wasserstein generative adversarial network for stochastic wind power output scenario generation

AbstractA novel Wasserstein generative adversarial network (WGAN) is proposed for stochastic wind power output scenario generation. Wasserstein distance with gradient penalty adapts to the gradient vanishing problem that is easy to occur in the new energy generation scenario. This model has better robustness and generalization ability than the traditional generative adversarial network. WGAN is optimized to simulate ideal wind power scenarios. The generated data are measured by cumulative distribution function (CDF) and continuously ranked probability score to evaluate the performance of the proposed model. Compared with the probability models, the proposed model is data‐driven, that is, it can simulate wind power scenarios based on historical samples rather than probability hypothesis, and it can independently learn the space‐time correlation of wind power generation in different locations. Experiments show that the CDF curve of data generated by the proposed WGAN is highly coincident with that of real data.

Research on micro parallax adversarial sample generation method based on texture sensitive region

Deep neural networks have been extensively applied in fields such as image classification, object detection, and face recognition. However, research has shown that adversarial samples with subtle perturbations can effectively deceive these networks. Existing methods for generating such adversarial images often lack stealth and robustness. In this study, we present an enhanced attack strategy based on traditional Generative Adversarial Networks (GANs). We integrate image texture into the unsupervised training scheme, guiding the model to focus perturbations in high-texture areas. We also introduce a dynamic equilibrium training strategy that employs Differential Evolution algorithms to adaptively adjust both network weight parameters and the training ratio between the generator and discriminator, achieving a self-balancing training process. Further, we propose an image local optimization algorithm to eliminate perturbations in non-sensitive areas through weighted filtering. The model is validated using benchmark datasets such as MNIST, ImageNet and SVHN. Through extensive experimental evaluations, our approach shows a 4.93% improvement in attack success rate against conventional models and a 10.23% increase against defense models compared to state-of-the-art attack methods.

Biomarkers-Aware Asymmetric Bibranch GAN with Adaptive Memory Batch Normalization for Prediction of Anti-VEGF Treatment Response in Neovascular Age-related Macular Degeneration.

The emergence of anti-vascular endothelial growth factor (anti-VEGF) therapy has revolutionized neovascular age-related macular degeneration (nAMD). Post-therapeutic optical coherence tomography (OCT) imaging facilitates the prediction of therapeutic response to anti-VEGF therapy for nAMD. Although the generative adversarial network (GAN) is a popular generative model for post-therapeutic OCT image generation, it is realistically challenging to gather sufficient pre- and post-therapeutic OCT image pairs, resulting in overfitting. Moreover, the available GAN-based methods ignore local details, such as the biomarkers that are essential for nAMD treatment. To address these issues, a Biomarkers-aware Asymmetric Bibranch GAN (BAABGAN) is proposed to efficiently generate post-therapeutic OCT images. Specifically, one branch is developed to learn prior knowledge with a high degree of transferability from large-scale data, termed the source branch. Then, the source branch transfer knowledge to another branch, which is trained on small-scale paired data, termed the target branch. To boost the transferability, a novel Adaptive Memory Batch Normalization (AMBN) is introduced in the source branch, which learns more effective global knowledge that is impervious to noise via memory mechanism. Also, a novel Adaptive Biomarkers-aware Attention (ABA) module is proposed to encode biomarkers information into latent features of target branches to learn finer local details of biomarkers. The proposed method outperforms traditional GAN models and can produce high-quality post-treatment OCT pictures with limited data sets, as shown by the results of experiments.

Wasserstein adversarial learning based temporal knowledge graph embedding

Research on knowledge graph embedding (KGE) has emerged as an active field in which most existing KGE approaches mainly focus on static structural data and ignore the influence of temporal variation involved in time-aware triples. In order to deal with this issue, several temporal knowledge graph embedding (TKGE) approaches have been proposed to integrate temporal and structural information. However, these methods only employ a uniformly random sampling to construct negative facts. As a consequence, the corrupted samples are often too simplistic for training an effective model. In this paper, we propose a new temporal knowledge graph embedding framework by introducing adversarial learning to further refine the performance of traditional TKGE models. In our framework, a generator is utilized to construct high-quality plausible quadruples and a discriminator learns to obtain the embeddings of entities and relations based on both positive and negative samples. Meanwhile, we also apply a Gumbel-Softmax relaxation and the Wasserstein distance to prevent vanishing gradient problems on discrete data; an inherent flaw in traditional generative adversarial networks. Through comprehensive experimentation on temporal datasets, the results indicate that our proposed framework can attain significant improvements based on benchmark models and also demonstrate the effectiveness and applicability of our framework.

Image Inpainting based on Gated Convolution and spectral Normalization

Traditional image inpainting methods based on deep learning have the problem of insufficient discrimination between the missing area and the global area information in the feature extraction of image inpainting tasks because of the characteristics of the model constructed by the convolutional layer. At the same time, the traditional generative adversarial network often has problems such as training difficulties and model collapse in the training process. To solve the above problems and improve the repair effect of the model, this paper proposes a dual discriminator image inpainting model based on generative adversarial network combining gated convolution and spectral normalization. The model is mainly composed of an image inpainting module and an image recognition module. The traditional image inpainting model considers all input pixels as valid pixels when extracting the features of the image to be inpainted, which is unreasonable for the image inpainting task. In order to solve this problem, the gated convolution is designed to replace the role of traditional convolution in the image inpainting module. Gated convolutions address the irrationality of ordinary convolutions in image inpainting tasks by providing learnable dynamic feature selection mechanisms for each channel at each spatial location in all layers. At the same time, generative image inpainting models usually have problems such as training hard mode collapse during the training process. In this paper, we intend to introduce the spectral normalization mechanism in the convolutional layer design of the discriminator module. By introducing Lipschitz continuity constraints from the spectral norm of the parameter matrix of each layer of the neural network, the neural network is better insensitive to input perturbations, making the training process more stable and easier to converge. It solves the problems of mode collapse and model training difficulty in the training of image inpainting model based on generative adversarial network. Finally, qualitative and quantitative experiments show that the image inpainting model based on gated convolution and spectral normalization solves the above problems, and the inpainted image has reasonable texture structure and contextual semantic information.

Jujube quality grading using a generative adversarial network with an imbalanced data set

Jujube is a significant economic crop in Xinjiang, China and its quality grading is the key link in the process of its storage, processing and distribution. However, there is a considerable error in the current jujube quality grading methods mainly due to the imbalanced data. Recently, Generative Adversarial Network (GAN), a novel data augmentation technology, has provided a promising solution to address data imbalance. Nevertheless, traditional GANs currently suffer from unstable training and poor generation quality. A new GAN model (named Jujube-GAN) is proposed for jujube data augmentation. In the Jujube-GAN, the self-attention mechanism is embedded into the generator and vectorising the scalar output of the discriminator stabilises the model and improves the quality of generated jujube images. The experiment result demonstrated that Jujube-GAN could synthesise higher quality images compared to traditional GANs, with lower Frechet Inception Distance (FID) scores of 63.18, 55.24 and 51.35 for three types of defective jujube images. Furthermore, the classification model achieved 13.27% increasing in the F1 score after data augmentation by Jujube-GAN. The data augmentation method proposed in this paper provides a fresh perspective and opens up possibilities for addressing data imbalance issues in other plant sciences.

CWGAN-GP with residual network model for lithium-ion battery thermal image data expansion with quantitative metrics

Lithium batteries find extensive applications in energy storage. Temperature is a crucial indicator for assessing the state of lithium-ion batteries, and numerous experiments require thermal images of lithium-ion batteries for research purposes. However, acquiring thermal imaging samples of lithium-ion battery faults is challenging due to factors such as high experimental costs and associated risks. To address this, our study proposes the utilization of a Conditional Wasserstein Generative Adversarial Network with Gradient Penalty and Residual Network (CWGAN-GP with Residual Network) to augment the dataset of thermal images depicting lithium-ion battery faults. We employ various evaluation metrics to quantitatively analyze and compare the generated thermal images of lithium-ion batteries. Subsequently, the expanded dataset, comprising four types of thermal images depicting lithium-ion battery faults, is input into a Mask Region-based Convolutional Neural Network for training. The results demonstrate that the proposed model surpasses both traditional Generative Adversarial Network and Wasserstein Generative Adversarial Network in terms of the quality of generated thermal images of lithium-ion batteries. Moreover, the augmentation of the dataset leads to an improvement in the fault diagnosis accuracy of the Mask Region-based Convolutional Neural Network.

Autoencoder-based conditional optimal transport generative adversarial network for medical image generation

Medical image generation has recently garnered significant interest among researchers. However, the primary generative models, such as Generative Adversarial Networks (GANs), often encounter challenges during training, including mode collapse. To address these issues, we proposed the AE-COT-GAN model (Autoencoder-based Conditional Optimal Transport Generative Adversarial Network) for the generation of medical images belonging to specific categories. The training process of our model comprises three fundamental components. The training process of our model encompasses three fundamental components. First, we employ an autoencoder model to obtain a low-dimensional manifold representation of real images. Second, we apply extended semi-discrete optimal transport to map Gaussian noise distribution to the latent space distribution and obtain corresponding labels effectively. This procedure leads to the generation of new latent codes with known labels. Finally, we integrate a GAN to train the decoder further to generate medical images. To evaluate the performance of the AE-COT-GAN model, we conducted experiments on two medical image datasets, namely DermaMNIST and BloodMNIST. The model’s performance was compared with state-of-the-art generative models. Results show that the AE-COT-GAN model had excellent performance in generating medical images. Moreover, it effectively addressed the common issues associated with traditional GANs.

2S-BUSGAN: A Novel Generative Adversarial Network for Realistic Breast Ultrasound Image with Corresponding Tumor Contour Based on Small Datasets.

Deep learning (DL) models in breast ultrasound (BUS) image analysis face challenges with data imbalance and limited atypical tumor samples. Generative Adversarial Networks (GAN) address these challenges by providing efficient data augmentation for small datasets. However, current GAN approaches fail to capture the structural features of BUS and generated images lack structural legitimacy and are unrealistic. Furthermore, generated images require manual annotation for different downstream tasks before they can be used. Therefore, we propose a two-stage GAN framework, 2s-BUSGAN, for generating annotated BUS images. It consists of the Mask Generation Stage (MGS) and the Image Generation Stage (IGS), generating benign and malignant BUS images using corresponding tumor contours. Moreover, we employ a Feature-Matching Loss (FML) to enhance the quality of generated images and utilize a Differential Augmentation Module (DAM) to improve GAN performance on small datasets. We conduct experiments on two datasets, BUSI and Collected. Moreover, results indicate that the quality of generated images is improved compared with traditional GAN methods. Additionally, our generated images underwent evaluation by ultrasound experts, demonstrating the possibility of deceiving doctors. A comparative evaluation showed that our method also outperforms traditional GAN methods when applied to training segmentation and classification models. Our method achieved a classification accuracy of 69% and 85.7% on two datasets, respectively, which is about 3% and 2% higher than that of the traditional augmentation model. The segmentation model trained using the 2s-BUSGAN augmented datasets achieved DICE scores of 75% and 73% on the two datasets, respectively, which were higher than the traditional augmentation methods. Our research tackles imbalanced and limited BUS image data challenges. Our 2s-BUSGAN augmentation method holds potential for enhancing deep learning model performance in the field.

EarlGAN: An enhanced actor–critic reinforcement learning agent-driven GAN for de novo drug design

Deep generative models, such as Generative Adversarial Networks (GANs), have attracted the attention of researchers in the denovo drug design field. However, traditional GANs are typically used for image processing. Therefore, they are unsuitable for Simplified Molecular-Input Line-Entry System (SMILES) strings due to their discrete nature. Previous studies addressed this problem by combining Reinforcement Learning (RL) approaches with Monte Carlo tree search. However, for large chemical datasets, the molecule generation process is time-consuming due to the lengthy atom-by-atom sampling process with cumulative reward, an essence of the Monte Carlo tree search-based RL approaches. To address this problem, we propose an enhanced actor–critic RL agent-driven GAN, called EarlGAN, for denovo drug design. Specifically, EarlGAN’s generator acts as an actor to generate SMILES strings, and the discriminator acts as a critic to perform discrimination. EarlGAN makes autoregressive predictions at the atomic level. While the generator is based on previously generated atoms, the discriminator discriminates using a bidirectional pass over the atoms, including the current atom that is being predicted. We integrate moment, global-level discrimination rewards, and information entropy maximization. The moment rewards reduce the computation time, and the global-level rewards ensure the consistency of the molecule, whereas the information entropy maximization leads to a more diverse sample generation. Experiments and ablation studies verify the effectiveness of EarlGAN for denovo drug design on the QM9 and ZINC datasets. In addition, the visualization analysis provides insight into EarlGAN and supports our conclusion.

A Novel Data Augmentation Method Based on CoralGAN for Prediction of Part Surface Roughness.

Deep learning networks can be applied to the field of intelligent prediction of part surface roughness. However, the surface roughness samples of parts have the problems of high collection cost, unbalanced categories, and complicated data distribution, which inevitably limit the application of deep learning network models in the field of intelligent prediction of part surface roughness. To solve these problems, this article proposes a novel data augmentation method based on CoralGAN for prediction of part surface roughness, which introduces the domain adaptive method deep coral function to help optimize the network parameters of the generator of generative adversarial network (GAN). Specifically, the vibration signal collected during processing is converted into frequency spectrum data and input into CoralGAN. The training of the generator is guided by coral loss, that is, the distance between the covariances of the real samples and generated samples features, not just the statistical consistency of the traditional GAN. Experiments have been carried out on the three-axis vertical machining center. Research shows that the proposed method can improve the prediction accuracy of part surface roughness to 95.5%.

Domain adaptive networks with limited data for rotating machinery fault diagnosis: a case of study of gears

Rotating machinery is one of the most common components in the industry. Therefore, timely and accurate fault diagnosis of rotating machinery is essential for the regular operation of equipment. At present, some achievements have been made in rotating machinery fault diagnosis based on a large number of marked fault data. However, most of the machines are in a normal state in actuality. Especially, the machines run under different loads, so it is costly to collect a large number of labeled fault data under different load distributions. To solve rotating machinery fault diagnosis in different load conditions with limited samples, a domain adaptive group convolutional neural network is proposed. Firstly, the least squares generative adversarial networks were used to expand the limited target sample data. By changing the objective function, the two defects of the low quality of the vibration signal generated by the traditional generative adversarial networks and the unstable training process are optimized. Secondly, the raw vibration signals in the source domain are pre-trained by the group convolutional neural network, and the group training network effectively reduces network parameters. Finally, the source domain signals and target domain signals were trained in domain adversarial networks to diagnose different distributed data in target domains. The proposed method is validated by collecting the raw vibration signals of gears under different loads and different health states, and the effectiveness of the proposed method is proved. Experimental validation shows that the method proposed in this paper achieves an average accuracy improvement of more than 12% compared to other existing methods.

Image Generation Method Based on Improved Generative Adversarial Network

Background: The image generation model based on generative adversarial network (GAN) has achieved remarkable achievements. However, traditional GAN has the disadvantage of unstable training, which affects the quality of the generated image. Objective: This method is to solve the GAN image generation problems of poor image quality, single image category, and slow model convergence. Methods: An improved image generation method is proposed based on GAN. Firstly, the attention mechanism is introduced into the convolution layer of the generator and discriminator and a batch normalization layer is added after each convolution layer. Secondly, the ReLU and leaky ReLU are used as the active layer of the generator and discriminator, respectively. Thirdly, the transposed convolution is used in the generator while the small step convolution is used in the discriminator, respectively. Fourthly, a new discarding method is applied in the dropout layer. Results: The experiments were carried out on Caltech 101 dataset. The experimental results showed that the image quality generated by the proposed method is better than that generated by GAN with attention mechanism (AM-GAN) and GAN with stable training strategy (STS-GAN) and the stability was improved. Conclusion: The proposed method is effectiveness for image generation with high quality.