Generative Adversarial Networks Research Articles

LIC-CGAN: fast lithography latent images calculation method for large-area masks using deep learning

Latent image calculation for large-area masks is an indispensable but time-consuming step in lithography simulation. This paper presents LIC-CGAN, a fast method for three-dimensional (3D) latent image calculation of large-area masks using deep learning. Initially, the library of mask clips and their corresponding latent images is established, which is then used to train conditional generative adversarial networks (CGANs). The large area layout is divided into mask clips based on local pattern features. If a mask clip matches one from the training library, its latent image can be obtained directly. Otherwise, the CGANs are employed to calculate its local latent image. Finally, all local latent images are synthesized to simulate the entire latent image. The proposed method is applied to lithography simulations for display panels, demonstrating high accuracy and a speed-up of 2.5 to 4.7 times compared to the rigorous process.

LW-DCGAN: a lightweight deep convolutional generative adversarial network for enhancing occluded face recognition

LW-DCGAN: a lightweight deep convolutional generative adversarial network for enhancing occluded face recognition

Enhanced Diabetes Detection and Blood Glucose Prediction Using TinyML-Integrated E-Nose and Breath Analysis: A Novel Approach Combining Synthetic and Real-World Data

Diabetes mellitus, a chronic condition affecting millions worldwide, necessitates continuous monitoring of blood glucose level (BGL). The increasing prevalence of diabetes has driven the development of non-invasive methods, such as electronic noses (e-noses), for analyzing exhaled breath and detecting biomarkers in volatile organic compounds (VOCs). Effective machine learning models require extensive patient data to ensure accurate BGL predictions, but previous studies have been limited by small sample sizes. This study addresses this limitation by employing conditional generative adversarial networks (CTGAN) to generate synthetic data from real-world tests involving 29 healthy and 29 diabetic participants, resulting in over 14,000 new synthetic samples. These data were used to validate machine learning models for diabetes detection and BGL prediction, integrated into a Tiny Machine Learning (TinyML) e-nose system for real-time analysis. The proposed models achieved an 86% accuracy in BGL identification using LightGBM (Light Gradient Boosting Machine) and a 94.14% accuracy in diabetes detection using Random Forest. These results demonstrate the efficacy of enhancing machine learning models with both real and synthetic data, particularly in non-invasive systems integrating e-noses with TinyML. This study signifies a major advancement in non-invasive diabetes monitoring, underscoring the transformative potential of TinyML-powered e-nose systems in healthcare applications.

Symmetric Connected U-Net with Multi-Head Self Attention (MHSA) and WGAN for Image Inpainting

This study presents a new image inpainting model based on U-Net and incorporating the Wasserstein Generative Adversarial Network (WGAN). The model uses skip connections to connect every encoder block to the corresponding decoder block, resulting in a strictly symmetrical architecture referred to as Symmetric Connected U-Net (SC-Unet). By combining SC-Unet with a GAN, the study aims to reconstruct images more effectively and seamlessly. The traditional discriminators only differentiate the entire image as true or false. In this study, the discriminator calculated the probability of each pixel belonging to the hole and non-hole regions, which provided the generator with more gradient loss information for image inpainting. Additionally, every block of SC-Unet incorporated a Dilated Convolutional Neural Network (DCNN) to increase the receptive field of the convolutional layers. Our model also integrated Multi-Head Self-Attention (MHSA) into selected blocks to enable it to efficiently search the entire image for suitable content to fill the missing areas. This study adopts the publicly available datasets CelebA-HQ and ImageNet for evaluation. Our proposed algorithm demonstrates a 10% improvement in PSNR and a 2.94% improvement in SSIM compared to existing representative image inpainting methods in the experiment.

DMFGAN: a multifeature data augmentation method for grape leaf disease identification.

The use of deep learning techniques to identify grape leaf diseases relies on large, high-quality datasets. However, a large number of images occupy more computing resources and are prone to pattern collapse during training. In this paper, a depth-separable multifeature generative adversarial network (DMFGAN) was proposed to enhance grape leaf disease data. First, a multifeature extraction block (MFEB) based on the four-channel feature fusion strategy is designed to improve the quality of the generated image and avoid the problem of poor feature learning ability of the adversarial generation network caused by the single-channel feature extraction method. Second, a depth-based D-discriminator is designed to improve the discriminator capability and reduce the number of model parameters. Third, SeLU activation function was substituted for DCGAN activation function to overcome the problem that DCGAN activation function was not enough to fit grape leaf disease image data. Finally, an MFLoss function with a gradient penalty term is proposed to reduce the mode collapse during the training of generative adversarial networks. By comparing the visual indicators and evaluation indicators of the images generated by different models, and using the recognition network to verify the enhanced grape disease data, the results show that the method is effective in enhancing grape leaf disease data. Under the same experimental conditions, DMFGAN generates higher quality and more diverse images with fewer parameters than other generative adversarial networks. The mode breakdown times of generative adversarial networks in training process are reduced, which is more effective in practical application.

A highly naturalistic facial expression generation method with embedded vein features based on diffusion model

Abstract Facial expression generation technology has achieved notable progress in computer vision and artificial intelligence. However, challenges persist regarding background consistency, expression clarity, and detailed representation. Additionally, the instability of generative adversarial networks (GANs) during training affects both image quality and diversity. While diffusion models have demonstrated potential advantages over GANs, research on controllable expression generation remains limited. To address these challenges, this paper proposes a highly natural facial expression generation method based on denoising diffusion implicit models (DDIM) with embedded vein features. This approach avoids adversarial training by employing gradual diffusion to generate specific expressions, thereby enhancing both the diversity and authenticity of the images. Vein features are introduced and embedded within the generated expression images to protect the intellectual property of algorithm-generated digital resources. Firstly, image and expression text guide words are combined as conditional inputs to improve the authenticity and diversity of the generated images. Secondly, a classification coding network is introduced to guide expression generation, thus enhancing the accuracy and consistency of the produced expressions. Furthermore, this paper proposes a vein feature fusion method based on multi-directional local dynamic feature coding operator (MLDFO) and integrates DDIM with frequency-domain watermarking technology to achieve image intellectual property protection. Experimental results demonstrate the effectiveness of this method across several public datasets, including FFHQ, CelebA, FV-USM, and SDUMLA-HMT. Notably, in the CelebA dataset, the average expression recognition rate increased by 11.41%, with a 100.00% recognition rate for happy expressions. The generated expression images exhibit a high degree of authenticity and consistency, and the video conversion tests reveal a natural and smooth effect. These results confirm that this method not only advances facial expression generation technology but also significantly enhances the steganographic protection of images.

A Decentralized Digital Watermarking Framework for Secure and Auditable Video Data in Smart Vehicular Networks

Thanks to the rapid advancements in Connected and Automated Vehicles (CAVs) and vehicular communication technologies, the concept of the Internet of Vehicles (IoVs) combined with Artificial Intelligence (AI) and big data promotes the vision of an Intelligent Transportation System (ITS). An ITS is critical in enhancing road safety, traffic efficiency, and the overall driving experience by enabling a comprehensive data exchange platform. However, the open and dynamic nature of IoV networks brings significant performance and security challenges to IoV data acquisition, storage, and usage. To comprehensively tackle these challenges, this paper proposes a Decentralized Digital Watermarking framework for smart Vehicular networks (D2WaVe). D2WaVe consists of two core components: FIAE-GAN, a novel feature-integrated and attention-enhanced robust image watermarking model based on a Generative Adversarial Network (GAN), and BloVA, a Blockchain-based Video frames Authentication scheme. By leveraging an encoder–noise–decoder framework, trained FIAE-GAN watermarking models can achieve the invisibility and robustness of watermarks that can be embedded in video frames to verify the authenticity of video data. BloVA ensures the integrity and auditability of IoV data in the storing and sharing stages. Experimental results based on a proof-of-concept prototype implementation validate the feasibility and effectiveness of our D2WaVe scheme for securing and auditing video data exchange in smart vehicular networks.

Research on Attack Detection for Traffic Signal Systems Based on Game Theory and Generative Adversarial Networks

Research on Attack Detection for Traffic Signal Systems Based on Game Theory and Generative Adversarial Networks

Enhanced MRI-based brain tumour classification with a novel Pix2pix generative adversarial network augmentation framework.

The scarcity of medical imaging datasets and privacy concerns pose significant challenges in artificial intelligence-based disease prediction. This poses major concerns to patient confidentiality as there are now tools capable of extracting patient information by merely analysing patient's imaging data. To address this, we propose the use of synthetic data generated by generative adversarial networks as a solution. Our study pioneers the utilisation of a novel Pix2Pix generative adversarial network model, specifically the 'image-to-image translation with conditional adversarial networks,' to generate synthetic datasets for brain tumour classification. We focus on classifying four tumour types: glioma, meningioma, pituitary and healthy. We introduce a novel conditional deep convolutional neural network architecture, developed from convolutional neural network architectures, to process the pre-processed generated synthetic datasets and the original datasets obtained from the Kaggle repository. Our evaluation metrics demonstrate the conditional deep convolutional neural network model's high performance with synthetic images, achieving an accuracy of 86%. Comparative analysis with state-of-the-art models such as Residual Network50, Visual Geometry Group 16, Visual Geometry Group 19 and InceptionV3 highlights the superior performance of our conditional deep convolutional neural network model in brain tumour detection, diagnosis and classification. Our findings underscore the efficacy of our novel Pix2Pix generative adversarial network augmentation technique in creating synthetic datasets for accurate brain tumour classification, offering a promising avenue for improved disease prediction and treatment planning.

Automated tree crown labeling with 3D radiative transfer modelling achieves human comparable performances for tree segmentation in semi-arid landscapes

Mapping tree crowns in arid or semi-arid areas, which cover around one-third of the Earth’s land surface, is a key methodology towards sustainable management of trees. Recent advances in deep learning have shown promising results for tree crown segmentation. However, a large amount of manually labeled data is still required. We here propose a novel method to delineate tree crowns from high resolution satellite imagery using deep learning trained with automatically generated labels from 3D radiative transfer modeling, intending to reduce human annotation significantly. The methodological steps consist of 1) simulating images with a 3D radiative transfer model, 2) image style transfer learning based on generative adversarial network (GAN) and 3) tree crown segmentation using U-net segmentation model. The delineation performances of the proposed method have been evaluated on a manually annotated dataset consisting of more than 40,000 tree crowns. Our approach, which relies solely on synthetic images, demonstrates high segmentation accuracy, with an F1 score exceeding 0.77 and an Intersection over Union (IoU) above 0.64. Particularly, it achieves impressive accuracy in extracting crown areas (r2 greater than 0.87) and crown densities (r2 greater than 0.72), comparable to that of a trained dataset with human annotations only. In this study, we demonstrated that the integration of a 3D radiative transfer model and GANs for automatically generating training labels can achieve performances comparable to human labeling, and can significantly reduce the time needed for manual labeling in remote sensing segmentation applications.

A TransISP Based Image Enhancement Method for Visual Disbalance in Low‐light Images

AbstractExisting image enhancement algorithms often fail to effectively address issues of visual disbalance, such as brightness unevenness and color distortion, in low‐light images. To overcome these challenges, we propose a TransISP‐based image enhancement method specifically designed for low‐light images. To mitigate color distortion, we design dual encoders based on decoupled representation learning, which enable complete decoupling of the reflection and illumination components, thereby preventing mutual interference during the image enhancement process. To address brightness unevenness, we introduce CNNformer, a hybrid model combining CNN and Transformer. This model efficiently captures local details and long‐distance dependencies between pixels, contributing to the enhancement of brightness features across various local regions. Additionally, we integrate traditional image signal processing algorithms to achieve efficient color correction and denoising of the reflection component. Furthermore, we employ a generative adversarial network (GAN) as the overarching framework to facilitate unsupervised learning. The experimental results show that, compared with six SOTA image enhancement algorithms, our method obtains significant improvement in evaluation indexes (e.g., on LOL, PSNR: 15.59%, SSIM: 9.77%, VIF: 9.65%), and it can improve visual disbalance defects in low‐light images captured from real‐world coal mine underground scenarios.

An advancement in AdaSyn for imbalanced learning: An application to fraud detection in digital transactions

Imbalanced Learning is a significant issue in machine learning, affecting the performance and accuracy of binary or multi-classification algorithms, especially in large-scale data handling and classification. There are some popular techniques to covert this imbalanced data into a balanced one such as undersampling, under-sampling with tomek links, randomized oversampling, synthetic minority oversampling technique (SMOTE), and adaptive synthetic generation (ADASYN). Generally, the ADASYN algorithm could be used to propagate minority sample points to rise the imbalanced ratio between majority and minority sample points, but in some cases, it may conflict with decision boundary points and noisy points. This paper proposed a Refitted AdaSyn Algorithm (RAA) with Gaussian Distribution (GD). So that new minority samples are distributed much closer to the center of the minority sample to spotlight the conflicts. The classification accuracy has improved with RAA over formal ADASYN. For examining the proposed work the imbalanced benchmark datasets like European, Banksim, Paymentcard, and UCI credit card are considered. Vanilla Generative Adversarial Network (GAN) is a deep learning model used to classify fraud and non-fraud transactions, demonstrating significant differences between balanced and imbalanced learning approaches and achieving an accuracy of 97.5% on dataset DS4.

Automated construction site layout design system for prefabricated buildings using transformer based conditional GAN

Construction site layout plans (CSLP) are crucial for efficient prefabricated construction project management. Traditional manual design process is costly and time-consuming, while optimization methods heavily depend on expert knowledge. Recent advancements in deep generative models present promising alternatives. However, their application to the generation of prefabricated construction site layouts is hindered by several challenges, including limited datasets, significant overlap between facilities, and the necessity to generate layouts based on fixed facilities with specific attributes such as minimal transportation costs. These challenges constrain the efficacy and applicability of the generated layouts. To address these issues, this study introduces an innovative automated generative design system for prefabricated construction site layouts, leveraging a novel Transformer-based conditional generative adversarial network (GAN). The data preparation module of the system collects and augments layout data for training. The CSLGAN module is designed to generate layouts that conform to spatial constraints and desired attributes, with minimal facility overlap. Furthermore, this study establishes benchmarks in terms of model capacity and specialized performance metrics. Extensive experiments demonstrate the effectiveness of the proposed system in automated construction site layout generation.

Report on the AAPM grand challenge on deep generative modeling for learning medical image statistics.

The findings of the 2023 AAPM Grand Challenge on Deep Generative Modeling for Learning Medical Image Statistics are reported in this SpecialReport. The goal of this challenge was to promote the development of deep generative models for medical imaging and to emphasize the need for their domain-relevant assessments via the analysis of relevant imagestatistics. As part of this Grand Challenge, a common training dataset and an evaluation procedure was developed for benchmarking deep generative models for medical image synthesis. To create the training dataset, an established 3D virtual breast phantom was adapted. The resulting dataset comprised about 108000 images of size 512 512. For the evaluation of submissions to the Challenge, an ensemble of 10000 DGM-generated images from each submission was employed. The evaluation procedure consisted of two stages. In the first stage, a preliminary check for memorization and image quality (via the Fréchet Inception Distance [FID]) was performed. Submissions that passed the first stage were then evaluated for the reproducibility of image statistics corresponding to several feature families including texture, morphology, image moments, fractal statistics, and skeleton statistics. A summary measure in this feature space was employed to rank the submissions. Additional analyses of submissions was performed to assess DGM performance specific to individual feature families, the four classes in the training data, and also to identify various artifacts. Fifty-eight submissions from 12 unique users were received for this Challenge. Out of these 12 submissions, 9 submissions passed the first stage of evaluation and were eligible for ranking. The top-ranked submission employed a conditional latent diffusion model, whereas the joint runners-up employed a generative adversarial network, followed by another network for image superresolution. In general, we observed that the overall ranking of the top 9 submissions according to our evaluation method (i) did not match the FID-based ranking, and (ii) differed with respect to individual feature families. Another important finding from our additional analyses was that different DGMs demonstrated similar kinds ofartifacts. This Grand Challenge highlighted the need for domain-specific evaluation to further DGM design as well as deployment. It also demonstrated that the specification of a DGM may differ depending on its intendeduse.

Deep learning assisted cancer disease prediction from gene expression data using WT-GAN

Several diverse fields including the healthcare system and drug development sectors have benefited immensely through the adoption of deep learning (DL), which is a subset of artificial intelligence (AI) and machine learning (ML). Cancer makes up a significant percentage of the illnesses that cause early human mortality across the globe, and this situation is likely to rise in the coming years, especially when non-communicable illnesses are not considered. As a result, cancer patients would greatly benefit from precise and timely diagnosis and prediction. Deep learning (DL) has become a common technique in healthcare due to the abundance of computational power. Gene expression datasets are frequently used in major DL-based applications for illness detection, notably in cancer therapy. The quantity of medical data, on the other hand, is often insufficient to fulfill deep learning requirements. Microarray gene expression datasets are used for training procedures despite their extreme dimensionality, limited volume of data samples, and sparsely available information. Data augmentation is commonly used to expand the training sample size for gene data. The Wasserstein Tabular Generative Adversarial Network (WT-GAN) model is used for the data augmentation process for generating synthetic data in this proposed work. The correlation-based feature selection technique selects the most relevant characteristics based on threshold values. Deep FNN and ML algorithms train and classify the gene expression samples. The augmented data give better classification results (> 97%) when using WT-GAN for cancer diagnosis.

Data-driven stochastic dynamic economic dispatch for combined heat and power systems using particle swarm optimization

In modern power systems, the uncertain behavior of renewable energy sources (RESs) may result in deviations from the optimal operating dispatches for the various power sources. The electric power generation from Combined Heat and Power (CHP) units is characterized by high efficiency with less pollution. To use CHP units more efficiently, the dynamic economic dispatch problem is applied to obtain the optimal power and heat sources’ outputs to satisfy heat and power demands while meeting the different operational constraints. In this paper, data-driven stochastic optimization is used to model these uncertainties utilizing the Generative Adversarial Networks in scenario generation that are accurate and do not require fitting models or modeling probability distributions. Then, the Fast Forward Selection technique is used to decrease the number of scenarios to improve system tractability. The results obtained indicate that the variability in electrical load, photovoltaic (PV), and wind turbine (WT) output significantly impacts the overall operational costs of the power system. Therefore, it is essential to incorporate the uncertainties associated with electrical load, PV, and WT when aiming for accurate results in economic dispatch. the study found that the overall operation cost of the dynamic economic dispatch of the electrical system for 24 hours after integrating RES achieved a daily savings of 2.5 % in the first scenario. Furthermore, the findings demonstrate that RESs positively contribute to reducing the total operational costs of the power system. The economic dispatch of CHP systems is influenced by the integration of RESs and the uncertainties associated with electrical parameters.

TransImg: A Translation Algorithm of Visible-to-Infrared Image Based on Generative Adversarial Network

Infrared images of sensitive targets are difficult to obtain and cannot meet the design and training needs of target detection and tracking algorithms for mobile platforms such as aircraft. This paper proposes an image translation algorithm TransImg, which can achieve visible light image translation to the infrared domain to enrich the dataset. First, the algorithm designed a generator structure consisting of a deep residual connected encoder and a region perception feature fusion module to enhance feature learning, thereby avoiding issues such as generating infrared images with insufficient details in the transfer task. Afterward, a multi-scale discriminator and a composite loss function were designed to further improve the transfer effect. Finally, an automatic mixed-precision training strategy was designed for the overall migration algorithm architecture to accelerate the training and generation of infrared images. Experiments have shown that the image translation algorithm TransImg has good algorithm accuracy, and the infrared image generated by visible light image translation has richer texture details, faster generation speed, and lower video memory consumption, and the performance exceeds the mainstream traditional algorithm, and the generated images can meet the requirements of target detection and tracking algorithms design and training for mobile platforms such as aircraft.

Retraction Note: Rainfall prediction using generative adversarial networks with convolution neural network

Retraction Note: Rainfall prediction using generative adversarial networks with convolution neural network

Frequency Domain-Based Super Resolution Using Two-Dimensional Structure Consistency for Ultra-High-Resolution Display

Recent advancements in the field of super resolution (SR) have seen the adoption of generative adversarial networks (GANs) for realistic images. In this case, when performing with low-resolution (LR) images, several challenges arise due to the loss of high-frequency details from high-resolution (HR) images, potentially leading to unwanted distortions in the generated SR images. Our paper presents a novel solution by using two-dimensional structure consistency (TSC) for image analysis. The TSC serves as a mask, enabling adaptive analysis based on the unique frequency characteristics of different image regions. Furthermore, a mutual loss mechanism, which dynamically adjusts the training process based on the results filtered by the TSC-based mask, is introduced. Additionally, the TSC loss is proposed to enhance our model capacity to generate precise TSC in high-frequency regions. As a result, our method effectively reduces distortions in high-frequency areas while preserving clarity in regions containing low-frequency components. Our method outperforms other SR techniques, demonstrating superior results in both qualitative and quantitative evaluations. Quantitative measurements, including PSNR, SSIM, and the perceptual metric LPIPS, show comparable PSNR and SSIM values, while the perceptual SR quality is notably improved according to the LPIPS metric.

Comparison Between Gans and Diffusion Models in the Generation of Synthetic Images for Enhancing Tree Species Recognition

Objective: This study investigates the application of Generative Adversarial Networks (GANs) and Diffusion Models in data augmentation to improve the classification of tree species images, which is essential for sustainable forest management. Theoretical Framework: Fundamental concepts of machine learning, generative and classifier networks, as well as data augmentation techniques through synthetic image generation, are presented, establishing a solid foundation for the research. Method: The research utilized 2,178 images of cross-sections of wood from 18 species, applying Deep Convolutional GAN (DCGAN) and U-Net with diffusion on a rare species, evaluated using FID and IS metrics. The generated images were used to train and validate classification models, assessed by F1-score. Results and Discussion: The results revealed that Diffusion Models generated more realistic images and performed better in the classification of the rare species. The discussion contextualizes these results in light of the theoretical framework, exploring their implications for environmental management. Limitations, such as the impact of indiscriminate addition of synthetic data, are also addressed. Research Implications: The practical and theoretical implications of this research are discussed, providing insights into how the results can be applied or influence practices in the field of forest management and environmental management. These implications may cover areas such as sustainable innovation and biodiversity conservation. Originality/Value: This study contributes to the literature by demonstrating the potential of Diffusion Models to outperform GANs in generating synthetic images for sustainable forest management purposes. The relevance and value of this research are evidenced by its practical application in promoting innovative and sustainable practices in environmental management.