Generative Adversarial Networks Research Articles

Enhancing X-ray Security Image Synthesis: Advanced Generative Models and Innovative Data Augmentation Techniques

This study addresses the field of X-ray security screening and focuses on synthesising realistic X-ray images using advanced generative models. Insufficient training data in this area pose a major challenge, which we address through innovative data augmentation techniques. We utilise the power of generative adversarial networks (GANs) and conditional GANs (cGANs), in particular the Pix2Pix and Pix2PixHD models, to investigate the generation of X-ray images from various inputs such as masks and edges. Our experiments conducted on a Korean dataset containing dangerous objects relevant to security screening show the effectiveness of these models in improving the quality and realism of image synthesis. Quantitative evaluations based on metrics such as PSNR, SSIM, LPIPS, FID, and FSIM, with scores of 19.93, 0.71, 0.12, 29.36, and 0.54, respectively, show the superiority of our strategy, especially when integrated with hybrid inputs containing both edges and masks. Overall, our results highlight the potential of advanced generative models to overcome the challenges of data scarcity in X-ray security screening and pave the way for more efficient and accurate inspection systems.

Investigating the causes of permeability anisotropy in heterogeneous conglomeratic sandstone using multiscale digital rock

Heterogeneous conglomeratic sandstone exhibits anisotropic physical properties, rendering a comprehensive analysis of its physical processes challenging with experimental measurements. Digital rock technology provides a visual and intuitive analysis of the microphysical processes in rocks, thereby aiding in scientific inquiry. Nevertheless, the multiscale characteristics of conglomeratic sandstone cannot be fully captured by a single-scale digital rock, thus limiting its ability to characterize the pore structure. Our work introduces a proposed workflow that uses multiscale digital rock fusion to investigate permeability anisotropy in heterogeneous rock. We use a cycle-consistent generative adversarial network to fuse computed tomography scan data of different resolutions, creating a large-scale and high-precision digital rock that comprehensively represents the conglomeratic sandstone pore structure. Subsequently, the digital rock is partitioned into multiple blocks, and the permeability of each block is simulated using a pore network. Finally, the total permeability of the sample is calculated by conducting an upscaling numerical simulation using the Darcy-Stokes equation. This process facilitates the analysis of the pore structure in conglomeratic sandstone and provides a step-by-step solution for permeability. From a multiscale perspective, this approach reveals that the anisotropy of permeability in conglomeratic sandstone stems from the layered distribution of grain sizes and differences in grain arrangement across different directions.

Machine learning methods for background potential estimation in 2DEGs

Two-dimensional electron gases (2DEGs) can show exceptional carrier mobility, making them promising candidates for future quantum technologies. However, impurities and defects can significantly degrade their performance, impacting transport, conductivity, and coherence times. We leverage scanning gate microscopy (SGM) and machine learning approaches to extract the potential landscape of 2DEGs from SGM data. We compare three techniques: image-to-image translation with generative adversarial networks (GANs), cellular neural networks (CNNs), and an evolutionary search algorithm. Notably, the evolutionary approach outperforms both alternatives in defect identification and analysis. This work clarifies the interaction between defects and 2DEG properties, demonstrating the potential of machine learning for understanding and manipulating quantum materials, facilitating advancements in quantum computing and nanoelectronics.

Continuous Glucose, Insulin and Lifestyle Data Augmentation in Artificial Pancreas Using Adaptive Generative and Discriminative Models.

Artificial pancreas requires data from multiple sources for accurate insulin dose estimation. These include data from continuous glucose sensors, past insulin dosage information, meal quantity and time and physical activity data. The effectiveness of closed-loop diabetes management systems might be hampered by the absence of these data caused by device error or lack of compliance by patients. In this study, we demonstrate the effect of output sequence length-driven generative and discriminative model selection in high quality data generation and augmentation. This novel generative adversarial network (GAN) based architecture automatically selects the generator and discriminator architecture based on the desired output sequence length. The proposed model is able to generate glucose, physical activity, meal information data for individual patients. The discriminative scores for Ohio T1DM (2018) dataset were 0.17 ±0.03 (Inputs: CGM, CHO, Insulin) and 0.15 ±0.02 (Inputs: CGM, CHO, Insulin, Heart Rate, Steps) and for Ohio T1D (2020) dataset was 0.16 ±0.02 (Inputs: CGM, CHO, Insulin) and 0.15 ±0.02 (Inputs: CGM, CHO, Insulin, acceleration). A mixture of generated and real data was used to test predictive scores for glucose forecasting models. The best RMSE and MARD achieved for OhioT1DM patients were 17.19 ±3.22 and 7.14 ±1.76 for PH=30min with CGM, CHO, Insulin, heartrate and steps as inputs. Similarly, the RMSE and MARD for real+synthetic data were 15.63 ±2.57 and 5.86 ±1.69 respectively. Compared to existing generative models, we demonstrate that sequence length based architecture selection leads to better synthetic data generation for multiple output sequences (CGM, CHO, Insulin) and forecasting accuracy.

Brain2GAN: Feature-disentangled neural encoding and decoding of visual perception in the primate brain.

A challenging goal of neural coding is to characterize the neural representations underlying visual perception. To this end, multi-unit activity (MUA) of macaque visual cortex was recorded in a passive fixation task upon presentation of faces and natural images. We analyzed the relationship between MUA and latent representations of state-of-the-art deep generative models, including the conventional and feature-disentangled representations of generative adversarial networks (GANs) (i.e., z- and w-latents of StyleGAN, respectively) and language-contrastive representations of latent diffusion networks (i.e., CLIP-latents of Stable Diffusion). A mass univariate neural encoding analysis of the latent representations showed that feature-disentangled w representations outperform both z and CLIP representations in explaining neural responses. Further, w-latent features were found to be positioned at the higher end of the complexity gradient which indicates that they capture visual information relevant to high-level neural activity. Subsequently, a multivariate neural decoding analysis of the feature-disentangled representations resulted in state-of-the-art spatiotemporal reconstructions of visual perception. Taken together, our results not only highlight the important role of feature-disentanglement in shaping high-level neural representations underlying visual perception but also serve as an important benchmark for the future of neural coding.

An improved pix2pix generative adversarial networks for sand-dust image enhancement

An improved pix2pix generative adversarial networks for sand-dust image enhancement

SURVEY ON STABLE DIFFUSION TEXT TO IMAGE USING AI

The Fusion Nexus Text-to-Image Synthesis Initiative integrates cutting-edge Generative Adversarial Networks (GANs) with Natural Language Processing (NLP) techniques to narrow the semantic divide between textual input and visual output. Built upon the robust Stable Diffusion training paradigm, this initiative is engineered to produce immersive, true-to-life images based on descriptive text prompts. While GANs have exhibited potential in image generation, issues like mode collapse and training instability have impeded their effectiveness. The Fusion Nexus initiative circumvents these challenges by leveraging the Stable Diffusion framework, which furnishes a stable and reliable training methodology for GANs. By amalgamating recent advancements in deep learning, this project spearheads a novel approach to text-to-image synthesis. Its primary aim is to craft cohesive and highly realistic visual representations from textual descriptions, thereby bridging the gap between linguistic expression and visual perception. This ambitious undertaking marks a significant stride at the convergence of GANs and NLP, presenting a promising solution to the intricate task of text-to-image generation. Keywords: Adversarial, Diffusion, Framework, Fusion, GANs, Generation, Generation, Image, Language, Natural, NLP, Processing, Robust, Stable, Text-to-image, Training, Visuals.

Detracking Autoencoding Conditional Generative Adversarial Network: Improved Generative Adversarial Network Method for Tabular Missing Value Imputation

Due to various reasons, such as limitations in data collection and interruptions in network transmission, gathered data often contain missing values. Existing state-of-the-art generative adversarial imputation methods face three main issues: limited applicability, neglect of latent categorical information that could reflect relationships among samples, and an inability to balance local and global information. We propose a novel generative adversarial model named DTAE-CGAN that incorporates detracking autoencoding and conditional labels to address these issues. This enhances the network’s ability to learn inter-sample correlations and makes full use of all data information in incomplete datasets, rather than learning random noise. We conducted experiments on six real datasets of varying sizes, comparing our method with four classic imputation baselines. The results demonstrate that our proposed model consistently exhibited superior imputation accuracy.

The Style Transformation of Gu-Zheng Music Based on the Improved Model of Cyclegan

In recent years, generative adversarial networks have excelled in the field of image style migration, however their performance in the field of music has been mediocre. Existing music style migration does not work well for style migration of gu-zheng music. In order to solve these problems, we first extract the features of gu-zheng music and the Mel-spectrum features, then use CycleGAN to do style transformation on the combined features and Mel-spectrum features, and then use WaveNet vocoder to decode the migrated spectrograms, and finally achieve the style migration with gu-zheng music. The proposed model was evaluated on the publicly available dataset FMA, and the average style migration rate of the compliant music reached 94.07%. Compared to other algorithms, the music produced by this method outperformed other algorithms in terms of style migration rate and audio quality.

GANSharp: High-definition image reconstruction using generative adversarial networks

In the realm of digital imaging, enhancing low-resolution images to high-definition quality is a pivotal challenge, particularly crucial for applications in medical imaging, security, and remote sensing. Traditional methods, primarily relying on basic interpolation techniques, often result in images that lack detail and fidelity. GANSharp introduces an innovative GAN-based framework that substantially improves the generator network, incorporating adversarial and perceptual loss functions for enhanced image reconstruction. The core issue addressed is the loss of critical information during down-sampling processes. To counteract this, we proposed a GAN-based method leveraging deep learning algorithms, trained using sets of both low- and high-resolution images. Our approach, which focuses on expanding the generator network’s size and depth and integrating adversarial and perceptual loss, was thoroughly evaluated on various benchmark datasets. The experimental results showed remarkable outcomes. On the Set5 dataset, our method achieved a PSNR of 34.18 dB and a SSIM of 0.956. Comparatively, on the Set14 dataset, it yielded a PSNR of 31.16 dB and an SSIM of 0.920, and on the B100 dataset, it achieved a PSNR of 30.51 dB and an SSIM of 0.912. These results were superior or comparable to those of existing advanced algorithms, demonstrating the proposed method’s potential in generating high-quality, high-resolution images. Our research underscores the potency of GANs in image super-resolution, making it a promising tool for applications spanning medical diagnostics, security systems, and remote sensing. Future exploration could extend to the utilization of alternative loss functions and novel training techniques, aiming to further refine the efficacy of GAN-based image restoration algorithms.

Transductive zero-shot learning with generative model-driven structure alignment

Zero-shot learning (ZSL) facilitates the transfer of knowledge from seen to unseen categories through high-dimensional vectors that capture both known and unknown class names. However it encounters challenges with domain shift arising from a lack of sufficient labeled data. Although transductive zero-shot learning (TZSL) addresses this bias by including samples from unseen classes, it still faces obstacles in enhancing TZSL performance. In this study, We introduce the Structure Alignment Variational Autoencoder Generative Adversarial Network (SA-VAEGAN), a novel approach that enhances the alignment between visual and auxiliary spaces. We delved into the underlying causes of domain shift and introduced a structural alignment (SA) strategy to tackle these challenges. The SA model thoroughly accounts for both inter-class and intra-class dynamics, designed to leverage the model’s comprehension of high-level semantic relations to disambiguate confusion among similar classes and mitigate intra-class confusion by penalizing atypical visual samples within classes. Assessed across four benchmark datasets, SA-VAEGAN has established a new performance standard, underscoring its efficiency in addressing the domain shift challenge within TZSL tasks, and achieving high accuracy.

Similarity indicator and CG-CGAN prediction model for remaining useful life of rolling bearings

To tackle the challenges of performing early fault warning and improving the prediction accuracy for the remaining useful life (RUL) of rolling bearings, this paper proposes a similarity health indicator and a predictive model of CG-conditional generative adversarial network (CGAN), which relies on a CGAN that combines one-dimensional convolutional neural network (CNN) with a bidirectional gate recurrent unit (Bi-GRU). This framework provides a comprehensive theoretical foundation for RUL prediction of rolling bearings. The similarity health indicator allows for early fault warning of rolling bearings without expert knowledge. Within the CGAN framework, the inclusion of constraints guides the generation of samples in a more targeted manner. Additionally, the proposed CG-CGAN model incorporates Bi-GRU to consider both forward and backward information, thus improving the precision of RUL forecasting. Firstly, the similarity indicator between the vibration signals of the rolling bearing over its full life span and the standard vibration signals (healthy status) is calculated. This indicator helps to determine the early deterioration points of the rolling bearings. Secondly, the feature matrix composed of traditional health indicators and similarity health indicator, is utilized to train and test the proposed CG-CGAN model for RUL prediction. Finally, to corroborate the efficacy of the proposed method, two sets of real experiment data of rolling bearing accelerated life from the Intelligent Maintenance Systems (IMS) are utilized. Experimental findings substantiate that the proposed similarity health indicator offers early fault alerts and precisely delineates the performance diminution of the rolling bearing. Furthermore, the put-forward CG-CGAN model achieves high-precision RUL prediction of rolling bearing.

Development of Process-Structure Linkage Using Conditional Generative Adversarial Networks

Development of Process-Structure Linkage Using Conditional Generative Adversarial Networks

Creating synthetic energy meter data using conditional diffusion and building metadata

Advances in machine learning and increased computational power have driven progress in energy-related research. However, limited access to private energy data from buildings hinders traditional regression models relying on historical data. While generative models offer a solution, previous studies have primarily focused on short-term generation periods (e.g., daily profiles) and a limited number of meters. Thus, the study proposes a conditional diffusion model for generating high-quality synthetic energy data using relevant metadata. Using a dataset comprising 1,828 power meters from various buildings and countries, this model is compared with traditional methods like Conditional Generative Adversarial Networks (CGAN) and Conditional Variational Auto-Encoders (CVAE). It explicitly handles long-term annual consumption profiles, harnessing metadata such as location, weather, building, and meter type to produce coherent synthetic data that closely resembles real-world energy consumption patterns. The results demonstrate the proposed diffusion model's superior performance, with a 36% reduction in Fréchet Inception Distance (FID) score and a 13% decrease in Kullback-Leibler divergence (KL divergence) compared to the following best method. The proposed method successfully generates high-quality energy data through metadata, and its code will be open-sourced, establishing a foundation for a broader array of energy data generation models in the future.

Atomic force microscopy simulations for CO-functionalized tips with deep learning

Atomic force microscopy (AFM) operating in the frequency modulation mode with a metal tip functionalized with a CO molecule is able to image the internal structure of molecules with an unprecedented resolution. The interpretation of these images is often difficult, making the support of theoretical simulations important. Current simulation methods, particularly the most accurate ones, require expertise and resources to perform ab initio calculations for the necessary inputs (i.e charge density and electrostatic potential of the molecule). Here, we propose a computationally inexpensive and fast alternative to the physical simulation of these AFM images based on a conditional generative adversarial network (CGAN), that avoids all force calculations, and uses as the only input a 2D ball–and–stick depiction of the molecule. We discuss the performance of the model when trained with different subsets extracted from the previously published QUAM-AFM database. Our CGAN reproduces accurately the intramolecular contrast observed in the simulated images for quasi–planar molecules, but has limitations for molecules with a substantial internal corrugation, due to the strictly 2D character of the input.

A time-saving fault diagnosis using simplified fast GAN and triple-type data transfer learning

Existing intelligent fault diagnosis approaches demand substantial data for training diagnostic models. However, factors such as the inherent characteristics of bearings, operating conditions, and privacy security make collecting comprehensive fault-bearing data very difficult. Although generating synthetic data through generative adversarial networks (GANs) is feasible, the data generation of GANs is a time-consuming process. To address these challenges, a fault diagnosis framework based on GAN and deep transfer learning (DTL) is proposed, termed the simplified fast GAN triple-type data transfer learning (SFGAN-TDTL) method. Initially, an SFGAN is proposed as a replacement for traditional GANs. The time-frequency image data generated by SFGAN serve to augment the training dataset, offering faster and higher-quality data generation compared to traditional GANs. To further reduce time consumption for GAN-based methods, the TDTL method is proposed. Differing from DTL, which utilizes synthetic data to construct a pre-trained model and conducts targeted fine-tuning with real data, TDTL employs open-source data, synthetic data, and real data to fill the weights of the task-insensitive layer, task-sensitive layer, and fully connected layer, respectively. Numerical results demonstrate that SFGAN-TDTL maintains higher diagnostic accuracy while significantly reducing time consumption.

Creating realistic anterior segment optical coherence tomography images using generative adversarial networks

AimsTo develop a generative adversarial network (GAN) capable of generating realistic high-resolution anterior segment optical coherence tomography (AS-OCT) images.MethodsThis study included 142 628 AS-OCT B-scans from the American University of...

Multicenter PET image harmonization using generative adversarial networks.

To improve reproducibility and predictive performance of PET radiomic features in multicentric studies by cycle-consistent generative adversarial network (GAN) harmonization approaches. GAN-harmonization was developed to harmonize whole-body PET scans to perform image style and texture translation between different centers and scanners. GAN-harmonization was evaluated by application to two retrospectively collected open datasets and different tasks. First, GAN-harmonization was performed on a dual-center lung cancer cohort (127 female, 138 male) where the reproducibility of radiomic features in healthy liver tissue was evaluated. Second, GAN-harmonization was applied to a head and neck cancer cohort (43 female, 154 male) acquired from three centers. Here, the clinical impact of GAN-harmonization was analyzed by predicting the development of distant metastases using a logistic regression model incorporating first-order statistics and texture features from baseline 18F-FDG PET before and after harmonization. Image quality remained high (structural similarity: left kidney 0.800, right kidney 0.806, liver 0.780, lung 0.838, spleen 0.793, whole-body 0.832) after image harmonization across all utilized datasets. Using GAN-harmonization, inter-site reproducibility of radiomic features in healthy liver tissue increased at least by 5 ± 14% (first-order), 16 ± 7% (GLCM), 19 ± 5% (GLRLM), 16 ± 8% (GLSZM), 17 ± 6% (GLDM), and 23 ± 14% (NGTDM). In the head and neck cancer cohort, the outcome prediction improved from AUC 0.68 (95% CI 0.66-0.71) to AUC 0.73 (0.71-0.75) by application of GAN-harmonization. GANs are capable of performing image harmonization and increase reproducibility and predictive performance of radiomic features derived from different centers and scanners.

Deep learning-based prediction of thermal residual stress and melt pool characteristics in laser-irradiated carbon steel

Residual stress is a crucial factor that must be managed in laser manufacturing processes. Neglecting it can result in product failure or unsuitability for the intended use. In this article, we present the first surrogate deep learning model based on generative adversarial network that predicts thermal residual stress and melt pool characteristics on the cross-section of laser-irradiated carbon steel. The model was trained using simulation data obtained from the unified momentum equation approach-based numerical model. Our model was designed to predict various aspects of the laser melting process only from three scalar inputs of laser power, beam diameter, and irradiation time at two different points in time (immediately after the laser heating is finished and when the steel is completely cooled down) and successfully predicted the distributions of 12 variables, including residual stress and temperature. Prediction results were not only qualitatively very similar to their ground truth, but also quantitatively accurate with the R2 values ranging from 0.975 to 0.999 and the mean absolute errors between 0.0015 and 0.0126. We demonstrated that complex experiments can be quickly performed using the surrogate model because a single deep learning prediction took only 0.13 s, compared to the 2.5–8 h numerical simulation times.

Scalable Integration of Multiomic Single Cell Data Using Generative Adversarial Networks.

Single cell profiling has become a common practice to investigate the complexity of tissues, organs and organisms. Recent technological advances are expanding our capabilities to profile various molecular layers beyond the transcriptome such as, but not limited to, the genome, the epigenome and the proteome. Depending on the experimental procedure, these data can be obtained from separate assays or from the very same cells. Despite development of computational methods for data integration is an active research field, most of the available strategies have been devised for the joint analysis of two modalities and cannot accommodate a high number of them. We here propose a multiomic data integration framework based on Wasserstein Generative Adversarial Networks (MOWGAN) suitable for the analysis of paired or unpaired data with high number of modalities (>2). At the core of our strategy is a single network trained on all modalities together, limiting the computational burden when many molecular layers are evaluated. Source code of our framework is available at https://github.com/vgiansanti/MOWGAN. Supplementary data are available at Bioinformatics online.