Generative Adversarial Networks Research Articles

Melanoma image synthesis: a review using generative adversarial networks

Melanoma is a highly malignant skin cancer that may be fatal if not promptly detected and treated. The limited availability of high-quality melanoma images, which are needed for training machine learning models, is one of the obstacles to detecting melanoma. Generative adversarial networks (GANs) have grown in popularity as a strong technique for image synthesis. This research is also targeted at the sustainable development goal (SDG) for health care. In this study, we survey existing GAN-based melanoma image synthesis methods. In this work, we briefly introduce GANs and how they may be used for generating synthetic images. Ensuring healthy lifestyles and promoting well-being for everyone, regardless of age, is the main aim. A comparative study is carried out on how GANs are used in current research to generate melanoma images and how they improve the classification performance of neural networks. Various public and proprietary datasets for training GANs in melanoma image synthesis are also discussed. Lastly, we assess the examined studies' performance using measures like the Frechet Inception distance (FID), Inception score, structural similarity ındex (SSIM), and various classification performance metrics. We compare the evaluated findings and suggest further GAN-based melanoma image-creation research.

Enhanced diabetic retinopathy detection and classification using fundus images with ResNet50 and CLAHE-GAN

Diabetic retinopathy (DR), a progressive eye disorder, can lead to irreversible vision impairment ranging from no DR to severe DR, necessitating precise identification for early treatment. This study introduces an innovative deep learning (DL) approach, surpassing traditional methods in detecting DR stages. It evaluated two scenarios for training DL models on balanced datasets. The first employed image enhancement via contrast limited adaptive histogram equalization (CLAHE) and a generative adversarial network (GAN), while the second did not involve any image enhancement. Tested on the Asia pacific tele-ophthalmology society 2019 blindness detection (APTOS-2019 BD) dataset, the enhanced model (scenario 1) reached 98% accuracy and a 99% Cohen kappa score (CKS), with the non-enhanced model (scenario 2) achieving 95.4% accuracy and a 90.5% CKS. The combination of CLAHE and GAN, termed CLANG, significantly boosted the model's performance and generalizability. This advancement is pivotal for early DR detection and intervention, offering a new pathway to prevent irreversible vision loss in diabetic patients.

Learning With Constraint Learning: New Perspective, Solution Strategy and Various Applications.

The complexity of learning problems, such as Generative Adversarial Network (GAN) and its variants, multi-task and meta-learning, hyper-parameter learning, and a variety of real-world vision applications, demands a deeper understanding of their underlying coupling mechanisms. Existing approaches often address these problems in isolation, lacking a unified perspective that can reveal commonalities and enable effective solutions. Therefore, in this work, we proposed a new framework, named Learning with Constraint Learning (LwCL), that can holistically examine challenges and provide a unified methodology to tackle all the above-mentioned complex learning and vision problems. Specifically, LwCL is designed as a general hierarchical optimization model that captures the essence of these diverse learning and vision problems. Furthermore, we develop a gradient-response based fast solution strategy to overcome optimization challenges of the LwCL framework. Our proposed framework efficiently addresses a wide range of applications in learning and vision, encompassing three categories and nine different problem types. Extensive experiments on synthetic tasks and real-world applications verify the effectiveness of our approach. The LwCL framework offers a comprehensive solution for tackling complex machine learning and computer vision problems, bridging the gap between theory and practice.

Advancing medical imaging with GAN-based anomaly detection

Anomaly detection in medical imaging is a complex challenge, exacerbated by limited annotated data. Recent advancements in generative adversarial networks (GANs) offer potential solutions, yet their effectiveness in medical imaging remains largely uncharted. We conducted a targeted exploration of the benefits and constraints associated with GAN-based anomaly detection techniques. Our investigations encompassed experiments employing eight anomaly detection methods on three medical imaging datasets representing diverse modalities and organ/tissue types. These experiments yielded notably diverse results. The results exhibited significant variability, with metrics spanning a wide range (area under the curve (AUC): 0.475-0.991; sensitivity: 0.17-0.98; specificity: 0.14-0.97). Furthermore, we offer guidance for implementing anomaly detection models in medical imaging and anticipate pivotal avenues for future research. Results unveil varying performances, influenced by factors like dataset size, anomaly subtlety, and dispersion. Our findings provide insights into the complex landscape of anomaly detection in medical imaging, offering recommendations for future research and deployment.

Reliability analysis of GAN based transmit modules for active array antenna of phased array radar

Reliability is one of the most important requirements in our day to day life considering consistency, availability and failure free performance of the product over it’s define mission time. As complexity of the system increases, design for reliable systems is a big challenge. The objective of the reliability prediction analysis is to evaluate the predicted reliability of the active transmit receive modules (TRMs) under specified operating conditions, and to demonstrate that the predicted reliability meets the requirements, also to identify any parts present in the design which leads to higher failure rates. The research shows reliability of generative adversarial network (GAN) based TRMs covering from design to finalization of components as early as practicable in today's short product lifecycles. Using the reliability prediction process, we describe a method for providing design engineers with reliability feedback on their decisions. Using a conventional reliability prediction model, the Telcordia (Bellcore) parts stress prediction model, and some standard rules of thumb, we describe an initial implementation of this technique. It provides systematic identification of likely modes of failure, possible effects of each failure, and the criticality of each failure with regard to reliability, system readiness, mission success, and demand for maintenance/logistic support.

A Multi-Split Cross-Strategy for Enhancing Machine Learning Algorithms Prediction Results with Data Generated by Conditional Generative Adversarial Network

A Multi-Split Cross-Strategy for Enhancing Machine Learning Algorithms Prediction Results with Data Generated by Conditional Generative Adversarial Network

Image Super-Resolution Reconstruction Method based on Improved Cyclic Generative Adversarial Network with Twin Attention Mechanism

Image Super-Resolution Reconstruction Method based on Improved Cyclic Generative Adversarial Network with Twin Attention Mechanism

An implicit transformer-based fusion method for hyperspectral and multispectral remote sensing image

There is an effective way to enhance the spatial resolution of hyperspectral remote sensing images by fusing them with multispectral remote sensing images. However, most of the existing deep fusion techniques adopt discretized explicit models to approximate the complex continuous nonlinear mapping in the fusion process, leading to limitations in enhancing the fidelity of spatial details. Additionally, existing algorithms commonly utilize discrete methods such as bilinear or bicubic interpolation during the hyperspectral upsampling process, leading to the loss of crucial spatial-spectral features. To this end, this study proposes a novel Implicit Transformer Fusion Generative Adversarial Network (ITF-GAN), which incorporates the continuity perception mechanism of implicit neural representation with the powerful self-attention mechanism of the Transformer architecture, which uses point-to-point implicit functions aiming to efficiently process information in both spatial and spectral dimensions. Besides, a guided implicit neural sampling module is introduced in the hyperspectral image up-sampling process to enhance the coordinated expression of features in the spatial and spectral domains, which improves the spatial resolution and spectral fidelity of the fused image during the upsampling process. A series of fusion experiments including 4x, 8x, and 16x scale factors have shown that ITF-GAN has significant advantages over current popular fusion algorithms in both objective evaluation indicators and subjective visual evaluation.

Black and White Image Colorization with Deep Learning

Abstract: The project "Black and White Image Colorization with Deep Learning" develops a system to automatically infuse colors into grayscale images through cutting-edge deep learning approaches. Utilizing convolutional neural networks (CNNs) and generative adversarial networks (GANs), the system aims to deliver both realistic and visually pleasing colorization outcomes. This process includes training the deep learning models on diverse image datasets, optimizing the model parameters, and using advanced loss functions to ensure precise color representation.This project addresses the complex challenges associated with adding colors to monochrome images and also seeks to improve user experience by introducing customizable options such as style modification and color tweaks. By merging deep learning with computer vision techniques, this initiative is poised to contribute significantly to various image processing fields, especially in the restoration of historical images and the enhancement of visual content production. With thorough experimentation and ongoing refinements, the project aspires to establish a reliable and easy-to-use system that advances the state of image colorization technology in the modern digital age..

GAN-Generated Face Detection Based on Multiple Attention Mechanism and Relational Embedding

The rapid development of the Generative Adversarial Network (GAN) makes generated face images more and more visually indistinguishable, and the detection performance of previous methods will degrade seriously when the testing samples are out-of-sample datasets or have been post-processed. To address the above problems, we propose a new relational embedding network based on “what to observe” and “where to attend” from a relational perspective for the task of generated face detection. In addition, we designed two attention modules to effectively utilize global and local features. Specifically, the dual-self attention module selectively enhances the representation of local features through both image space and channel dimensions. The cross-correlation attention module computes similarity between images to capture the global information of the output in the image. We conducted extensive experiments to validate our method, and the proposed algorithm can effectively extract the correlations between features and achieve satisfactory generalization and robustness in generating face detection. In addition, we also explored the design of the model structure and the inspection performance on more categories of generated images (not limited to faces). The results show that RENet also has good detection performance on datasets other than faces.

Stellar spectral template library construction based on generative adversarial networks

Stellar spectral template libraries play an important role in the automated analysis of stellar spectra. Synthetic template libraries cover a very large parameter space but suffer from poor matching with observed spectra. In this study, we propose a synthetic-to-observed spectral translation (SOST) method based on generative adversarial networks. The SOST method is able to calibrate synthetic spectra by converting them to the corresponding observed spectra. We applied this method to Kurucz synthetic spectra and observed spectra data from the Large Sky Area Multi-Object Fiber Spectroscopic Telescope (LAMOST). After that, we constructed a stellar spectral library with uniform and broad parameter distributions using the SOST-corrected Kurucz synthetic spectra. Our stellar spectral template library contains 2431 spectra spanning a parameter space of 3500–8000 K for effective temperature (Teff), 0.0–5.0 dex for surface gravity (log g), and −2.0–0.5 dex for metallicity ([Fe/H]). The spectra in the library have a resolution of R ∼ 1800 and cover the wavelength range 3900–8700 Å. In order to verify the accuracy of this template library, we used the template library and the template-matching algorithm to derive the parameters of the PASTEL database. Compared to measurements using the original synthetic template library, the accuracies of the three parameters, Teff, log g, and [Fe/H], are improved, from 140 K, 0.31 dex, and 0.21 dex to 121 K, 0.26 dex, and 0.13 dex, respectively. In addition, we re-parameterised more than six million stellar spectra released by LAMOST DR8.

Nighttime road scene image enhancement based on cycle-consistent generative adversarial network.

During nighttime road scenes, images are often affected by contrast distortion, loss of detailed information, and a significant amount of noise. These factors can negatively impact the accuracy of segmentation and object detection in nighttime road scenes. A cycle-consistent generative adversarial network has been proposed to address this issue to improve the quality of nighttime road scene images. The network includes two generative networks with identical structures and two adversarial networks with identical structures. The generative network comprises an encoder network and a corresponding decoder network. A context feature extraction module is designed as the foundational element of the encoder-decoder network to capture more contextual semantic information with different receptive fields. A receptive field residual module is also designed to increase the receptive field in the encoder network.The illumination attention module is inserted between the encoder and decoder to transfer critical features extracted by the encoder to the decoder. The network also includes a multiscale discriminative network to discriminate better whether the image is a real high-quality or generated image. Additionally, an improved loss function is proposed to enhance the efficacy of image enhancement. Compared to state-of-the-art methods, the proposed approach achieves the highest performance in enhancing nighttime images, making them clearer and more natural.

Voxel-Wise Medical Image Generalization for Eliminating Distribution Shift

Currently, the medical field is witnessing an increase in the use of machine learning techniques. Supervised learning methods adopted in classification, prediction, and segmentation tasks for medical images always experience decreased performance when the training and testing datasets do not follow the independent and identically distributed assumption. These distribution shift situations seriously influence machine learning applications’ robustness, fairness, and trustworthiness in the medical domain. Hence, in this article, we adopt the CycleGAN (generative adversarial network) method to cycle train the computed tomography data from different scanners/manufacturers. It aims to eliminate the distribution shift from diverse data terminals based on our previous work [ 14 ]. However, due to the model collapse problem and generative mechanisms of the GAN-based model, the images we generated contained serious artifacts. To remove the boundary marks and artifacts, we adopt score-based diffusion generative models to refine the images voxel-wisely. This innovative combination of two generative models enhances the quality of data providers while maintaining significant features. Meanwhile, we use five paired patients’ medical images to deal with the evaluation experiments with structural similarity index measure metrics and the segmentation model’s performance comparison. We conclude that CycleGAN can be utilized as an efficient data augmentation technique rather than a distribution-shift-eliminating method. In contrast, the denoising diffusion the denoising diffusion model is more suitable for dealing with the distribution shift problem aroused by the different terminal modules. The limitation of generative methods applied in medical images is the difficulty in obtaining large and diverse datasets that accurately capture the complexity of biological structure and variability. In our following research, we plan to assess the initial and generated datasets to explore more possibilities to overcome the above limitation. We will also incorporate the generative methods into the federated learning architecture, which can maintain their advantages and resolve the distribution shift issue on a larger scale.

Brushstrokes of Tomorrow: Exploring the Art of AI

In recent years, the advancement of Artificial Intelligence (AI) technology, particularly in deep learning algorithms like Generative Adversarial Networks (GANs) and Variational Autoencoders (VAE), has led to significant developments in AI-based art generation across various sectors within the art industry. The year 2022 witnessed an explosion of AI-generated art, particularly in creative design, resulting in the production of numerous outstanding works that have enhanced the efficiency of art design processes. This study delves into the application and design characteristics of AI generation technology within two specific sub-fields: AI painting and AI animation production. A comparative analysis between traditional painting methods and AI-generated painting techniques is conducted to discern differences. Through this research, the paper synthesizes the advantages and challenges inherent in the AI creative design process. Despite technical limitations and issues such as copyright and income distribution, AI art designs demonstrate promise in facilitating artistic innovation and technological integration within the art domain. Their potential for advancing sub divisional artistic practices and their intersection with technology renders them highly valuable subjects for further research and exploration.

Enhanced image steganalysis through reinforcement learning and generative adversarial networks

With the surging prominence of digital communication platforms, there has been an intensified emphasis on ensuring robust security and privacy measures. Against this backdrop, image steganalysis has emerged as a critical discipline, employing advanced methods to detect clandestine data within image files. At the core of our research is an innovative exploration into image steganalysis using an amalgamation of enhanced reinforcement learning techniques and online data augmentation. This methodology ensures the meticulous identification of concealed data within images. Our design integrates triple parallel dilated convolutions, enabling concurrent extraction of feature vectors from the input images. Once extracted, these vectors are synthesized, paving the way for subsequent classification tasks. To substantiate the efficacy of our approach, we conducted tests on a comprehensive dataset sourced from BossBase 1.01. Furthermore, to discern the influence of transfer learning on our proposed model, the BOWS dataset was employed. Notably, these datasets present a challenge due to its inherent imbalance. To counteract this, we incorporated an advanced Reinforcement Learning (RL) framework. Herein, the dataset samples are envisioned as states in a sequence of interrelated decisions, with the neural network playing the role of the decision-making agent. This agent is then incentivized or reprimanded based on its accuracy in discerning between the minority and majority classes. To bolster our classification capabilities, we innovatively employed data augmentation using images generated by a Generative Adversarial Network (GAN). Concurrently, a regularization mechanism was instituted to alleviate prevalent GAN-related challenges, such as mode collapse and unstable training dynamics. Our experimental outcomes underscore the potency of our methodology. The results highlight a remarkable capability to discern between pristine and steganographic images, registering an average accuracy rate of 85%.

Temper wolf hunt optimization enabled GAN for robust image encryption

In today’s digital era, the security of sensitive data, particularly in the realm of multimedia, is of paramount importance. Image encryption serves as a vital shield against unauthorized access and ensures the confidentiality and integrity of visual information. As such, the continuous pursuit of robust and efficient encryption techniques remains a pressing concern. This research introduces a Temper Wolf Hunt Optimization enabled Generative Adversarial Network Encryption model (TWHO-GAN), designed to address the challenges of image encryption in the modern digital landscape. TWHO, inspired by the collective hunting behavior of wolf and coyote packs, is employed to generate highly secure encryption keys. This algorithm excels in exploring complex solution spaces, creating robust, attack-resistant keys. In TWHO-GAN model, GANs are employed to create encrypted images that are virtually indistinguishable from their original counterparts, adding a layer of security by generating complex encryption keys and ensuring robust protection against attacks. The GAN component reconstructs the encrypted images to their original form when decrypted with the correct keys, ensuring data integrity while maintaining confidentiality. Further, the significance of the proposed model relies on the TWHO algorithm formulated by the integration of the adaptability and coordinated hunting strategies to optimize the chaotic map generation in image encryption protecting the sensitive visual information from unauthorized access as well as potential threats. Through extensive experimentation and comparative analysis, TWHO-GAN demonstrates superior performance in image encryption, surpassing former methods in terms of Cs, 𝐻𝑖𝑠C, MSE, PSNR, RMSE, and SSIM attaining values of 0.93, 94.19, 3.274, 59.70 dB, 1.8095, and 0.940 respectively for 5 numbers of images. Moreover, the TWHO-GAN approach attained the values of 0.91,92.22, 2.03, 49.74 dB, 1.42, and 0.88 for Cs, HisC, MSE, PSNR, RMSE, and SSIM respectively utilizing the Airplanes dataset. The model exhibits robust resistance to various attacks, making it a compelling choice for secure image transmission and storage.

Deep generative denoising networks enhance quality and accuracy of gated cardiac PET data.

Cardiac positron emission tomography (PET) can visualize and quantify the molecular and physiological pathways of cardiac function. However, cardiac and respiratory motion can introduce blurring that reduces PET image quality and quantitative accuracy. Dual cardiac- and respiratory-gated PET reconstruction can mitigate motion artifacts but increases noise as only a subset of data are used for each time frame of the cardiac cycle. The objective of this study is to create a zero-shot image denoising framework using a conditional generative adversarial networks (cGANs) for improving image quality and quantitative accuracy in non-gated and dual-gated cardiac PET images. Our study included retrospective list-mode data from 40 patients who underwent an 18F-fluorodeoxyglucose (18F-FDG) cardiac PET study. We initially trained and evaluated a 3D cGAN-known as Pix2Pix-on simulated non-gated low-count PET data paired with corresponding full-count target data, and then deployed the model on an unseen test set acquired on the same PET/CT system including both non-gated and dual-gated PET data. Quantitative analysis demonstrated that the 3D Pix2Pix network architecture achieved significantly (p value<0.05) enhanced image quality and accuracy in both non-gated and gated cardiac PET images. At 5%, 10%, and 15% preserved count statistics, the model increased peak signal-to-noise ratio (PSNR) by 33.7%, 21.2%, and 15.5%, structural similarity index (SSIM) by 7.1%, 3.3%, and 2.2%, and reduced mean absolute error (MAE) by 61.4%, 54.3%, and 49.7%, respectively. When tested on dual-gated PET data, the model consistently reduced noise, irrespective of cardiac/respiratory motion phases, while maintaining image resolution and accuracy. Significant improvements were observed across all gates, including a 34.7% increase in PSNR, a 7.8% improvement in SSIM, and a 60.3% reduction in MAE. The findings of this study indicate that dual-gated cardiac PET images, which often have post-reconstruction artifacts potentially affecting diagnostic performance, can be effectively improved using a generative pre-trained denoising network.

Detection of Train Wheelset Tread Defects with Small Samples Based on Local Inference Constraint Network

Due to the long-term service through wheel-rail rolling contact, the train wheelset tread will inevitably suffer from different types of defects, such as wear, cracks, and scratches. The effective detection of wheelset tread defects can provide critical support for the operation and maintenance of trains. In this paper, a new method based on a local inference constraint network is proposed to detect wheelset tread defects, and the main purpose is to address the issue of insufficient feature spaces caused by small samples. First, a generative adversarial network is applied to generate diverse samples with semantic consistency. An attention mechanism module is introduced into the feature extraction network to increase the importance of defect features. Then, the residual spine network for local input decisions is constructed to establish an association between sample features and defect types. Furthermore, the network’s activation function is improved to obtain higher learning speed and accuracy with fewer parameters. Finally, the validity and feasibility of the proposed method are verified using experimental data.

Enhancing electrical resistance tomography reconstruction from conductive fabric: a comparative study of CycleGAN and pix2pixGAN

Abstract Monitoring fetal health is crucial in prenatal care, and existing techniques for assessing fetal movements are often expensive and limited to clinical environments. This research investigates the potential of utilizing electrical resistance tomography (ERT) with a conductive fabric to create a cost-effective and non-invasive imaging solution for fetal monitoring. The fusion of ERT with wearable e-textile devices facilitates continuous and portable monitoring. To improve the quality of ERT-generated images, we propose the application of CycleGAN and pix2pixGAN, both machine learning models based on generative adversarial networks. These models learn to map reconstructed images to target images, thereby enhancing reconstruction precision and image quality. The outcomes of this research highlight the effectiveness of the suggested method in managing noisy data and achieving superior image generation. This work presents a promising approach to fetal monitoring using ERT and deep learning techniques, opening possibilities for more affordable and accessible prenatal care.

Digital ray: enhancing cataractous fundus images using style transfer generative adversarial networks to improve retinopathy detection

Background/aimsThe aim of this study was to develop and evaluate digital ray, based on preoperative and postoperative image pairs using style transfer generative adversarial networks (GANs), to enhance cataractous fundus...