Cycle-consistent Generative Adversarial Networks Research Articles

Breaking the Haze: Enhanced CycleGAN Models with Self-Attention for Single-Image Dehazing

A common preprocessing step in vision-based applications is image dehazing, which has drawbacks such color distortion and detail loss. Conventional techniques frequently fall short of sufficiently retrieving sharp images from foggy situations. In order to improve single-image dehazing, this paper presents a unique method that combines Self-Attention Generative Adversarial Networks (SAGANs) with Cycle-Consistent Generative Adversarial Networks (CycleGAN). In order to capture long-range dependencies and enhance the perceptual quality of the dehazed images, our method includes self-attention processes from SAGANs and makes use of CycleGAN's strengths for efficient image-to-image translation without the need for paired training samples. We present a dual-framework design in which SAGANs improve textural and contextual details by focusing on relevant characteristics throughout the whole image space, while CycleGAN assures content fidelity through cycle consistency. We show through our studies that this integrated strategy achieves higher scores in common metrics like Peak Signal-to-Noise Ratio (PSNR) of 34.146 dB and Structural Similarity Index (SSIM) of 0.964, greatly outperforming both qualitative and quantitative assessments of previous methods. Together with establishing a new standard for single-image dehazing, this work shows how effective hybrid generative models can be when handling challenging image processing problems.

Unpaired data training enables super-resolution confocal microscopy from low-resolution acquisitions.

Supervised deep-learning models have enabled super-resolution imaging in several microscopic imaging modalities, increasing the spatial lateral bandwidth of the original input images beyond the diffraction limit. Despite their success, their practical application poses several challenges in terms of the amount of training data and its quality, requiring the experimental acquisition of large, paired databases to generate an accurate generalized model whose performance remains invariant to unseen data. Cycle-consistent generative adversarial networks (cycleGANs) are unsupervised models for image-to-image translation tasks that are trained on unpaired datasets. This paper introduces a cycleGAN framework specifically designed to increase the lateral resolution limit in confocal microscopy by training a cycleGAN model using low- and high-resolution unpaired confocal images of human glioblastoma cells. Training and testing performances of the cycleGAN model have been assessed by measuring specific metrics such as background standard deviation, peak-to-noise ratio, and a customized frequency content measure. Our cycleGAN model has been evaluated in terms of image fidelity and resolution improvement using a paired dataset, showing superior performance than other reported methods. This work highlights the efficacy and promise of cycleGAN models in tackling super-resolution microscopic imaging without paired training, paving the path for turning home-built low-resolution microscopic systems into low-cost super-resolution instruments by means of unsupervised deep learning.

Precise Size Determination of Supported Catalyst Nanoparticles via Generative AI and Scanning Transmission Electron Microscopy.

Transmission electron microscopy (TEM) plays a crucial role in heterogeneous catalysis for assessing the size distribution of supported metal nanoparticles. Typically, nanoparticle size is quantified by measuring the diameter under the assumption of spherical geometry, a simplification that limits the precision needed for advancing synthesis-structure-performance relationships. Currently, there is a lack of techniques that can reliably extract more meaningful information from atomically resolved TEM images, like nuclearity or geometry. Here, cycle-consistent generative adversarial networks (CycleGANs) are explored to bridge experimental and simulated images, directly linking experimental observations with information from their underlying atomic structure. Using the versatile Pt/CeO2 (Pt particles centered ≈2nm) catalyst synthesized by impregnation, large datasets of experimental scanning transmission electron micrographs and physical image simulations are created to train a CycleGAN. A subsequent size-estimation network is developed to determine the nuclearity of imaged nanoparticles, providing plausible estimates for ≈70% of experimentally observed particles. This automatic approach enables precise size determination of supported nanoparticle-based catalysts overcoming crystal orientation limitations of conventional techniques, promising high accuracy with sufficient training data. Tools like this are envisioned to be of great use in designing and characterizing catalytic materials with improved atomic precision.

Cross-domain fault diagnosis for multimode green ammonia synthesis process based on DA-CycleGAN

Green ammonia is a crucial strategy for reducing carbon emissions and promoting sustainable development. However, in industrial applications, the production load of the ammonia synthesis section must be adjusted to accommodate fluctuations in renewable energy generation and hydrogen production. Consequently, the green ammonia synthesis process operates under multiple conditions with varying production loads. The operations of this multimode process introduce new challenges for process safety. Traditional fault diagnosis methods experience significant performance degradation when production conditions change. In new conditions, only a small number of normal samples can be obtained, and no fault samples are available. To address this issue, a novel transfer learning method named DA-CycleGAN is proposed for the multimode green ammonia synthesis process. This method combines a two-dimensional generation model based on CycleGAN (Cycle-Consistent Generative Adversarial Networks) with domain adaptation to enhance model performance in cross-domain tasks. The feasibility of the proposed method was initially validated using the benchmark Tennessee-Eastman process for fault diagnosis. Subsequently, a case study of the green ammonia synthesis process demonstrated that it significantly enhances performance in multimode processes, ensuring process safety and reducing losses for industrial applications.

Weakly supervised low-dose computed tomography denoising based on generative adversarial networks.

Low-dose computed tomography (LDCT) is a diagnostic imaging technique designed to minimize radiation exposure to the patient. However, this reduction in radiation may compromise computed tomography (CT) image quality, adversely impacting clinical diagnoses. Various advanced LDCT methods have emerged to mitigate this challenge, relying on well-matched LDCT and normal-dose CT (NDCT) image pairs for training. Nevertheless, these methods often face difficulties in distinguishing image details from nonuniformly distributed noise, limiting their denoising efficacy. Additionally, acquiring suitably paired datasets in the medical domain poses challenges, further constraining their applicability. Hence, the objective of this study was to develop an innovative denoising framework for LDCT images employing unpaired data. In this paper, we propose a LDCT denoising network (DNCNN) that alleviates the need for aligning LDCT and NDCT images. Our approach employs generative adversarial networks (GANs) to learn and model the noise present in LDCT images, establishing a mapping from the pseudo-LDCT to the actual NDCT domain without the need for paired CT images. Within the domain of weakly supervised methods, our proposed model exhibited superior objective metrics on the simulated dataset when compared to CycleGAN and selective kernel-based cycle-consistent GAN (SKFCycleGAN): the peak signal-to-noise ratio (PSNR) was 43.9441, the structural similarity index measure (SSIM) was 0.9660, and the visual information fidelity (VIF) was 0.7707. In the clinical dataset, we conducted a visual effect analysis by observing various tissues through different observation windows. Our proposed method achieved a no-reference structural sharpness (NRSS) value of 0.6171, which was closest to that of the NDCT images (NRSS =0.6049), demonstrating its superiority over other denoising techniques in preserving details, maintaining structural integrity, and enhancing edge contrast. Through extensive experiments on both simulated and clinical datasets, we demonstrated the superior efficacy of our proposed method in terms of denoising quality and quantity. Our method exhibits superiority over both supervised techniques, including block-matching and 3D filtering (BM3D), residual encoder-decoder convolutional neural network (RED-CNN), and Wasserstein generative adversarial network-VGG (WGAN-VGG), and over weakly supervised approaches, including CycleGAN and SKFCycleGAN.

Neural style transfer of weak lensing mass maps

We propose a new generative model of projected cosmic mass density maps inferred from weak gravitational lensing observations of distant galaxies (weak lensing mass maps). We construct the model based on a neural style transfer so that it can transform Gaussian weak lensing mass maps into deeply non-Gaussian counterparts as predicted in ray-tracing lensing simulations. We develop an unpaired image-to-image translation method with Cycle-Consistent Generative Adversarial Networks (Cycle GAN), which learn efficient mapping from an input domain to a target domain. Our model is designed to enjoy important advantages; it is trainable with no need for paired simulation data, flexible to make the input domain visually meaningful, and expandable to rapidly-produce a map with a larger sky coverage than training data without additional learning. Using 10,000 lensing simulations, we find that appropriate labeling of training data based on field variance allows the model to reproduce a correct scatter in summary statistics for weak lensing mass maps. Compared with a popular log-normal model, our model improves in predicting the statistical natures of three-point correlations and local properties of rare high-density regions. We also demonstrate that our model enables us to produce a continuous map with a sky coverage of ∼166deg2 but similar non-Gaussian features to training data covering ∼12deg2 in a GPU minute. Hence, our model can be beneficial to massive productions of synthetic weak lensing mass maps, which is of great importance in future precise real-world analyses.

Application of an improved U-Net with image-to-image translation and transfer learning in peach orchard segmentation

Peach cultivation holds a significant economic importance, and obtaining the spatial distribution of peach orchards is helpful for yield prediction and precision agriculture. In this study, we introduce a new U-Net semantic segmentation model, utilizing ResNet50 as a backbone network, augmented with an Efficient Multi-Scale Attention (EMA) mechanism module and a LayerScale adaptive scaling parameter. To address style differences between images from Unmanned Aerial Vehicle (UAV), Google Earth, and Sentinel-2 satellite, we incorporate Cycle-Consistent Generative Adversarial Networks (CycleGAN). This synthesis ensures that UAV images conform to a comparable style found in Google Earth and Sentinel-2 images, while feature details of high spatial resolution UAV images are transferred to Google Earth and Sentinel-2 images through transfer learning. The results demonstrate that using ResNet50 as a backbone network for the U-Net model yields higher accuracy compared to using VGG16 for the U-Net model. Specifically, the Mean Intersection over Union (MIoU) values for UAV and Sentinel-2 images are higher by 0.49 % and 0.95 %, respectively. The MIoU values for UAV, Google Earth, and Sentinel-2 images increased by 0.87 %, 1.71 %, and 1.74 %, respectively, with the introduction of EMA. Additionally, with the introduction of LayerScale adaptive scaling parameters, the MIoU values increased by 0.31 %, 0.33 %, and 1.44 %, respectively, further enhancing the segmentation accuracy of the model. After applying CycleGAN and transfer learning, the MIoU increased by 1.02 %, 0.15 %, and 1.57 % for UAV, Google Earth, and Sentinel-2 images, respectively, resulting in MIoU values of 97.39 %, 92.08 %, and 84.54 %. The comparative analysis with DeepLabV3+, PSPNet, and HRNet models demonstrates the superior mapping performance of the proposed method. Moreover, the method exhibits good generalization and mapping speed across six test sites in the research area. Overall, this approach ensures high precision and efficiency in peach orchard mapping, accommodating various spatial resolutions, and holds potential for addressing diverse requirements in peach orchard mapping applications.

A novel RCACycleGAN model is proposed for the high-precision reconstruction of sparse TFM images

The sparse total focusing method (TFM) has been shown to enhance the computational efficacy of ultrasound imaging but the image quality of ultrasound regrettably deteriorates with an increase in the sparsity rate of array elements. Deep learning has made remarkable advancements in image processing and cycle-consistent generative adversarial networks (CycleGANs) have been extensively employed to reconstruct diverse image categories. However, due to the incomplete extraction of image feature information by the generator and discriminator in a CycleGAN, high-quality sparse TFM images cannot be directly reconstructed using CycleGANs. There is also a risk of losing crucial feature information related to minor defects. As a result, this paper modifies the generator and discriminator in the CycleGAN to construct a new relativistic discriminator and coordinate attention CycleGAN (RCACycleGAN) model, which enables high-precision reconstruction of sparse TFM images. The addition of the coordinate attention module to the CycleGAN enhances the defective feature representation by fully considering the channel and spatial correlation between regions and using the fusion of spatially perceived feature maps in different directions. It solves the problem of easy loss of defective key feature information. The relativistic discriminator replaces the PatchGAN discriminator in the CycleGAN and evaluates the quality of both real and sparse TFM reconstructed images to ensure a relative image quality evaluation. This process solves the problem of unstable image quality of the sparse TFM reconstructed image. Experimental results demonstrate that RCACycleGAN can stably reconstruct sparse TFM images even in small sample dataset scenarios. The proposed network model reconstructs images with better accuracy, including in terms of structural similarity, defect roundness and area, and has a shorter training time than several existing network models.

Reinventing Web Security: An Enhanced Cycle-Consistent Generative Adversarial Network Approach to Intrusion Detection

Web3.0, as the link between the physical and digital domains, faces increasing security threats due to its inherent complexity and openness. Traditional intrusion detection systems (IDSs) encounter formidable challenges in grappling with the multidimensional and nonlinear traffic data characteristic of the Web3.0 environment. Such challenges include insufficient samples of attack data, inadequate feature extraction, and resultant inaccuracies in model classification. Moreover, the scarcity of certain traffic data available for analysis by IDSs impedes the system’s capacity to document instances of malicious behavior. In response to these exigencies, this paper presents a novel approach to Web3.0 intrusion detection, predicated on the utilization of cycle-consistent generative adversarial networks (CycleGANs). Leveraging the data transformation capabilities of its generator, this method facilitates bidirectional conversion between normal Web3.0 behavioral data and potentially intrusive behavioral data. This transformative process not only augments the diversity and volume of recorded intrusive behaviors but also clandestinely simulates various attack scenarios. Furthermore, through fostering mutual competition and learning between the discriminator and generator, the approach enhances the ability to discern the defining characteristics of potential intrusive behaviors, thereby bolstering the accuracy of intrusion detection. To substantiate the efficacy of the CycleGAN-based intrusion detection method, simulation experiments were conducted utilizing public datasets, including KDD CUP 1999 (KDD), CIC-DDOS2019, CIC-IDS2018, and SR-BH 2020. The experimental findings evince the method’s remarkable accuracies across the four datasets, attaining rates of 99.81%, 97.79%, 89.25%, and 95.15%, respectively, while concurrently maintaining low false-positive rates. This research contributes novel insights and methodologies toward the advancement of Web3.0 intrusion detection through the application of CycleGAN technology, which is poised to play a pivotal role in fortifying the security landscape of Web3.0.

SM-CycleGAN: crop image data enhancement method based on self-attention mechanism CycleGAN

Crop disease detection and crop baking stage judgement require large image data to improve accuracy. However, the existing crop disease image datasets have high asymmetry, and the poor baking environment leads to image acquisition difficulties and colour distortion. Therefore, we explore the potential of the self-attention mechanism on crop image datasets and propose an innovative crop image data-enhancement method for recurrent generative adversarial networks (GANs) fused with the self-attention mechanism to significantly enhance the perception and information capture capabilities of recurrent GANs. By introducing the self-attention mechanism module, the cycle-consistent GAN (CycleGAN) is more adept at capturing the internal correlations and dependencies of image data, thus more effectively capturing the critical information among image data. Furthermore, we propose a new enhanced loss function for crop image data to optimise the model performance and meet specific task requirements. We further investigate crop image data enhancement in different contexts to validate the performance and stability of the model. The experimental results show that, the peak signal-to-noise ratio of the SM-CycleGAN for tobacco images and tea leaf disease images are improved by 2.13% and 3.55%, and the structural similarity index measure is improved by 1.16% and 2.48% compared to CycleGAN, respectively.

A Data Augmentation Method for Side-Channel Attacks on Cryptographic Integrated Circuits

The leaked signals, including electromagnetic, power, timing, and temperature generated during the operation of cryptographic devices, contain highly correlated key value information, leading to security vulnerabilities. In practical operations, due to information collection conditions and time limitations, attackers can only obtain limited valid data. At the same time, the attacker’s data environment cannot be ideal, and noise can affect the acquisition of valid information. Therefore, to improve the effectiveness of obtaining key values from side-channel information analysis in cryptographic devices, we proposed a data augmentation method based on cycle-consistent generative adversarial networks named EME-CycleGAN. By using generators and discriminators, new data are generated to expand the original electromagnetic information dataset, aiming for better modeling effects. For evaluating the data augmentation effect on side-channel electromagnetic signals, we employed the Kolmogorov–Smirnov test to characterize the original and generated data, serving as the evaluation standard for our network model and work. We utilize the existing data to model and verify side-channel attacks, evaluating the impact of the generated information on the overall experimental results. The proposed structure consists of three main parts: side-channel information acquisition, data verification analysis, and determination of attack positions. Experimental results demonstrate that effective attacks on encryption algorithms can be achieved under small-sample dataset conditions.

CycleGAN generated pneumonia chest x-ray images: Evaluation with vision transformer

The use of generative models in image synthesis has become increasingly prevalent. Synthetic medical imaging data is of paramount importance, primarily because medical imaging data is scarce, costly, and encumbered by legal considerations pertaining to patient confidentiality. Synthetic medical images offer a potential answer to these issues. The predominant approaches primarily assess the quality of images and the degree of resemblance between these images and the original ones employed for their generation.The central idea of the work can be summarized in the question: Do the performance metrics of Frechet Inception Distance(FID) and Inception Score(IS) in the Cycle-consistent Generative Adversarial Networks (CycleGAN) model are adequate to determine how real a generated chest x-ray pneumonia image is? In this study, a CycleGAN model was employed to produce artificial images depicting 3 classes of chest x-ray pneumonia images: general(any type), bacterial, and viral pneumonia. The quality of the images were evaluated assessing and contrasting 3 criteria: performance metric of CycleGAN model, clinical assessment of respiratory experts and the results of classification of a visual transformer(ViT). The overall results showed that the evaluation metrics of the CycleGAN are insufficient to establish realism in generated medical images.

Feasibility of proton dosimetry overriding planning CT with daily CBCT elaborated through generative artificial intelligence tools

Radiotherapy commonly utilizes cone beam computed tomography (CBCT) for patient positioning and treatment monitoring. CBCT is deemed to be secure for patients, making it suitable for the delivery of fractional doses. However, limitations such as a narrow field of view, beam hardening, scattered radiation artifacts, and variability in pixel intensity hinder the direct use of raw CBCT for dose recalculation during treatment. To address this issue, reliable correction techniques are necessary to remove artifacts and remap pixel intensity into Hounsfield Units (HU) values. This study proposes a deep-learning framework for calibrating CBCT images acquired with narrow field of view (FOV) systems and demonstrates its potential use in proton treatment planning updates. Cycle-consistent generative adversarial networks (cGAN) processes raw CBCT to reduce scatter and remap HU. Monte Carlo simulation is used to generate CBCT scans, enabling the possibility to focus solely on the algorithm’s ability to reduce artifacts and cupping effects without considering intra-patient longitudinal variability and producing a fair comparison between planning CT (pCT) and calibrated CBCT dosimetry. To showcase the viability of the approach using real-world data, experiments were also conducted using real CBCT. Tests were performed on a publicly available dataset of 40 patients who received ablative radiation therapy for pancreatic cancer. The simulated CBCT calibration led to a difference in proton dosimetry of less than 2%, compared to the planning CT. The potential toxicity effect on the organs at risk decreased from about 50% (uncalibrated) up the 2% (calibrated). The gamma pass rate at 3%/2 mm produced an improvement of about 37% in replicating the prescribed dose before and after calibration (53.78% vs 90.26%). Real data also confirmed this with slightly inferior performances for the same criteria (65.36% vs 87.20%). These results may confirm that generative artificial intelligence brings the use of narrow FOV CBCT scans incrementally closer to clinical translation in proton therapy planning updates.

A 4D-CBCT correction network based on contrastive learning for dose calculation in lung cancer

ObjectiveThis study aimed to present a deep-learning network called contrastive learning-based cycle generative adversarial networks (CLCGAN) to mitigate streak artifacts and correct the CT value in four-dimensional cone beam computed tomography (4D-CBCT) for dose calculation in lung cancer patients.Methods4D-CBCT and 4D computed tomography (CT) of 20 patients with locally advanced non-small cell lung cancer were used to paired train the deep-learning model. The lung tumors were located in the right upper lobe, right lower lobe, left upper lobe, and left lower lobe, or in the mediastinum. Additionally, five patients to create 4D synthetic computed tomography (sCT) for test. Using the 4D-CT as the ground truth, the quality of the 4D-sCT images was evaluated by quantitative and qualitative assessment methods. The correction of CT values was evaluated holistically and locally. To further validate the accuracy of the dose calculations, we compared the dose distributions and calculations of 4D-CBCT and 4D-sCT with those of 4D-CT.ResultsThe structural similarity index measure (SSIM) and peak signal-to-noise ratio (PSNR) of the 4D-sCT increased from 87% and 22.31 dB to 98% and 29.15 dB, respectively. Compared with cycle consistent generative adversarial networks, CLCGAN enhanced SSIM and PSNR by 1.1% (p < 0.01) and 0.42% (p < 0.01). Furthermore, CLCGAN significantly decreased the absolute mean differences of CT value in lungs, bones, and soft tissues. The dose calculation results revealed a significant improvement in 4D-sCT compared to 4D-CBCT. CLCGAN was the most accurate in dose calculations for left lung (V5Gy), right lung (V5Gy), right lung (V20Gy), PTV (D98%), and spinal cord (D2%), with the relative dose difference were reduced by 6.84%, 3.84%, 1.46%, 0.86%, 3.32% compared to 4D-CBCT.ConclusionsBased on the satisfactory results obtained in terms of image quality, CT value measurement, it can be concluded that CLCGAN-based corrected 4D-CBCT can be utilized for dose calculation in lung cancer.

Implementation of Anatomy Constrained Contrastive Learning for Heart Chamber Segmentation

The article covers the implementation of anatomy constrained contrastive (ACC) learning networks on a dataset of echocardiographic images with the purpose of segmenting heart chambers for further study. The dataset was partially synthetic, generated using slicing of 3D models of the heart. The images were generated using cycle-consistent generative adversarial networks (CycleGAN) and contrastive unpaired translation (CUT) approaches. By utilizing both CycleGAN and CUT approaches, the generated images captured a wide range of variations in heart chamber segmentation, allowing for a more comprehensive study of the different anatomical features. Proposed approach augments the implementation of ACC learning networks and partially solves the data scarcity problem when it comes to segmentation implementation.

Asymmetric CycleGANs for inverse design of photonic metastructures

Using deep learning to develop nanophotonic structures has been an active field of research in recent years to reduce the time intensive iterative solutions found in finite-difference time-domain simulations. Existing work has primarily used a specific type of generative network: conditional deep convolutional generative adversarial networks. However, these networks have issues with producing clear optical structures in image files; for example, a large number of images show speckled noise, which often results in non-manufacturable structures. Here, we report the first use of cycle-consistent generative adversarial networks to design nanophotonic structures. This approach significantly reduces the amount of speckled noise present in generated geometric structures and allows shapes to have clear edges. We demonstrate that for a given input reflectance spectra, the system generates designs in the form of images, and a complementary network generates reflectance spectra if an image containing a shape is provided as an input. The results show a higher Frechet Inception Distance score than previous approaches, which indicates that the generated structures are of higher quality and are able to learn nonlinear relationships between both datasets. This method of designing nanophotonics provides alternative avenues for development that are more noise robust while still adhering to desired optical properties.

AGLC-GAN: Attention-based global-local cycle-consistent generative adversarial networks for unpaired single image dehazing

Image dehazing is a critical image pre-processing task to estimate the haze-free images corresponding to the input hazy images. Despite the recent advances, the task of image dehazing remains challenging, especially in the unsupervised scenario. Several efforts can be found in the literature to dehaze images in a supervised set-up, where a huge number of paired (clear and hazy images) images are required for training. The supervised approaches often become biased towards the nature of haze present in the training hazy images, and produce less realistic images for query hazy images. We propose an Attention-based Global–Local Cycle-consistent Generative Adversarial Network (AGLC-GAN) for Unpaired Single Image Dehazing. The proposed CycleGAN-based AGLC-GAN model contains a dehazing generator encapsulating an autoencoder-like network with an attention mechanism comprising channel attention and pixel attention to deal with uneven haze intensity across the image. We use a global–local consistent discriminator to identify spatially varying haze and improve the stability of the discriminator. We adopt cyclic perceptual consistency loss to maintain consistency in the feature space. A dynamic feature enhancement module and an adaptive mix-up module are included in the proposed generator to dynamically obtain more spatially structured features and hence, adaptively preserve the flow of shallow features. Furthermore, we extensively experiment with the proposed model on multiple benchmark datasets for evaluating the efficacy of removing haze. The results of the experiments conducted in the study, demonstrate a significant quantitative and qualitative improvement over the existing methods for unpaired image dehazing. 11The code is available at the following link: https://github.com/jr925/AGLC-GAN

An approach to remove the numerical dispersion in elastic wave modeling using R-Cycle-GAN networks

Abstract The finite difference forward modeling has been widely used in geophysics exploration and petroleum fields. Because of its high efficiency and easy application for graphical processing units, it has been widely concerned by industry and academia. However, owing to many factors, the problem of numerical dispersion has been an important factor hindering this method. To overcome the numerical dispersion, this paper proposes a method for removing numerical dispersion using deep learning. Unlike the conventional optimized algorithms target to optimize the finite difference coefficients, our strategy is based on big data training to eliminate the dispersion data after seismic data modeling. We design a neural network architecture based on cycle-consistent generative adversarial networks (Cycle-GANs) and residual learning for elastic wave propagation. Under the premise of not significantly increasing the calculation time, we can obtain higher calculation accuracy. Compared with the high-order finite difference algorithm, the calculation time is the advantage of our proposed deep learning method. Tests prove the efficiency and stability of our proposed algorithm.

Spatiotemporal denoising of low-dose cardiac CT image sequences using RecycleGAN.

Electrocardiogram (ECG)-gated multi-phase computed tomography angiography (MP-CTA) is frequently used for diagnosis of coronary artery disease. Radiation dose may become a potential concern as the scan needs to cover a wide range of cardiac phases during a heart cycle. A common method to reduce radiation is to limit the full-dose acquisition to a predefined range of phases while reducing the radiation dose for the rest. Our goal in this study is to develop a spatiotemporal deep learning method to enhance the quality of low-dose CTA images at phases acquired at reduced radiation dose. Recently, we demonstrated that a deep learning method, Cycle-Consistent generative adversarial networks (CycleGAN), could effectively denoise low-dose CT images through spatial image translation without labeled image pairs in both low-dose and full-dose image domains. As CycleGAN does not utilize the temporal information in its denoising mechanism, we propose to use RecycleGAN, which could translate a series of images ordered in time from the low-dose domain to the full-dose domain through an additional recurrent network. To evaluate RecycleGAN, we use the XCAT phantom program, a highly realistic simulation tool based on real patient data, to generate MP-CTA image sequences for 18 patients (14 for training, 2 for validation and 2 for test). Our simulation results show that RecycleGAN can achieve better denoising performance than CycleGAN based on both visual inspection and quantitative metrics. We further demonstrate the superior denoising performance of RecycleGAN using clinical MP-CTA images from 50 patients.

Large-scale apple orchard mapping from multi-source data using the semantic segmentation model with image- to- image translation and transfer learning

Large-scale mapping of apple orchards through remote sensing is of great significance for apple production management and the sustainable development of the apple industry. The flexible unmanned aerial vehicle (UAV) data and a wide swath of Sentinel-2 (S2) data provided the opportunity to map apple orchards accurately and in a timely manner over a large area. In order to fully combine the advantages of these data and realize the accurate monitoring of apple orchards, this study proposed a semantic segmentation method based on Cycle-Consistent Generative Adversarial Networks (CycleGAN) and transfer learning model (Trans_GAN). First, semantic segmentation models (Fully Convolutional Networks, U-Net, SegNet, DeepLabv3+) were compared. The model with the best performance on both S2 and UAV was selected as the optimal apple orchard recognition model. Second, to solve the problem of domain differences between S2 and UAV, CycleGAN was introduced to convert UAV images into the style of S2 images (Fake S2, F_S2). Finally, this study introduced the transfer learning method and used the F_S2 to assist S2 images to complete the task of extracting large-scale apple planting. Trans_GAN was tested in Zibo and Yantai. The results showed the ability of SegNet to refine the segmentation results and, as a result, achieve the highest extraction accuracy on both UAV and S2 images. The proposed method produced results as high as 20.93% in recall and 15.93% in F1 when compared to the SegNet method based on S2 without image-to-image translation and transfer learning. Therefore, the Trans_GAN method opens a new window for large-scale remote sensing apple orchard mapping.