Cycle-consistent Generative Adversarial Network Research Articles

Mask-guided dual-perception generative adversarial network for synthesizing complex maize diseased leaves to augment datasets

In practice, acquiring and annotating data in specialized domains can be costly, thereby constraining the performance and applicability of deep learning. Utilizing generative models to synthesize data proves to be an effective augmentation technique. Therefore, this research proposes a diseased leaf generation pipeline to diversify the maize disease datasets. We introduce the Dual-Perception Cycle-Consistent Generative Adversarial Network (DP-CycleGAN). During training, incorporating our proposed Structure Perception (SP) loss and Texture Perception (TP) loss functions. These losses guide the model's attention areas through activation reconstruction and mask mechanisms, thereby improving the overall perceptual quality of the generated images and the realism of the disease lesions. We constructed a maize leaf mixed disease dataset to simulate the complex conditions of real-world disease occurrence. Experimental results show that the DP-CycleGAN generates higher-quality and more realistic diseased leaf images. Compared to CycleGAN and state-of-the-art method, DP-CycleGAN shows a 29.6% and 15.7% reduction in Fréchet Inception Distance (FID) scores and a 125.3% and 61.5% increase in Structural Similarity (SSIM) values, respectively. Simultaneously, by incorporating synthetic data during training, our approach significantly enhances the performance of the recognition model in scenarios of both data abundance and scarcity, with improvement rates exceeding two times those of existing state-of-the-art methods. This contributes to the application of Artificial Intelligence (AI) in agricultural production practices.

Nighttime road scene image enhancement based on cycle-consistent generative adversarial network.

During nighttime road scenes, images are often affected by contrast distortion, loss of detailed information, and a significant amount of noise. These factors can negatively impact the accuracy of segmentation and object detection in nighttime road scenes. A cycle-consistent generative adversarial network has been proposed to address this issue to improve the quality of nighttime road scene images. The network includes two generative networks with identical structures and two adversarial networks with identical structures. The generative network comprises an encoder network and a corresponding decoder network. A context feature extraction module is designed as the foundational element of the encoder-decoder network to capture more contextual semantic information with different receptive fields. A receptive field residual module is also designed to increase the receptive field in the encoder network.The illumination attention module is inserted between the encoder and decoder to transfer critical features extracted by the encoder to the decoder. The network also includes a multiscale discriminative network to discriminate better whether the image is a real high-quality or generated image. Additionally, an improved loss function is proposed to enhance the efficacy of image enhancement. Compared to state-of-the-art methods, the proposed approach achieves the highest performance in enhancing nighttime images, making them clearer and more natural.

Pre-stack seismic inversion based on model-constrained generative adversarial network

Pre-stack seismic inversion usually uses various traditional algorithms to estimate elastic parameters such as P-wave velocity, S-wave velocity, and density. It is hard to derive accurate elastic parameters due to their non-uniqueness and high dimensionality between elastic parameters and seismic data, the calculation of elastic parameters is inaccurate. Convolutional Neural Networks (CNNs) have high-dimensional feature space mapping capabilities, which are utilized to establish mapping relationships between seismic data and elasticity parameters. However, their effectiveness is greatly affected by label data, and at the same time, due to the lack of enough label data, resulting in a low degree of fitting between prediction results and real data. In addition, conventional seismic inversion methods based on CNNs lack physical model constraints, resulting in low accuracy and poor interpretability of prediction results. We propose a Cycle-consistent Generative Adversarial Network based on a geophysical mechanism (SeisInv-CycleGAN). Deterministic inversion results and labeled data are combined into hybrid geophysical data as a training set of SeisInv-CycleGAN with geophysical constraints. At the same time, the residual (seismic loss) between the seismic data synthesized by forward modeling and the actual data is used as part of the loss function. The SeisInv-CycleGAN does not require building an initial model, and it can achieve higher accuracy in prediction results with a small amount of labeled data.

Variational mode decomposition-enhanced thin cloud removal using UNet vision transformer cycle-consistent generative adversarial network

Variational mode decomposition-enhanced thin cloud removal using UNet vision transformer cycle-consistent generative adversarial network

CycleGAN-Driven MR-Based Pseudo-CT Synthesis for Knee Imaging Studies

In the field of knee imaging, the incorporation of MR-based pseudo-CT synthesis holds the potential to mitigate the need for separate CT scans, simplifying workflows, enhancing patient comfort, and reducing radiation exposure. In this work, we present a novel DL framework, grounded in the development of the Cycle-Consistent Generative Adversarial Network (CycleGAN) method, tailored specifically for the synthesis of pseudo-CT images in knee imaging to surmount the limitations of current methods. Upon visually examining the outcomes, it is evident that the synthesized pseudo-CTs show an excellent quality and high robustness. Despite the limited dataset employed, the method is able to capture the particularities of the bone contours in the resulting image. The experimental Mean Absolute Error (MAE), Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index Measure (SSIM), Zero-Normalized Cross Correlation (ZNCC), Mutual Information (MI), Relative Change (RC), and absolute Relative Change (|RC|) report values of 30.4638 ± 7.4770, 28.1168 ± 1.5245, 0.9230 ± 0.0217, 0.9807 ± 0.0071, 0.8548 ± 0.1019, 0.0055 ± 0.0265, and 0.0302 ± 0.0218 (median ± median absolute deviation), respectively. The voxel-by-voxel correlation plot shows an excellent correlation between pseudo-CT and ground-truth CT Hounsfield units (m = 0.9785; adjusted R2 = 0.9988; ρ = 0.9849; p < 0.001). The Bland–Altman plot shows that the average of the differences is low ((HUCT−HUpseudo−CT = 0.7199 ± 35.2490; 95% confidence interval [−68.3681, 69.8079]). This study represents the first reported effort in the field of MR-based knee pseudo-CT synthesis, shedding light to significantly advance the field of knee imaging.

Investigating the causes of permeability anisotropy in heterogeneous conglomeratic sandstone using multiscale digital rock

Heterogeneous conglomeratic sandstone exhibits anisotropic physical properties, rendering a comprehensive analysis of its physical processes challenging with experimental measurements. Digital rock technology provides a visual and intuitive analysis of the microphysical processes in rocks, thereby aiding in scientific inquiry. Nevertheless, the multiscale characteristics of conglomeratic sandstone cannot be fully captured by a single-scale digital rock, thus limiting its ability to characterize the pore structure. Our work introduces a proposed workflow that uses multiscale digital rock fusion to investigate permeability anisotropy in heterogeneous rock. We use a cycle-consistent generative adversarial network to fuse computed tomography scan data of different resolutions, creating a large-scale and high-precision digital rock that comprehensively represents the conglomeratic sandstone pore structure. Subsequently, the digital rock is partitioned into multiple blocks, and the permeability of each block is simulated using a pore network. Finally, the total permeability of the sample is calculated by conducting an upscaling numerical simulation using the Darcy-Stokes equation. This process facilitates the analysis of the pore structure in conglomeratic sandstone and provides a step-by-step solution for permeability. From a multiscale perspective, this approach reveals that the anisotropy of permeability in conglomeratic sandstone stems from the layered distribution of grain sizes and differences in grain arrangement across different directions.

Multicenter PET image harmonization using generative adversarial networks

PurposeTo improve reproducibility and predictive performance of PET radiomic features in multicentric studies by cycle-consistent generative adversarial network (GAN) harmonization approaches.MethodsGAN-harmonization was developed to harmonize whole-body PET scans to perform image style and texture translation between different centers and scanners. GAN-harmonization was evaluated by application to two retrospectively collected open datasets and different tasks. First, GAN-harmonization was performed on a dual-center lung cancer cohort (127 female, 138 male) where the reproducibility of radiomic features in healthy liver tissue was evaluated. Second, GAN-harmonization was applied to a head and neck cancer cohort (43 female, 154 male) acquired from three centers. Here, the clinical impact of GAN-harmonization was analyzed by predicting the development of distant metastases using a logistic regression model incorporating first-order statistics and texture features from baseline 18F-FDG PET before and after harmonization.ResultsImage quality remained high (structural similarity: left kidney ≥\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\ge$$\\end{document} 0.800, right kidney ≥\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\ge$$\\end{document} 0.806, liver ≥\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\ge$$\\end{document} 0.780, lung ≥\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\ge$$\\end{document} 0.838, spleen ≥\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\ge$$\\end{document} 0.793, whole-body ≥\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\ge$$\\end{document} 0.832) after image harmonization across all utilized datasets. Using GAN-harmonization, inter-site reproducibility of radiomic features in healthy liver tissue increased at least by ≥\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\ge$$\\end{document} 5 ± 14% (first-order), ≥\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\ge$$\\end{document} 16 ± 7% (GLCM), ≥\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\ge$$\\end{document} 19 ± 5% (GLRLM), ≥\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\ge$$\\end{document} 16 ± 8% (GLSZM), ≥\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\ge$$\\end{document} 17 ± 6% (GLDM), and ≥\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\ge$$\\end{document} 23 ± 14% (NGTDM). In the head and neck cancer cohort, the outcome prediction improved from AUC 0.68 (95% CI 0.66–0.71) to AUC 0.73 (0.71–0.75) by application of GAN-harmonization.ConclusionsGANs are capable of performing image harmonization and increase reproducibility and predictive performance of radiomic features derived from different centers and scanners.

Deep Learning Synthesis of White-Blood From Dark-Blood Late Gadolinium Enhancement Cardiac Magnetic Resonance.

Dark-blood late gadolinium enhancement (DB-LGE) cardiac magnetic resonance has been proposed as an alternative to standard white-blood LGE (WB-LGE) imaging protocols to enhance scar-to-blood contrast without compromising scar-to-myocardium contrast. In practice, both DB and WB contrasts may have clinical utility, but acquiring both has the drawback of additional acquisition time. The aim of this study was to develop and evaluate a deep learning method to generate synthetic WB-LGE images from DB-LGE, allowing the assessment of both contrasts without additional scan time. DB-LGE and WB-LGE data from 215 patients were used to train 2 types of unpaired image-to-image translation deep learning models, cycle-consistent generative adversarial network (CycleGAN) and contrastive unpaired translation, with 5 different loss function hyperparameter settings each. Initially, the best hyperparameter setting was determined for each model type based on the Fréchet inception distance and the visual assessment of expert readers. Then, the CycleGAN and contrastive unpaired translation models with the optimal hyperparameters were directly compared. Finally, with the best model chosen, the quantification of scar based on the synthetic WB-LGE images was compared with the truly acquired WB-LGE. The CycleGAN architecture for unpaired image-to-image translation was found to provide the most realistic synthetic WB-LGE images from DB-LGE images. The results showed that it was difficult for visual readers to distinguish if an image was true or synthetic (55% correctly classified). In addition, scar burden quantification with the synthetic data was highly correlated with the analysis of the truly acquired images. Bland-Altman analysis found a mean bias in percentage scar burden between the quantification of the real WB and synthetic white-blood images of 0.44% with limits of agreement from -10.85% to 11.74%. The mean image quality of the real WB images (3.53/5) was scored higher than the synthetic white-blood images (3.03), P = 0.009. This study proposed a CycleGAN model to generate synthetic WB-LGE from DB-LGE images to allow assessment of both image contrasts without additional scan time. This work represents a clinically focused assessment of synthetic medical images generated by artificial intelligence, a topic with significant potential for a multitude of applications. However, further evaluation is warranted before clinical adoption.

The Application and Exploration of Data Visualization Technology in Artistic Expression and Exhibition

Embarking on the transformation of virtual art museums, the confluence of data visualization technology and deep learning emerges as a catalyst for redefining artistic expression. Driven by the desire to change traditional approaches, this work utilizes a sophisticated deep learning model, the self-attention-based cycle-consistent generative adversarial network (SA-CCGAN), for transformative purposes. The objective of proposed Modernizing Virtual Art: The Collaboration of Data Visualization Technology and Self-Attention-Based Cycle-Consistent Generative Adversarial Network for Artistic Expression (MVA: DVT & SA-CCGAN) is to elevate artistic expression and exhibition practices within the virtual space, marking a paradigm shift in traditional approaches. SA-CCGAN, renowned for its ability to generate realistic and coherent artistic representations, serves as the cornerstone of this transformative endeavor. The workflow intricately incorporates SA-CCGAN into the virtual art museum context, enhancing stylized outcomes and capturing global geometric features. This yields a computationally efficient representation of high-dimensional visual data, revolutionizing digital art presentation. The fusion of data visualization technology, creativity, and meticulous design not only signals innovation but also has the potential to redefine the virtual art landscape, fostering accessibility, engagement, and a deeper cultural exchange in digital art exploration.

Reinventing Web Security: An Enhanced Cycle-Consistent Generative Adversarial Network Approach to Intrusion Detection

Web3.0, as the link between the physical and digital domains, faces increasing security threats due to its inherent complexity and openness. Traditional intrusion detection systems (IDSs) encounter formidable challenges in grappling with the multidimensional and nonlinear traffic data characteristic of the Web3.0 environment. Such challenges include insufficient samples of attack data, inadequate feature extraction, and resultant inaccuracies in model classification. Moreover, the scarcity of certain traffic data available for analysis by IDSs impedes the system’s capacity to document instances of malicious behavior. In response to these exigencies, this paper presents a novel approach to Web3.0 intrusion detection, predicated on the utilization of cycle-consistent generative adversarial networks (CycleGANs). Leveraging the data transformation capabilities of its generator, this method facilitates bidirectional conversion between normal Web3.0 behavioral data and potentially intrusive behavioral data. This transformative process not only augments the diversity and volume of recorded intrusive behaviors but also clandestinely simulates various attack scenarios. Furthermore, through fostering mutual competition and learning between the discriminator and generator, the approach enhances the ability to discern the defining characteristics of potential intrusive behaviors, thereby bolstering the accuracy of intrusion detection. To substantiate the efficacy of the CycleGAN-based intrusion detection method, simulation experiments were conducted utilizing public datasets, including KDD CUP 1999 (KDD), CIC-DDOS2019, CIC-IDS2018, and SR-BH 2020. The experimental findings evince the method’s remarkable accuracies across the four datasets, attaining rates of 99.81%, 97.79%, 89.25%, and 95.15%, respectively, while concurrently maintaining low false-positive rates. This research contributes novel insights and methodologies toward the advancement of Web3.0 intrusion detection through the application of CycleGAN technology, which is poised to play a pivotal role in fortifying the security landscape of Web3.0.

Communication and Influence of Traditional Culture Based on Social Network Analysis

Social media significantly impacts traditional cultures, yet existing methods struggle to capture the delicate complexities of online cultural discussions. This lack of depth limits understanding of how traditions evolve and adapt in the digital age. To address this, CaCCGAN: Social Culture Analysis, a framework leveraging social network analysis (SNA) is proposed. Content-Aware Cycle-Consistent Generative Adversarial Network (CaCCGAN) explores communication and influence dynamics surrounding traditional culture on social media platforms. CaCCGAN gathers real-time data on traditional culture from Weibo, a prominent Chinese platform, focusing on relevant keywords. By employing Term Frequency-Inverse Document Frequency (TF-IDF) and a Binary Waterwheel Plant Optimization Algorithm (BWPOA), CaCCGAN identifies user-tradition relationships, influential users, and online communities. Furthermore, the CaCCGAN model analyzes communication styles and generates content reflecting these relationships. Analyzing this generated content offers valuable insights for stakeholders interested in cultural preservation, targeted marketing campaigns, and content creation strategies. The proposed CaCCGAN: Social Culture Analysis model attains 25.23%, 21.06% and 55.11% higher Precision value and 29.096%, 41.64% and 33.33% higher F1 Score value compared with the existing method such as PA-BiLSTM Neural Encoding for Cultural Adaptation and Emotion Analysis in Modern Media Communication (PA-BiLSTM), MFCSNet: Modeling Musician-Follower Dynamics in a Complex Social Network to Measure Musical Influence (MFCSNet) and Utilizing Social Media Data Analytics to Integrate Traditional Culture Through Digital Media Interaction and Dissemination (I-RankClus-WOA) respectively. This proposed CaCCGAN: Social Culture Analysis model presents a promising approach to understanding the evolving dynamics of communication and influence surrounding traditional culture online.

Unsupervised speckle denoising in digital holographic interferometry based on 4-f optical simulation integrated cycle-consistent generative adversarial network

The speckle noise generated during digital holographic interferometry (DHI) is unavoidable and difficult to eliminate, thus reducing its accuracy. We propose a self-supervised deep-learning speckle denoising method using a cycle-consistent generative adversarial network to mitigate the effect of speckle noise. The proposed method integrates a 4-f optical speckle noise simulation module with a parameter generator. In addition, it uses an unpaired dataset for training to overcome the difficulty in obtaining noise-free images and paired data from experiments. The proposed method was tested on both simulated and experimental data, with results showing a 6.9% performance improvement compared with a conventional method and a 2.6% performance improvement compared with unsupervised deep learning in terms of the peak signal-to-noise ratio. Thus, the proposed method exhibits superior denoising performance and potential for DHI, being particularly suitable for processing large datasets.

Deep learning-based glomerulus detection and classification with generative morphology augmentation in renal pathology images

Glomerulus morphology on renal pathology images provides valuable diagnosis and outcome prediction information. To provide better care, an efficient, standardized, and scalable method is urgently needed to optimize the time-consuming and labor-intensive interpretation process by renal pathologists. This paper proposes a deep convolutional neural network (CNN)-based approach to automatically detect and classify glomeruli with different stains in renal pathology images. In the glomerulus detection stage, this paper proposes a flattened Xception with a feature pyramid network (FX-FPN). The FX-FPN is employed as a backbone in the framework of faster region-based CNN to improve glomerulus detection performance. In the classification stage, this paper considers classifications of five glomerulus morphologies using a flattened Xception classifier. To endow the classifier with higher discriminability, this paper proposes a generative data augmentation approach for patch-based glomerulus morphology augmentation. New glomerulus patches of different morphologies are generated for data augmentation through the cycle-consistent generative adversarial network (CycleGAN). The single detection model shows the F1 score up to 0.9524 in H&E and PAS stains. The classification result shows that the average sensitivity and specificity are 0.7077 and 0.9316, respectively, by using the flattened Xception with the original training data. The sensitivity and specificity increase to 0.7623 and 0.9443, respectively, by using the generative data augmentation. Comparisons with different deep CNN models show the effectiveness and superiority of the proposed approach.

Enhancing scanning electron microscopy imaging quality of weakly conductive samples through unsupervised learning

Scanning electron microscopy (SEM) is a crucial tool for analyzing submicron-scale structures. However, the attainment of high-quality SEM images is contingent upon the high conductivity of the material due to constraints imposed by its imaging principles. For weakly conductive materials or structures induced by intrinsic properties or organic doping, the SEM imaging quality is significantly compromised, thereby impeding the accuracy of subsequent structure-related analyses. Moreover, the unavailability of paired high–low quality images in this context renders the supervised-based image processing methods ineffective in addressing this challenge. Here, an unsupervised method based on Cycle-consistent Generative Adversarial Network (CycleGAN) was proposed to enhance the quality of SEM images for weakly conductive samples. The unsupervised model can perform end-to-end learning using unpaired blurred and clear SEM images from weakly and well-conductive samples, respectively. To address the requirements of material structure analysis, an edge loss function was further introduced to recover finer details in the network-generated images. Various quantitative evaluations substantiate the efficacy of the proposed method in SEM image quality improvement with better performance than the traditional methods. Our framework broadens the application of artificial intelligence in materials analysis, holding significant implications in fields such as materials science and image restoration.

Domain adaptation for structural health monitoring via physics-informed and self-attention-enhanced generative adversarial learning

Health monitoring technologies, empowered by sensor-driven information and model updating, play an important role in assessing the status of civil structures and detecting anomalies. However, significant domain discrepancies exist in the distribution of physical parameters between numerical structural models and their real-world counterparts. As a result, many health monitoring theories struggle to be effective in practice, even if they perform perfectly in the simulated data. To bridge the domain discrepancies, this paper proposes an unsupervised domain adaptation approach based on an adapted cycle-consistent generative adversarial network (CycleGAN) that incorporates physical constraints and a self-attention mechanism. The approach focuses on the mutual transformation of the multi-channel time series obtained from numerical models and actual structures. Specifically, the physical constraints are derived from the governing equation of linear dynamic systems, while the self-attention mechanism is achieved by adding transformer structures to both the generator and discriminator. Through free-vibration experiments on a steel beam and a large-scale steel bridge model, the physical constraints and transformer structures have proven beneficial for improving the learning capability and training stability of the GAN model. Furthermore, the proposed approach is not only verified as effective in transforming acceleration responses between the test structures and their corresponding finite element models in both time and frequency domains but has also been shown to reproduce mode shapes accurately.

Learning CT-free attenuation-corrected total-body PET images through deep learning.

Total-body PET/CT scanners with long axial fields of view have enabled unprecedented image quality and quantitative accuracy. However, the ionizing radiation from CT is a major issue in PET imaging, which is more evident with reduced radiopharmaceutical doses in total-body PET/CT. Therefore, we attempted to generate CT-free attenuation-corrected (CTF-AC) total-body PET images through deep learning. Based on total-body PET data from 122 subjects (29 females and 93 males), a well-established cycle-consistent generative adversarial network (Cycle-GAN) was employed to generate CTF-AC total-body PET images directly while introducing site structures as prior information. Statistical analyses, including Pearson correlation coefficient (PCC) and t-tests, were utilized for the correlation measurements. The generated CTF-AC total-body PET images closely resembled real AC PET images, showing reduced noise and good contrast in different tissue structures. The obtained peak signal-to-noise ratio and structural similarity index measure values were 36.92 ± 5.49 dB (p < 0.01) and 0.980 ± 0.041 (p < 0.01), respectively. Furthermore, the standardized uptake value (SUV) distribution was consistent with that of real AC PET images. Our approach could directly generate CTF-AC total-body PET images, greatly reducing the radiation risk to patients from redundant anatomical examinations. Moreover, the model was validated based on a multidose-level NAC-AC PET dataset, demonstrating the potential of our method for low-dose PET attenuation correction. In future work, we will attempt to validate the proposed method with total-body PET/CT systems in more clinical practices. The ionizing radiation from CT is a major issue in PET imaging, which is more evident with reduced radiopharmaceutical doses in total-body PET/CT. Our CT-free PET attenuation correction method would be beneficial for a wide range of patient populations, especially for pediatric examinations and patients who need multiple scans or who require long-term follow-up. • CT is the main source of radiation in PET/CT imaging, especially for total-body PET/CT devices, and reduced radiopharmaceutical doses make the radiation burden from CT more obvious. • The CT-free PET attenuation correction method would be beneficial for patients who need multiple scans or long-term follow-up by reducing additional radiation from redundant anatomical examinations. • The proposed method could directly generate CT-free attenuation-corrected (CTF-AC) total-body PET images, which is beneficial for PET/MRI or PET-only devices lacking CT image poses.

Optimized self-attention based cycle-consistent generative adversarial network adopted melanoma classification from dermoscopic images.

Skin is the exposed part of the human body that constantly protected from UV rays, heat, light, dust, and other hazardous radiation. One of the most dangerous illnesses that affect people is skin cancer. A type of skin cancer called melanoma starts in the melanocytes, which regulate the colour in human skin. Reducing the fatality rate from skin cancer requires early detection and diagnosis of conditions like melanoma. In this article, a Self-attention based cycle-consistent generative adversarial network optimized with Archerfish Hunting Optimization Algorithm adopted Melanoma Classification (SACCGAN-AHOA-MC-DI) from dermoscopic images is proposed. Primarily, the input Skin dermoscopic images are gathered via the dataset of ISIC 2019. Then, the input Skin dermoscopic images is pre-processed using adjusted quick shift phase preserving dynamic range compression (AQSP-DRC) for removing noise and increase the quality of Skin dermoscopic images. These pre-processed images are fed to the piecewise fuzzy C-means clustering (PF-CMC) for ROI region segmentation. The segmented ROI region is supplied to the Hexadecimal Local Adaptive Binary Pattern (HLABP) to extract the Radiomic features, like Grayscale statistic features (standard deviation, mean, kurtosis, and skewness) together with Haralick Texture features (contrast, energy, entropy, homogeneity, and inverse different moments). The extracted features are fed to self-attention based cycle-consistent generative adversarial network (SACCGAN) which classifies the skin cancers as Melanocytic nevus, Basal cell carcinoma, Actinic Keratosis, Benign keratosis, Dermatofibroma, Vascular lesion, Squamous cell carcinoma and melanoma. In general, SACCGAN not adapt any optimization modes to define the ideal parameters to assure accurate classification of skin cancer. Hence, Archerfish Hunting Optimization Algorithm (AHOA) is considered to maximize the SACCGAN classifier, which categorizes the skin cancer accurately. The proposed method attains 23.01%, 14.96%, and 45.31% higher accuracy and 32.16%, 11.32%, and 24.56% lesser computational time evaluated to the existing methods, like melanoma prediction method for unbalanced data utilizing optimized Squeeze Net through bald eagle search optimization (CNN-BES-MC-DI), hyper-parameter optimized CNN depending on Grey wolf optimization algorithm (CNN-GWOA-MC-DI), DEANN incited skin cancer finding depending on fuzzy c-means clustering (DEANN-MC-DI). RESEARCH HIGHLIGHTS: This manuscript, self-attention based cycle-consistent. SACCGAN-AHOA-MC-DI method is implemented in Python. (SACCGAN-AHOA-MC-DI) from dermoscopic images is proposed. Adjusted quick shift phase preserving dynamic range compression (AQSP-DRC). Removing noise and increase the quality of Skin dermoscopic images.

Diagnostic performance of deep-learning-based virtual chromoendoscopy in gastric neoplasms.

Cycle-consistent generative adversarial network (CycleGAN) is a deep neural network model that performs image-to-image translations. We generated virtual indigo carmine (IC) chromoendoscopy images of gastric neoplasms using CycleGAN and compared their diagnostic performance with that of white light endoscopy (WLE). WLE and IC images of 176 patients with gastric neoplasms who underwent endoscopic resection were obtained. We used 1,633 images (911 WLE and 722 IC) of 146 cases in the training dataset to develop virtual IC images using CycleGAN. The remaining 30 WLE images were translated into 30 virtual IC images using the trained CycleGAN and used for validation. The lesion borders were evaluated by 118 endoscopists from 22 institutions using the 60 paired virtual IC and WLE images. The lesion area concordance rate and successful whole-lesion diagnosis were compared. The lesion area concordance rate based on the pathological diagnosis in virtual IC was lower than in WLE (44.1% vs. 48.5%, p < 0.01). The successful whole-lesion diagnosis was higher in the virtual IC than in WLE images; however, the difference was insignificant (28.2% vs. 26.4%, p = 0.11). Conversely, subgroup analyses revealed a significantly higher diagnosis in virtual IC than in WLE for depressed morphology (41.9% vs. 36.9%, p = 0.02), differentiated histology (27.6% vs. 24.8%, p = 0.02), smaller lesion size (42.3% vs. 38.3%, p = 0.01), and assessed by expert endoscopists (27.3% vs. 23.6%, p = 0.03). The diagnostic ability of virtual IC was higher for some lesions, but not completely superior to that of WLE. Adjustments are required to improve the imaging system's performance.

Cycle-Retinex: Unpaired Low-Light Image Enhancement via Retinex-Inline CycleGAN

Low-light image enhancement aims to recover normal-light images from the images captured under dim environments. Most existing methods could just improve the light appearance globally whereas failing to handle other degradation such as dense noise, color offset and extremely low-light. Moreover, unsupervised methods proposed in recent years lack reliable physical model as the basis, thus universality is greatly limited. To address these problems, we propose a novel low-light image enhancement method via Retinex-inline cycle-consistent generative adversarial network named Cycle-Retinex, whose training is totally dependent on unpaired datasets. Specifically, we organically combine Retinex theory with CycleGAN, by which we decouple low-light image enhancement task into two sub-tasks, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> illumination map enhancement and reflectance map restoration. Retinex theory helps CycleGAN simplify low-light image enhancement problem and CycleGAN provides synthetic paired images to guide the training of Retinex decomposition network. We further introduce a self-augmented method to address the color distortion and noise problem, thus making the network learn to enhance low-light images adaptively. Extensive experiments show that the proposed method can achieve promising results. The source code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/mummmml/Cycle-Retinex</uri> .

A Sim-to-Real Learning-Based Framework for Contact-Rich Assembly by Utilizing CycleGAN and Force Control

Deep reinforcement learning (RL) has succeeded in robotic manipulation applications. However, training robots in the real world is challenging due to sample efficiency and safety concerns. Sim-to-real transfer has been proposed to address the aforementioned concerns but introduces the reality gap. In this work, we introduce a sim-to-real learning framework for vision-based assembly tasks and perform training in a simulation environment by employing raw image inputs from a single camera to address the aforementioned issues. We build a robotic Peg-in-Hole (PiH) training environment that requires low-level simulation knowledge. We also present a domain adaptation method based on a cycle-consistent generative adversarial network (CycleGAN) and a force control transfer approach to bridge the reality gap. The proposed framework, trained in a simulation environment with different environmental scenes, can be successfully transferred to a real PiH setup with a UR5e robot. We then reproduce these results with a Diana7 robot and different peg shapes to verify the generalization ability of the framework.