Generative Network Architecture Research Articles

Image-domain material decomposition for dual-energy CT using unsupervised learning with data-fidelity loss.

Dual-energy computed tomography (DECT) and material decomposition play vital roles in quantitative medical imaging. However, the decomposition process may suffer from significant noise amplification, leading to severely degraded image signal-to-noise ratios (SNRs). While existing iterative algorithms perform noise suppression using different image priors, these heuristic image priors cannot accurately represent the features of the target image manifold. Although deep learning-based decomposition methods have been reported, these methods are in the supervised-learning framework requiring paired data for training, which is not readily available in clinical settings. This work aims to develop an unsupervised-learning framework with data-measurement consistency for image-domain material decomposition in DECT. The proposed framework combines iterative decomposition and deep learning-based image prior in a generative adversarial network (GAN) architecture. In the generator module, a data-fidelity loss is introduced to enforce the measurement consistency in material decomposition. In the discriminator module, the discriminator is trained to differentiate the low-noise material-specific images from the high-noise images. In this scheme, paired images of DECT and ground-truth material-specific images are not required for the model training. Once trained, the generator can perform image-domain material decomposition with noise suppression in a single step. In the simulation studies of head and lung digital phantoms, the proposed method reduced the standard deviation (SD) in decomposed images by 97% and 91% from the values in direct inversion results. It also generated decomposed images with structural similarity index measures (SSIMs) greater than 0.95 against the ground truth. In the clinical head and lung patient studies, the proposed method suppressed the SD by 95% and 93% compared to the decomposed images of matrix inversion. Since the invention of DECT, noise amplification during material decomposition has been one of the biggest challenges, impeding its quantitative use in clinical practice. The proposed method performs accurate material decomposition with efficient noise suppression. Furthermore, the proposed method is within an unsupervised-learning framework, which does not require paired data for model training and resolves the issue of lack of ground-truth data in clinical scenarios.

Improved brain metastases segmentation using generative adversarial network and conditional random field optimization mask R-CNN.

In radiotherapy, the delineation of the gross tumor volume (GTV) in brain metastases using computed tomography (CT) simulation localization is very important. However, despite the criticality of this process, a pronounced gap exists in the availability of tools tailored for the automatic segmentation of the GTV based on CT simulation localization images. This study aims to fill this gap by devising an effective tool specifically for the automatic segmentation of the GTV using CT simulation localization images. A dual-network generative adversarial network (GAN) architecture was developed, wherein the generator focused on refining CT images for more precise delineation, and the discriminator differentiated between real and augmented images. This architecture was coupled with the Mask R-CNN model to achieve meticulous GTV segmentation. An end-to-end training process facilitated the integration between the GAN and Mask R-CNN functionalities. Furthermore, a conditional random field (CRF) was incorporated to refine the initial masks generated by the Mask R-CNN model to ensure optimal segmentation accuracy. The performance was assessed using key metrics, namely, the Dice coefficient (DSC), intersection over union (IoU), accuracy, specificity, and sensitivity. The GAN+Mask R-CNN+CRF integration method in this study performs well in GTV segmentation. In particular, the model has an overall average DSC of 0.819±0.102 and an IoU of 0.712±0.111 in the internal validation. The overall average DSC in the external validation data is 0.726±0.128 and the IoU is 0.640±0.136. It demonstrates favorable generalization ability. The integration of the GAN, Mask R-CNN, and CRF optimization provides a pioneering tool for the sophisticated segmentation of the GTV in brain metastases using CT simulation localization images. The method proposed in this study can provide a robust automatic segmentation approach for brain metastases in the absence of MRI.

Transfer-Learning-Enhanced Regression Generative Adversarial Networks for Optimal eVTOL Takeoff Trajectory Prediction

Electric vertical takeoff and landing (eVTOL) aircraft represent a crucial aviation technology to transform future transportation systems. The unique characteristics of eVTOL aircraft include reduced noise, low pollutant emission, efficient operating cost, and flexible maneuverability, which in the meantime pose critical challenges to advanced power retention techniques. Thus, optimal takeoff trajectory design is essential due to immense power demands during eVTOL takeoffs. Conventional design optimizations, however, adopt high-fidelity simulation models in an iterative manner resulting in a computationally intensive mechanism. In this work, we implement a surrogate-enabled inverse mapping optimization architecture, i.e., directly predicting optimal designs from design requirements (including flight conditions and design constraints). A trained inverse mapping surrogate performs real-time optimal eVTOL takeoff trajectory predictions with no need for running optimizations; however, one training sample requires one design optimization in this inverse mapping setup. The excessive training cost of inverse mapping and the characteristics of optimal eVTOL takeoff trajectories necessitate the development of the regression generative adversarial network (regGAN) surrogate. We propose to further enhance regGAN predictive performance through the transfer learning (TL) technique, creating a scheme termed regGAN-TL. In particular, the proposed regGAN-TL scheme leverages the generative adversarial network (GAN) architecture consisting of a generator network and a discriminator network, with a combined loss of the mean squared error (MSE) and binary cross-entropy (BC) losses, for regression tasks. In this work, the generator employs design requirements as input and produces optimal takeoff trajectory profiles, while the discriminator differentiates the generated profiles and real optimal profiles in the training set. The combined loss facilitates the generator training in the dual aspects: the MSE loss targets minimum differences between generated profiles and training counterparts, while the BC loss drives the generated profiles to share analogous patterns with the training set. We demonstrated the utility of regGAN-TL on optimal takeoff trajectory designs for the Airbus A3 Vahana and compared its performance against representative surrogates, including the multi-output Gaussian process, the conditional GAN, and the vanilla regGAN. Results showed that regGAN-TL reached the 99.5% generalization accuracy threshold with only 200 training samples while the best reference surrogate required 400 samples. The 50% reduction in training expense and reduced standard deviations of generalization accuracy achieved by regGAN-TL confirmed its outstanding predictive performance and broad engineering application potential.

Deep learning-based whole-body PSMA PET/CT attenuation correction utilizing Pix-2-Pix GAN.

Sequential PET/CT studies oncology patients can undergo during their treatment follow-up course is limited by radiation dosage. We propose an artificial intelligence (AI) tool to produce attenuation-corrected PET (AC-PET) images from non-attenuation-corrected PET (NAC-PET) images to reduce need for low-dose CT scans. A deep learning algorithm based on 2D Pix-2-Pix generative adversarial network (GAN) architecture was developed from paired AC-PET and NAC-PET images. 18F-DCFPyL PSMA PET-CT studies from 302 prostate cancer patients, split into training, validation, and testing cohorts (n = 183, 60, 59, respectively). Models were trained with two normalization strategies: Standard Uptake Value (SUV)-based and SUV-Nyul-based. Scan-level performance was evaluated by normalized mean square error (NMSE), mean absolute error (MAE), structural similarity index (SSIM), and peak signal-to-noise ratio (PSNR). Lesion-level analysis was performed in regions-of-interest prospectively from nuclear medicine physicians. SUV metrics were evaluated using intraclass correlation coefficient (ICC), repeatability coefficient (RC), and linear mixed-effects modeling. Median NMSE, MAE, SSIM, and PSNR were 13.26%, 3.59%, 0.891, and 26.82, respectively, in the independent test cohort. ICC for SUVmax and SUVmean were 0.88 and 0.89, which indicated a high correlation between original and AI-generated quantitative imaging markers. Lesion location, density (Hounsfield units), and lesion uptake were all shown to impact relative error in generated SUV metrics (all p < 0.05). The Pix-2-Pix GAN model for generating AC-PET demonstrates SUV metrics that highly correlate with original images. AI-generated PET images show clinical potential for reducing the need for CT scans for attenuation correction while preserving quantitative markers and image quality.

Efficient Generation of Conformer Ensembles Using Internal Coordinates and a Generative Directional Graph Convolution Neural Network.

We present a neural-network-based high-throughput molecular conformer-generation algorithm. A chemical graph-convolutional network is trained to predict low-energy conformers in internal coordinate representation (bond lengths, bond, and torsion angles), starting from two-dimensional (2D) chemical topology. Generative neural network (NN) architecture performs denoising from torsion space, producing conformer ensembles with populations that are well correlated with torsion energy profiles. Short force-field-based energy minimization is applied to refine final conformers. All computation-intensive stages of the algorithm are GPU-optimized. The procedure (termed GINGER) is benchmarked on a commonly used test set of bioactive three-dimensional (3D) conformers from the PDB. We demonstrate highly competitive results in conformer recovery and throughput rates suitable for giga-scale compound library processing. A web server that allows interactive conformer ensemble generation by GINGER and their viewing is made freely available at https://www.molsoft.com/gingerdemo.html.

A novel approach for improving open scene text translation with modified GAN

AbstractText, as a vital tool for communication, is playing an imperative role in modern society. Precise high-level text translation systems are essential requirements in a wide range of real-world applications, such as robot navigation, industrial automation, image search, and instant translation. Regardless of improved research, a series of grand challenges may still become upon when translating text automatically in the real-world from open scene images. The difficulties mainly stem from multiplicity and inconsistency of text in open scenes, complication and obstruction of backgrounds, and deficient imaging conditions in uncontrolled circumstances for open scene images. The existing deep learning-based text translation systems do not eliminate the text for translation, and these applications just replace text on the reconstructed scene. To address the abovementioned shortcomings, this study proposed a novel approach for open scene text translation. Our system consists of five modules including scene text detection, text recognition, text elimination, text translation, and text insertion along with scene reconstruction. The novelty presented by our model lies in the idea of first eliminating the text from the open scene for accurate translation and then reconstructs the translated text on the image for its proper alignment. We specifically modified the existing generative adversarial network (GAN) architecture for improved performance of text elimination by introducing a novel strategy of text and scene concatenation to reduce the overall loss function. For this purpose, we created a synthetic dataset to train our GAN for text elimination module. Experiments on various standard text translation systems demonstrate that our integrated system is able to outperform state-of-the-art approaches in terms of result quality. We have achieved 90.87% of precision, 83.66% of recall, 87.116% of F1-score, and reduced both losses ($$l_1$$ l 1 and $$l_2$$ l 2 ) up to 50% which is remarkable upon state-of-the-art translation systems.

Audio-deepfake detection: Adversarial attacks and countermeasures

Audio has always been a powerful resource for biometric authentication: thus, numerous AI-based audio authentication systems (classifiers) have been proposed. While these classifiers are effective in identifying legitimate human-generated input their security, to the best of our knowledge, has not been explored thoroughly when confronted with advanced attacks that leverage AI-generated deepfake audio. This issue presents a serious concern regarding the security of these classifiers because, e.g., samples generated using adversarial attacks might fool such classifiers, resulting in incorrect classification. In this study, we prove the point: we demonstrate that state-of-the-art audio deepfake classifiers are vulnerable to adversarial attacks. In particular, we design two adversarial attacks on a state-of-the-art audio-deepfake classifier, i.e., the Deep4SNet classification model, which achieves 98.5% accuracy in detecting fake audio samples. The designed adversarial attacks11The code of the attacks will be released open-source in the camera ready. leverage a generative adversarial network architecture and reduce the detector’s accuracy to nearly 0%. In particular, under graybox attack scenarios, we demonstrate that when starting from random noise, we can reduce the accuracy of the state-of-the-art detector from 98.5% to only 0.08%. To mitigate the effect of adversarial attacks on audio-deepfake detectors, we propose a highly generalizable, lightweight, simple, and effective add-on defense mechanism that can be implemented in any audio-deepfake detector. Finally, we discuss promising research directions.

RDGT-GAN: Robust Distribution Generalization of Transformers for COVID-19 Fake News Detection

Social media platforms have become a vital source of information during the outbreak of the pandemic (COVID-19). The phenomena of fake information or news spread through social media have become increasingly prevalent and a powerful tool for information proliferation. Detecting fake news is crucial for the betterment of society. Existing fake news detection models focus on increasing the performance which leads to overfitting and lag generalizability. Hence, these models require training for various datasets of the same domain with significant variations in the distribution. In our work, we have addressed this overfitting issue by designing a robust distribution generalization of transformers-based generative adversarial network (RDGT-GAN) architecture, which can generalize the model for COVID-19 fake news datasets with different distributions without retraining. Based on our experimental findings, it is evident that the proposed model outperforms the current state-of-the-art (SOTA) models in terms of performance.

Abstract 3517: A deep learning-based virtual staining system with multimodal-attention for precision medicine

Abstract Purpose: This study introduces a novel multimodal-attention-based virtual mIF staining (MAS) system designed for efficient, reliable virtual mIF staining from label-free fluorescence images. Our goal was to overcome the performance bottleneck and time-cost limitations associated with traditional mIF techniques and thereby enhance their clinical utility. Method and Materials: Our approach involved developing a cutting-edge MAS model. This model is built upon a sophisticated end-to-end generative convolutional neural network (CNN) architecture. It ingeniously leverages autofluorescence and 4',6-diamidino-2-phenylindole (DAPI) slides as inputs to generate mIF images. To achieve this, we equipped the model with feature extractors enhanced by pretrained masked auto-encoders (MAEs) and a self-attention combination strategy. These components worked harmoniously to extract antigen-label-related features and precisely locate specific cells within the images. Results: In our comprehensive study, we engaged 94 gastric cancer patients, utilizing the MAS system for automated virtual mIF staining of seven biomarkers, namely CD3, CD20, FOXP3, PD1, CD8, CD163, and PDL1, in both cancerous and non-cancerous tissues. Importantly, the MAS-produced virtual mIF stains matched the quality of traditional manual stains. Furthermore, we validated the prognostic accuracy for gastric cancer using these virtual mIF images, demonstrating their ability to provide clinical information that is as reliable and valuable as that obtained from manually stained mIF images. Conclusions: Our study identifies the MAS system as an important advancement in mIF staining, enhancing personalized medicine with its efficiency and quality. This tool holds significant potential to enable personalized medicine efficiently and cost-effectively, streamlining the process of diagnosing diseases, making prognoses, and developing treatment plans. Quantitative Comparison of Ablation Experiments for MAS system in predicting multiple biomarkers Biomarkers Index U-Net (Avg±Std) ReU-Net (Avg±Std) Att-ReU-Net (Avg±Std) MAS (Avg±Std) CD3 PSNR 27.053±4.10 27.419±4.15 28.073±4.00 28.546±4.40 CD3 SSIM 0.733±0.07 0.724±0.07 0.728±0.05 0.742±0.07 CD20 PSNR 28.835±4.66 29.244±4.76 28.672±4.52 29.243±4.99 CD20 SSIM 0.921±0.058 0.921±0.057 0.915±0.051 0.923±0.051 FOXP3 PSNR 31.769±3.29 31.838±3.10 31.098±3.20 31.862±3.11 FOXP3 SSIM 0.613±0.11 0.618±0.11 0.617±0.112 0.617±0.12 PD1 PSNR 27.908±4.47 27.950±4.43 30.283±4.37 31.717±4.17 PD1 SSIM 0.654±0.13 0.659±0.13 0.694±0.12 0.726±0.11 Citation Format: Zixia Zhou, Yuming Jiang, Ruijiang Li, Lei Xing. A deep learning-based virtual staining system with multimodal-attention for precision medicine [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 3517.

Abstract 6185: Generative AI for virtual HE-staining of whole slide images of unstained breast cancer tissue

Abstract Introduction Hematoxylin and eosin (HE) is the standard stain used in histology to make tissue visible to the human eye by highlighting certain cellular and tissue structures, and it is a technique that is widely used in the diagnosis of cancer and other pathologies. Chemical staining, however, is irreversible, making the tissue unusable for subsequent measurements, such as spatial transcriptomics. Here we utilize generative AI method based on pix2pix image-to-image translation to generate virtual HE-staining for whole slide images (WSIs) acquired of unstained tissue with brightfield microscopy and perform a thorough histological evaluation for the feasibility in breast cancer diagnostics. Materials and Methods We optimized sample preparation and imaging setup for virtual staining purposes, and developed a custom generative adversarial network architecture for learning the virtual staining from paires samples of unstained and H&amp;E stained tissue. Here, we focus on the utility of virtual staining in breast cancer diagnostics, using a set of breast cancer samples for acquiring whole slide images from unstained tissue before H&amp;E staining and from reference H&amp;E stained tissue after chemical staining. Hold-out set of sample pairs are left for validation, allowing us to evaluate the virtual staining performance for a vast array of tissue components and to examine the potential shortcomings in staining reproduction. We use a comprehensive set of quantitative metrics both on pixel and object level to evaluate virtual staining quality. In addition, we perform thorough visual evaluation of histological feasibility by histology experts to examine the computational staining. Results We demonstrate that by careful optimization of both sample preparation and imaging workflow, as well as the computational methods, generative adversarial networks can be used for virtual staining of whole slide images of breast cancer tissue acquired from unstained tissue using regular bright field microscopy. We analyzed the virtual staining performance quantitatively and visually, and highlight the potential of the method through successful cases and demonstrate the applicability in breast cancer diagnostics, as well as discuss the challenges and shortcomings of virtual staining for clinical samples. Conclusions We demonstrate the potential of virtual HE staining for clinical histopathology of breast cancer tissue. Notably, our virtual staining based on generative AI shows promise towards more sustainable and streamlined sample processing and staining process in digital pathology. Citation Format: Leena Latonen, Umair Khan, Sonja Koivukoski, Johan Hartman, Pekka Ruusuvuori. Generative AI for virtual HE-staining of whole slide images of unstained breast cancer tissue [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 6185.

Approach to Perforation Interval Selection Uses AI-Assisted Image Interpretation

_ This article, written by JPT Technology Editor Chris Carpenter, contains highlights of paper SPE 216856, “ML-Driven Integrated Approach for Perforation Interval Selection Based on Advanced Borehole-Image AI-Assisted Interpretation,” by Alexander Petrov, SPE, Mounir Belouahchia, SPE, and Abdelwahab Noufal, SPE, ADNOC. The paper has not been peer reviewed. _ In the complete paper, the authors propose an artificial-intelligence (AI)-assisted work flow that uses machine-learning (ML) techniques to identify sweet spots in carbonate reservoirs. This process involves annotation of geologic features using a well database, with supervision from subject-matter experts (SMEs). The resulting ML model is tested on new wells and can identify pay zones, perforation intervals, and stress analysis. The models successfully detect fractures, breakouts, bedding planes, vugs, and slippage passages with pixel-level precision, reducing borehole-image (BHI) analysis time. BHI Interpretation and Preprocessing The use of BHIs requires manual interpretation and data identification, heavily relying on the expertise and time of SMEs. A widely adopted approach to address this challenge is the use of supervised computer-vision algorithms, a subfield of AI. These algorithms optimize the task function or model based on examples they have learned from data during training. However, when applied to BHIs, certain ML challenges must be addressed, including the following: - Detecting features in wells from different reservoirs using a model trained on wells from one reservoir can be highly challenging because reservoirs may exhibit distinct geological characteristics. - The handling of parts of BHIs with missing data, depicted by vertically slanted white strips, poses considerable difficulty. Therefore, the authors created a deep-learning approach based on a generative adversarial network architecture to fill the gaps automatically (Fig. 1). - The labels provided by geologists often do not have pixelwise precision, causing the machine to become confused while trying to learn inconsistent patterns. The authors use a convolutional neural network (CNN) to compute a probability map for pixels belonging to specific classes. In this application, a class is defined as any of the heterogeneities in the BHI; however, this method is applicable to any type of heterogeneities in an image. After training, the CNN module provides the optimal probability for each pixel in the image. To classify regions in the BHI based on heterogeneities, a class-specific threshold is established. Pixels with values above the thresholds are assigned to the corresponding class, while those below the thresholds are assigned to the background. BHI-Derived Porosity A new approach for borehole-derived porosity was developed in-house to overcome the limitations of existing techniques widely used in the industry. This approach capitalizes on BHIs for multiple analyses, including structural dip assessment, fault and fracture identification, and determination of minimum and maximum horizontal stress orientation. However, its primary strength lies in quantifying the fraction of secondary porosity in heterogeneous or dual-porosity carbonate formations. The authors have devised a novel method that uses borehole electrical images to conduct porosity and image connectivity analysis. By implementing this technique, essential information can be extracted regarding the spatial distribution of porosity and the extent of secondary porosity fraction.

Generating Ultrasonic Foliage Echoes with Variational Autoencoders

Navigation through dense foliage presents a fundamental challenge to autonomous systems, and achieving a performance level similar to echolocating bats could have important applications in areas such as forestry and farming. However, the clutter echoes originating from such environments have been difficult to analyze. To study the problem of sonar‐based navigation in dense foliage in simulation, an artificial generation system for leaf impulse responses (IRs) based on variational auto‐encoders is proposed. The system is to aid the construction of artificial foliage echo environments. A dataset of leaf echoes was collected in an anechoic chamber and convolved with the original signal to estimate the IR of each leaf. A modified version of the conditional variational autoencoder ‐ generative adversarial network (cVAE‐GAN) architecture was trained successfully on this dataset to produce a generative model that was conditional on leaf viewing angles, size, and species. The IRs generated by the model capture quantitative and qualitative similarity to the measured IRs. It surpasses the previous state of the art foliage echo model based on reflecting disks. The model's computational efficiency and its success suggest its potential use for simulating large environments of foliage to study bat biosonar and aid in engineering biomimetic sonar devices.

Low Latency 5G IP Transmission Backhaul Network Architecture: A Techno-Economic Analysis

The steeply rising demand for mobile data drives the investigation of the transmission backhaul network architecture and cost for the fifth generation (5G) of mobile technologies. The proposed backhaul architecture will facilitate high throughput, low latency, scalability, low cost of ownership, and high capacity backhaul for 5G mobile technologies. This paper presents a transmission backhaul network architecture for 5G technology; the proposed internet protocol (IP) transmission backhauling architecture includes the data center, core network, distribution network, and access or IP random access network. A mathematical model for the data center IP core network, IP distributed network, and the IP access network for capital expenditure (Capex), operational expenditure (Opex), and the total cost of ownership (TCO) are presented, as well as a mathematical model for the entire backhauling architecture. The result shows that the increase in IP sites is positively proportional to the Capex and negatively proportional to the Opex. The selectivity analysis shows that the increase in bandwidth is directly proportional to the Capex, Opex, and TCO in the IP core network. The increase in data centers is directly proportional to the Capex, Opex, and TCO of the entire backhauling architecture.

High-precision Drosophila heart segmentation and dynamic cardiac parameter measurement for optogenetics-OCT-based cardiac function research.

Drosophila model has been widely used to study cardiac functions, especially combined with optogenetics and optical coherence tomography (OCT) that can continuously acquire mass cross-sectional images of the Drosophila heart in vivo over time. It's urgent to quickly and accurately obtain dynamic Drosophila cardiac parameters such as heartbeat rate for cardiac function quantitative analysis through these mass cross-sectional images of the Drosophila heart. Here we present a deep-learning method that integrates U-Net and generative adversarial network architectures while incorporating residually connected convolutions for high-precision OCT image segmentation of Drosophila heart and dynamic cardiac parameter measurements for optogenetics-OCT-based cardiac function research. We compared our proposed network with the previous approaches and our segmentation results achieved the accuracy of intersection over union and Dice similarity coefficient higher than 98%, which can be used to better quantify dynamic heart parameters and improve the efficiency of Drosophila-model-based cardiac research via the optogenetics-OCT-based platform.

PlantBiCNet: A new paradigm in plant science with bi-directional cascade neural network for detection and counting

Deep learning is increasingly popular for precise plant detection and counting in computer vision applications. Despite the rising cross-disciplinary research in this field, an important issue has been neglected: due to the inherent constraints imposed by the natural growth patterns of plants, the costly and time-consuming process of collecting plant image data, resulting in a scarcity of real-world available data. Consequently, researchers are often constrained to exploring large-scale data representations or limited applications. Currently, still no generalized network architecture excels in demanding environments. To address this, we introduce two new bi-directional cascade neural network paradigms, PlantBiCNet and its lightweight variant, PlantBiCNet-Lite. Unlike previous research, we place greater emphasis on the network decoding process itself rather than the encoder. Our bi-directional cascade and weight fusion decoding approach maximizes high-level semantics and low-level spatial data, enhancing the localization signal in a way that respects gradient diversity. We evaluate PlantBiCNet and PlantBiCNet-Lite on five public plant detection and counting datasets and a novel plant dataset. The average precision AP50 on these six small-scale datasets reached 0.905, with a high coefficient of determination R2 of 0.935, while the reported frame rate was 196 FPS. These results significantly outperform state-of-the-art computer vision methods. Moreover, we found that the introduction of the Dropout mechanism markedly enhances the model's performance and generalization ability. Further analysis supports our experiments and provides valuable insights into network optimization. Overall, our organization's open datasets and network modeling details provide a solid foundation for future research in the field of plant science.

Optimal Tilt-Wing eVTOL Takeoff Trajectory Prediction Using Regression Generative Adversarial Networks

Electric vertical takeoff and landing (eVTOL) aircraft have attracted tremendous attention nowadays due to their flexible maneuverability, precise control, cost efficiency, and low noise. The optimal takeoff trajectory design is a key component of cost-effective and passenger-friendly eVTOL systems. However, conventional design optimization is typically computationally prohibitive due to the adoption of high-fidelity simulation models in an iterative manner. Machine learning (ML) allows rapid decision making; however, new ML surrogate modeling architectures and strategies are still desired to address large-scale problems. Therefore, we showcase a novel regression generative adversarial network (regGAN) surrogate for fast interactive optimal takeoff trajectory predictions of eVTOL aircraft. The regGAN leverages generative adversarial network architectures for regression tasks with a combined loss function of a mean squared error (MSE) loss and an adversarial binary cross-entropy (BC) loss. Moreover, we introduce a surrogate-based inverse mapping concept into eVTOL optimal trajectory designs for the first time. In particular, an inverse-mapping surrogate takes design requirements (including design constraints and flight condition parameters) as input and directly predicts optimal trajectory designs, with no need to run design optimizations once trained. We demonstrated the regGAN on optimal takeoff trajectory designs for the Airbus A3 Vahana. The results revealed that regGAN outperformed reference surrogate strategies, including multi-output Gaussian processes and conditional generative adversarial network surrogates, by matching simulation-based ground truth with 99.6% relative testing accuracy using 1000 training samples. A parametric study showed that a regGAN surrogate with an MSE weight of one and a BC weight of 0.01 consistently achieved over 99.5% accuracy (denoting negligible predictive errors) using 400 training samples, while other regGAN models require at least 800 samples.

Attenuation Correction of Long Axial Field-of-View Positron Emission Tomography Using Synthetic Computed Tomography Derived from the Emission Data: Application to Low-Count Studies and Multiple Tracers.

Recent advancements in PET/CT, including the emergence of long axial field-of-view (LAFOV) PET/CT scanners, have increased PET sensitivity substantially. Consequently, there has been a significant reduction in the required tracer activity, shifting the primary source of patient radiation dose exposure to the attenuation correction (AC) CT scan during PET imaging. This study proposes a parameter-transferred conditional generative adversarial network (PT-cGAN) architecture to generate synthetic CT (sCT) images from non-attenuation corrected (NAC) PET images, with separate networks for [18F]FDG and [15O]H2O tracers. The study includes a total of 1018 subjects (n = 972 [18F]FDG, n = 46 [15O]H2O). Testing was performed on the LAFOV scanner for both datasets. Qualitative analysis found no differences in image quality in 30 out of 36 cases in FDG patients, with minor insignificant differences in the remaining 6 cases. Reduced artifacts due to motion between NAC PET and CT were found. For the selected organs, a mean average error of 0.45% was found for the FDG cohort, and that of 3.12% was found for the H2O cohort. Simulated low-count images were included in testing, which demonstrated good performance down to 45 s scans. These findings show that the AC of total-body PET is feasible across tracers and in low-count studies and might reduce the artifacts due to motion and metal implants.

Lensless Image Restoration Based on Multi-Stage Deep Neural Networks and Pix2pix Architecture

Lensless imaging represents a significant advancement in imaging technology, offering unique benefits over traditional optical systems due to its compact form factor, ideal for applications within the Internet of Things (IoT) ecosystem. Despite its potential, the intensive computational requirements of current lensless imaging reconstruction algorithms pose a challenge, often exceeding the resource constraints typical of IoT devices. To meet this challenge, a novel approach is introduced, merging multi-level image restoration with the pix2pix generative adversarial network architecture within the lensless imaging sphere. Building on the foundation provided by U-Net, a Multi-level Attention-based Lensless Image Restoration Network (MARN) is introduced to further augment the generator’s capabilities. In this methodology, images reconstructed through Tikhonov regularization are perceived as degraded images, forming the foundation for further refinement via the Pix2pix network. This process is enhanced by incorporating an attention-focused mechanism in the encoder--decoder structure and by implementing stage-wise supervised training within the deep convolutional network, contributing markedly to the improvement of the final image quality. Through detailed comparative evaluations, the superiority of the introduced method is affirmed, outperforming existing techniques and underscoring its suitability for addressing the computational challenges in lensless imaging within IoT environments. This method can produce excellent lensless image reconstructions when sufficient computational resources are available, and it consistently delivers optimal results across varying computational resource constraints. This algorithm enhances the applicability of lensless imaging in applications such as the Internet of Things, providing higher-quality image acquisition and processing capabilities for these domains.

Blind Attention Geometric Restraint Neural Network for Single Image Dynamic/Defocus Deblurring.

Based on the information loss analysis of the blur accumulation model, a novel single-image deblurring method is proposed. We apply the recurrent neural network architecture to capture the attention perception map and the generative adversarial network (GAN) architecture to yield the deblurring image. Considering that the attention mechanism has to make hard decisions about specific parts of the input image to be focused on since blurry regions are not given, we propose a new adaptive attention disentanglement model based on the variation blind source separation, which provides the global geometric restraint to reduce the large solution space, so that the generator can realistically restore details on blurry regions, and the discriminator can accurately assess the content consistency of the restored regions. Since we combine blind source separation, attention geometric restraint with GANs, we name the proposed method BAGdeblur. Extensive evaluations on quantitative and qualitative experiments show that the proposed method achieves the state-of-the-art performance on both synthetic datasets and real-world blurry images.

Real-Time Defect Detection Model in Industrial Environment Based on Lightweight Deep Learning Network

Surface defect detection in industrial environments is crucial for quality management and has significant research value. General detection networks, such as the YOLO series, have proven effective in various dataset detections. However, due to the complex and varied surface defects of industrial products, many defects occupy a small proportion of the surface and fall into the category of typical small target detection problems. Moreover, the complexity of general detection network architectures relies on high-tech hardware, making it difficult to deploy on devices without GPUs or on edge computing and mobile devices. To meet the practical needs of industrial product defect inspection applications, this paper proposes a lightweight network specifically designed for defect detection in industrial fields. This network is composed of four parts: a backbone network, a multiscale feature aggregation network, a residual enhancement network, and an attention enhancement network. The network includes a backbone network that integrates attention layers for feature extraction, a multiscale feature aggregation network for semantic information, a residual enhancement network for spatial focus, and an attention enhancement network for global–local feature interaction. These components enhance detection performance for diverse defects while maintaining low hardware requirements. Experimental results show that this network outperforms the latest and most popular YOLOv5n and YOLOv8n models in the five indicators P, R, F1, mAP@.5, and GFLOPS when used on four public datasets. It even approaches or surpasses the YOLOv8s and YOLOv5s models with several times the GFLOPS computation. It balances the requirements of lightweight real-time and accuracy in the scenario of industrial product surface defect detection.