Synthetic Image Generation Research Articles

Personnel Detection in Dark Aquatic Environments Based on Infrared Thermal Imaging Technology and an Improved YOLOv5s Model.

This study presents a novel method for the nighttime detection of waterborne individuals using an enhanced YOLOv5s algorithm tailored for infrared thermal imaging. To address the unique challenges of nighttime water rescue operations, we have constructed a specialized dataset comprising 5736 thermal images collected from diverse aquatic environments. This dataset was further expanded through synthetic image generation using CycleGAN and a newly developed color gamut transformation technique, which significantly improves the data variance and model training effectiveness. Furthermore, we integrated the Convolutional Block Attention Module (CBAM) at the end of the last encoder's feedforward network. This integration maximizes the utilization of channel and spatial information to capture more intricate details in the feature maps. To decrease the computational demands of the network while maintaining model accuracy, Ghost convolution was employed, thereby boosting the inference speed as much as possible. Additionally, we applied hyperparameter evolution to refine the training parameters. The improved algorithm achieved an average detection accuracy of 85.49% on our proprietary dataset, significantly outperforming its predecessor, with a prediction speed of 23.51 FPS. The experimental outcomes demonstrate the proposed solution's high recognition capabilities and robustness, fulfilling the demands of intelligent lifesaving missions.

Generative Model-Driven Synthetic Training Image Generation: An Approach to Cognition in Railway Defect Detection

Generative Model-Driven Synthetic Training Image Generation: An Approach to Cognition in Railway Defect Detection

BraNet: a mobil application for breast image classification based on deep learning algorithms

Mobile health apps are widely used for breast cancer detection using artificial intelligence algorithms, providing radiologists with second opinions and reducing false diagnoses. This study aims to develop an open-source mobile app named “BraNet” for 2D breast imaging segmentation and classification using deep learning algorithms. During the phase off-line, an SNGAN model was previously trained for synthetic image generation, and subsequently, these images were used to pre-trained SAM and ResNet18 segmentation and classification models. During phase online, the BraNet app was developed using the react native framework, offering a modular deep-learning pipeline for mammography (DM) and ultrasound (US) breast imaging classification. This application operates on a client–server architecture and was implemented in Python for iOS and Android devices. Then, two diagnostic radiologists were given a reading test of 290 total original RoI images to assign the perceived breast tissue type. The reader’s agreement was assessed using the kappa coefficient. The BraNet App Mobil exhibited the highest accuracy in benign and malignant US images (94.7%/93.6%) classification compared to DM during training I (80.9%/76.9%) and training II (73.7/72.3%). The information contrasts with radiological experts’ accuracy, with DM classification being 29%, concerning US 70% for both readers, because they achieved a higher accuracy in US ROI classification than DM images. The kappa value indicates a fair agreement (0.3) for DM images and moderate agreement (0.4) for US images in both readers. It means that not only the amount of data is essential in training deep learning algorithms. Also, it is vital to consider the variety of abnormalities, especially in the mammography data, where several BI-RADS categories are present (microcalcifications, nodules, mass, asymmetry, and dense breasts) and can affect the API accuracy model.Graphical abstract

ECG-Image-Kit: a synthetic image generation toolbox to facilitate deep learning-based electrocardiogram digitization

Objective. Cardiovascular diseases are a major cause of mortality globally, and electrocardiograms (ECGs) are crucial for diagnosing them. Traditionally, ECGs are stored in printed formats. However, these printouts, even when scanned, are incompatible with advanced ECG diagnosis software that require time-series data. Digitizing ECG images is vital for training machine learning models in ECG diagnosis, leveraging the extensive global archives collected over decades. Deep learning models for image processing are promising in this regard, although the lack of clinical ECG archives with reference time-series data is challenging. Data augmentation techniques using realistic generative data models provide a solution. Approach. We introduce ECG-Image-Kit, an open-source toolbox for generating synthetic multi-lead ECG images with realistic artifacts from time-series data, aimed at automating the conversion of scanned ECG images to ECG data points. The tool synthesizes ECG images from real time-series data, applying distortions like text artifacts, wrinkles, and creases on a standard ECG paper background. Main results. As a case study, we used ECG-Image-Kit to create a dataset of 21 801 ECG images from the PhysioNet QT database. We developed and trained a combination of a traditional computer vision and deep neural network model on this dataset to convert synthetic images into time-series data for evaluation. We assessed digitization quality by calculating the signal-to-noise ratio and compared clinical parameters like QRS width, RR, and QT intervals recovered from this pipeline, with the ground truth extracted from ECG time-series. The results show that this deep learning pipeline accurately digitizes paper ECGs, maintaining clinical parameters, and highlights a generative approach to digitization. Significance. The toolbox has broad applications, including model development for ECG image digitization and classification. The toolbox currently supports data augmentation for the 2024 PhysioNet Challenge, focusing on digitizing and classifying paper ECG images.

W-DRAG: A joint framework of WGAN with data random augmentation optimized for generative networks for bone marrow edema detection in dual energy CT

Dual-energy computed tomography (CT) is an excellent substitute for identifying bone marrow edema in magnetic resonance imaging. However, it is rarely used in practice owing to its low contrast. To overcome this problem, we constructed a framework based on deep learning techniques to screen for diseases using axial bone images and to identify the local positions of bone lesions. To address the limited availability of labeled samples, we developed a new generative adversarial network (GAN) that extends expressions beyond conventional augmentation (CA) methods based on geometric transformations. We theoretically and experimentally determined that combining the concepts of data augmentation optimized for GAN training (DAG) and Wasserstein GAN yields a considerably stable generation of synthetic images and effectively aligns their distribution with that of real images, thereby achieving a high degree of similarity. The classification model was trained using real and synthetic samples. Consequently, the GAN technique used in the diagnostic test had an improved F1 score of approximately 7.8% compared with CA. The final F1 score was 80.24%, and the recall and precision were 84.3% and 88.7%, respectively. The results obtained using the augmented samples outperformed those obtained using pure real samples without augmentation. In addition, we adopted explainable AI techniques that leverage a class activation map (CAM) and principal component analysis to facilitate visual analysis of the network’s results. The framework was designed to suggest an attention map and scattering plot to visually explain the disease predictions of the network.

Synthetic Low-Energy Monochromatic Image Generation in Single-Energy Computed Tomography System Using a Transformer-Based Deep Learning Model.

While dual-energy computed tomography (DECT) technology introduces energy-specific information in clinical practice, single-energy CT (SECT) is predominantly used, limiting the number of people who can benefit from DECT. This study proposed a novel method to generate synthetic low-energy virtual monochromatic images at 50keV (sVMI50keV) from SECT images using a transformer-based deep learning model, SwinUNETR. Data were obtained from 85 patients who underwent head and neck radiotherapy. Among these, the model was built using data from 70 patients for whom only DECT images were available. The remaining 15 patients, for whom both DECT and SECT images were available, were used to predict from the actual SECT images. We used the SwinUNETR model to generate sVMI50keV. The image quality was evaluated, and the results were compared with those of the convolutional neural network-based model, Unet. The mean absolute errors from the true VMI50keV were 36.5 ± 4.9 and 33.0 ± 4.4 Hounsfield units for Unet and SwinUNETR, respectively. SwinUNETR yielded smaller errors in tissue attenuation values compared with those of Unet. The contrast changes in sVMI50keV generated by SwinUNETR from SECT were closer to those of DECT-derived VMI50keV than the contrast changes in Unet-generated sVMI50keV. This study demonstrated the potential of transformer-based models for generating synthetic low-energy VMIs from SECT images, thereby improving the image quality of head and neck cancer imaging. It provides a practical and feasible solution to obtain low-energy VMIs from SECT data that can benefit a large number of facilities and patients without access to DECT technology.

Generative adversarial networks for anonymous acneic face dataset generation.

It is well known that the performance of any classification model is effective if the dataset used for the training process and the test process satisfy some specific requirements. In other words, the more the dataset size is large, balanced, and representative, the more one can trust the proposed model's effectiveness and, consequently, the obtained results. Unfortunately, large-size anonymous datasets are generally not publicly available in biomedical applications, especially those dealing with pathological human face images. This concern makes using deep-learning-based approaches challenging to deploy and difficult to reproduce or verify some published results. In this paper, we propose an efficient method to generate a realistic anonymous synthetic dataset of human faces, focusing on attributes related to acne disorders at three distinct levels of severity (Mild, Moderate, and Severe). Notably, our approach initiates from a small dataset of facial acne images, leveraging generative techniques to augment and diversify the dataset, ensuring comprehensive coverage of acne severity levels while maintaining anonymity and realism in the synthetic data. Therefore, a specific hierarchy StyleGAN-based algorithm trained at distinct levels is considered. Moreover, the utilization of generative adversarial networks for augmentation offers a means to circumvent potential privacy or legal concerns associated with acquiring medical datasets. This is attributed to the synthetic nature of the generated data, where no actual subjects are present, thereby ensuring compliance with privacy regulations and legal considerations. To evaluate the performance of the proposed scheme, we consider a CNN-based classification system, trained using the generated synthetic acneic face images and tested using authentic face images. Consequently, we show that an accuracy of 97.6% is achieved using InceptionResNetv2. As a result, this work allows the scientific community to employ the generated synthetic dataset for any data processing application without restrictions on legal or ethical concerns. Moreover, this approach can also be extended to other applications requiring the generation of synthetic medical images.

Leveraging Quantum computing for synthetic image generation and recognition with Generative Adversarial Networks and Convolutional Neural Networks

Leveraging Quantum computing for synthetic image generation and recognition with Generative Adversarial Networks and Convolutional Neural Networks

RSUIGM: Realistic Synthetic Underwater Image Generation with Image Formation Model

In this paper, we propose to synthesize realistic underwater images with a novel image formation model, considering both downwelling depth and line of sight (LOS) distance as cue and call it as Realistic Synthetic Underwater Image Generation Model, RSUIGM. The light interaction in the ocean is a complex process and demands specific modeling of direct and backscattering phenomenon to capture the degradations. Most of the image formation models rely on complex radiative transfer models and in-situ measurements for synthesizing and restoration of underwater images. Typical image formation models consider only line of sight distance z and ignore downwelling depth d in the estimation of effect of direct light scattering. We derive the dependencies of downwelling irradiance in direct light estimation for generation of synthetic underwater images unlike state-of-the-art image formation models. We propose to incorporate the derived downwelling irradiance in estimation of direct light scattering for modeling the image formation process and generate realistic synthetic underwater images with the proposed RSUIGM, and name it as RSUIGM dataset . We demonstrate the effectiveness of the proposed RSUIGM by using RSUIGM dataset in training deep learning based restoration methods. We compare the quality of restored images with state-of-the-art methods using benchmark real underwater image datasets and achieve improved results. In addition, we validate the distribution of realistic synthetic underwater images versus real underwater images both qualitatively and quantitatively. The proposed RSUIGM dataset is available here.

Robust deep learning for eye fundus images: Bridging real and synthetic data for enhancing generalization

Deep learning applications for assessing medical images are limited because the datasets are often small and imbalanced. The use of synthetic data has been proposed in the literature, but neither a robust comparison of the different methods nor generalizability has been reported. Our approach integrates a retinal image quality assessment model and StyleGAN2 architecture to enhance Age-related Macular Degeneration (AMD) detection capabilities and improve generalizability. This work compares ten different Generative Adversarial Network (GAN) architectures to generate synthetic eye-fundus images with and without AMD. We combined subsets of three public databases (iChallenge-AMD, ODIR-2019, and RIADD) to form a single training and test set. We employed the STARE dataset for external validation, ensuring a comprehensive assessment of the proposed approach. The results show that StyleGAN2 reached the lowest Fréchet Inception Distance (166.17), and clinicians could not accurately differentiate between real and synthetic images. ResNet-18 architecture obtained the best performance with 85% accuracy and outperformed the two human experts (80% and 75%) in detecting AMD fundus images. The accuracy rates were 82.8% for the test set and 81.3% for the STARE dataset, demonstrating the model’s generalizability. The proposed methodology for synthetic medical image generation has been validated for robustness and accuracy, with free access to its code for further research and development in this field.

Elevating semantic segmentation: a conditional generative adversarial network (CGAN)-based synthetic scene image generation for enhanced precision

Elevating semantic segmentation: a conditional generative adversarial network (CGAN)-based synthetic scene image generation for enhanced precision

Optimized signal of calcifications in wide-angle digital breast tomosynthesis: a virtual imaging trial.

Evaluate microcalcification detectability in digital breast tomosynthesis (DBT) and synthetic 2D mammography (SM) for different acquisition setups using a virtual imaging trial (VIT) approach. Medio-lateral oblique (MLO) DBT acquisitions on eight patients were performed at twice the automatic exposure controlled (AEC) dose. The noise was added to the projections to simulate a given dose trajectory. Virtual microcalcification models were added to a given projection set using an in-house VIT framework. Three setups were evaluated: (1) standard acquisition with 25 projections at AEC dose, (2) 25 projections with a convex dose distribution, and (3) sparse setup with 13 projections, every second one over the angular range. The total scan dose and angular range remained constant. DBT volume reconstruction and synthetic mammography image generation were performed using a Siemens prototype algorithm. Lesion detectability was assessed through a Jackknife-alternative free-response receiver operating characteristic (JAFROC) study with six observers. For DBT, the area under the curve (AUC) was 0.97 ± 0.01 for the standard, 0.95 ± 0.02 for the convex, and 0.89 ± 0.03 for the sparse setup. There was no significant difference between standard and convex dose distributions (p = 0.309). Sparse projections significantly reduced detectability (p = 0.001). Synthetic images had a higher AUC with the convex setup, though not significantly (p = 0.435). DBT required four times more reading time than synthetic mammography. A convex setup did not significantly improve detectability in DBT compared to the standard setup. Synthetic images exhibited a non-significant increase in detectability with the convex setup. Sparse setup significantly reduced detectability in both DBT and synthetic mammography. This virtual imaging trial study allowed the design and efficient testing of different dose distribution trajectories with real mammography images, using a dose-neutral protocol. • In DBT, a convex dose distribution did not increase the detectability of microcalcifications compared to the current standard setup but increased detectability for the SM images. • A sparse setup decreased microcalcification detectability in both DBT and SM images compared to the convex and current clinical setups. • Optimal microcalcification cluster detection in the system studied was achieved using either the standard or convex dose setting, with the default number of projections.

MAM-E: Mammographic Synthetic Image Generation with Diffusion Models.

Generative models are used as an alternative data augmentation technique to alleviate the data scarcity problem faced in the medical imaging field. Diffusion models have gathered special attention due to their innovative generation approach, the high quality of the generated images, and their relatively less complex training process compared with Generative Adversarial Networks. Still, the implementation of such models in the medical domain remains at an early stage. In this work, we propose exploring the use of diffusion models for the generation of high-quality, full-field digital mammograms using state-of-the-art conditional diffusion pipelines. Additionally, we propose using stable diffusion models for the inpainting of synthetic mass-like lesions on healthy mammograms. We introduce MAM-E, a pipeline of generative models for high-quality mammography synthesis controlled by a text prompt and capable of generating synthetic mass-like lesions on specific regions of the breast. Finally, we provide quantitative and qualitative assessment of the generated images and easy-to-use graphical user interfaces for mammography synthesis.

An End-to-End Deep Learning Framework for Predicting Hematoma Expansion in Hemorrhagic Stroke Patients from CT Images

Hematoma expansion (HE) occurs in 20% of patients with hemorrhagic stroke within 24 h of onset, and it is associated with a poorer patient outcome. From a clinical point of view, predicting HE from the initial patient computed tomography (CT) image is useful to improve therapeutic decisions and minimize prognosis errors. In this work, we propose an end-to-end deep learning framework for predicting the final hematoma expansion and its corresponding lesion mask. We also explore the problem of having limited data and propose to augment the available dataset with synthetic images. The obtained results show an improved HE prediction when incorporating the use of synthetic images into the model, with a mean Dice score of the HE growth area of 0.506 and an average prediction error in hematoma volume of −3.44 mL. The proposed approach achieved results in line with state-of-the-art methods with far fewer data by using synthetic image generation and without requiring the inclusion of patient clinical data.

A siamese-based verification system for open-set architecture attribution of synthetic images

Despite the wide variety of methods developed for synthetic image attribution, most of them can only attribute images generated by models or architectures included in the training set and do not work with unknown architectures, hindering their applicability in real-world scenarios. In this paper, we propose a verification framework that relies on a Siamese Network to address the problem of open-set attribution of synthetic images to the architecture that generated them. We consider two different settings. In the first setting, the system determines whether two images have been produced by the same generative architecture or not. In the second setting, the system verifies a claim about the architecture used to generate a synthetic image, utilizing one or multiple reference images generated by the claimed architecture. The main strength of the proposed system is its ability to operate in both closed and open-set scenarios so that the input images, either the query and reference images, can belong to the architectures considered during training or not. Experimental evaluations encompassing various generative architectures such as GANs, diffusion models, and transformers, focusing on synthetic face image generation, confirm the excellent performance of our method in both closed and open-set settings, as well as its strong generalization capabilities.

The Industrial Application of Artificial Intelligence-Based Optical Character Recognition in Modern Manufacturing Innovations

This paper presents the development of a comprehensive, on-site industrial Optical Character Recognition (OCR) system tailored for reading text on iron plates. Initially, the system utilizes a text region detection network to identify the text area, enabling camera adjustments along the x and y axes and zoom enhancements for clearer text imagery. Subsequently, the detected text region undergoes line-by-line division through a text segmentation network. Each line is then transformed into rectangular patches for character recognition by the text recognition network, comprising a vision-based text recognition model and a language network. The vision network performs preliminary recognition, followed by refinement through the language model. The OCR results are then converted into digital characters and recorded in the iron plate registration system. This paper’s contributions are threefold: (1) the design of a comprehensive, on-site industrial OCR system for autonomous registration of iron plates; (2) the development of a realistic synthetic image generation strategy and a robust data augmentation strategy to address data scarcity; and (3) demonstrated impressive experimental results, indicating potential for on-site industrial applications. The designed autonomous system enhances iron plate registration efficiency and significantly reduces factory time and labor costs.

The TNG50-SKIRT Atlas: Post-processing methodology and first data release

Galaxy morphology is a powerful diagnostic to assess the realism of cosmological hydrodynamical simulations. Determining the morphology of simulated galaxies requires the generation of synthetic images through 3D radiative transfer post-processing that properly accounts for different stellar populations and interstellar dust attenuation. We use the SKIRT code to generate the TNG50-SKIRT Atlas, a synthetic UV to near-infrared broadband image atlas for a complete stellar-mass selected sample of 1154 galaxies extracted from the TNG50 cosmological simulation at z = 0. The images have a high spatial resolution (100 pc) and a wide field of view (160 kpc). In addition to the dust-obscured images, we also release dust-free images and physical parameter property maps with matching characteristics. As a sanity check and preview application we discuss the UVJ diagram of the galaxy sample. We investigate the effect of dust attenuation on the UVJ diagram and find that it affects both the star-forming and the quiescent galaxy populations. The quiescent galaxy region is polluted by younger and star-forming highly inclined galaxies, while dust attenuation induces a separation in inclination of the star-forming galaxy population, with low-inclination galaxies remaining at the blue side of the diagram and high-inclination galaxies systematically moving towards the red side. This image atlas can be used for a variety of other applications, including galaxy morphology studies and the investigation of local scaling relations. We publicly release the images and parameter maps, and we invite the community to use them.

Метод синтеза данных для повышения эффективности обучения нейронных сетей

The article is devoted to the formation of the initial data set for training neural networks. A proven method for creating synthetic data based on a graphics processor using a graphics pipeline is presented. A distinctive feature of this method is its modular architecture, which makes it easy to modify, delete or add individual stages to the synthetic image generation pipeline. A one-stage automatic detector based on a convolutional neural network of the Yolo type has been trained, the quality of the trained model and the operation of the recognition algorithm were also evaluated. Conclusions are drawn regarding the possibility of using this approach when creating representative samples of a large volume and their further use for training neural networks in pattern recognition.

Sensor-based characterization of construction and demolition waste at high occupancy densities using synthetic training data and deep learning.

Sensor-based monitoring of construction and demolition waste (CDW) streams plays an important role in recycling (RC). Extracted knowledge about the composition of a material stream helps identifying RC paths, optimizing processing plants and form the basis for sorting. To enable economical use, it is necessary to ensure robust detection of individual objects even with high material throughput. Conventional algorithms struggle with resulting high occupancy densities and object overlap, making deep learning object detection methods more promising. In this study, different deep learning architectures for object detection (Region-based CNN/Region-based Convolutional Neural Network (Faster R-CNN), You only look once (YOLOv3), Single Shot MultiBox Detector (SSD)) are investigated with respect to their suitability for CDW characterization. A mixture of brick and sand-lime brick is considered as an exemplary waste stream. Particular attention is paid to detection performance with increasing occupancy density and particle overlap. A method for the generation of synthetic training images is presented, which avoids time-consuming manual labelling. By testing the models trained on synthetic data on real images, the success of the method is demonstrated. Requirements for synthetic training data composition, potential improvements and simplifications of different architecture approaches are discussed based on the characteristic of the detection task. In addition, the required inference time of the presented models is investigated to ensure their suitability for use under real-time conditions.

Generation of synthetic generative adversarial network-based multispectral satellite images with improved sharpness

Generation of synthetic generative adversarial network-based multispectral satellite images with improved sharpness