Synthetic Image Data Research Articles

Generative artificial intelligence enables the generation of bone scintigraphy images and improves generalization of deep learning models in data-constrained environments.

Advancements of deep learning in medical imaging are often constrained by the limited availability of large, annotated datasets, resulting in underperforming models when deployed under real-world conditions. This study investigated a generative artificial intelligence (AI) approach to create synthetic medical images taking the example of bone scintigraphy scans, to increase the data diversity of small-scale datasets for more effective model training and improved generalization. We trained a generative model on 99mTc-bone scintigraphy scans from 9,170 patients in one center to generate high-quality and fully anonymized annotated scans of patients representing two distinct disease patterns: abnormal uptake indicative of (i) bone metastases and (ii) cardiac uptake indicative of cardiac amyloidosis. A blinded reader study was performed to assess the clinical validity and quality of the generated data. We investigated the added value of the generated data by augmenting an independent small single-center dataset with synthetic data and by training a deep learning model to detect abnormal uptake in a downstream classification task. We tested this model on 7,472 scans from 6,448 patients across four external sites in a cross-tracer and cross-scanner setting and associated the resulting model predictions with clinical outcomes. The clinical value and high quality of the synthetic imaging data were confirmed by four readers, who were unable to distinguish synthetic scans from real scans (average accuracy: 0.48% [95% CI 0.46-0.51]), disagreeing in 239 (60%) of 400 cases (Fleiss' kappa: 0.18). Adding synthetic data to the training set improved model performance by a mean (± SD) of 33(± 10)% AUC (p < 0.0001) for detecting abnormal uptake indicative of bone metastases and by 5(± 4)% AUC (p < 0.0001) for detecting uptake indicative of cardiac amyloidosis across both internal and external testing cohorts, compared to models without synthetic training data. Patients with predicted abnormal uptake had adverse clinical outcomes (log-rank: p < 0.0001). Generative AI enables the targeted generation of bone scintigraphy images representing different clinical conditions. Our findings point to the potential of synthetic data to overcome challenges in data sharing and in developing reliable and prognostic deep learning models in data-limited environments.

New ways to use imaging data in cardiovascular research: survey of opinions on federated learning and synthetic data

Abstract Introduction Federated learning and the creation of synthetic data are emerging tools which may enhance the use of imaging data in cardiovascular research. This study sought to understand the perspectives of cardiovascular imaging researchers on the potential benefits and challenges associated with these technologies. Methods The British Heart Foundation Data Science Centre conducted a series of online surveys and a virtual workshop to gather insights from stakeholders involved in cardiovascular imaging research about federated learning and synthetic data generation. Results The federated learning survey included 67 respondents: 18% (n=12) were currently using federated learning, 4% (n=3) had previously used it, 31% (n=21) were planning to use it, and 46% (n=31) were neither using nor planning to use it. Highlighted benefits included data privacy and enhanced collaboration, while challenges included data heterogeneity and technical complexity. The synthetic data survey had 22 respondents: 50% (n=11) were currently using synthetic imaging data, 36% (n=8) expressed interest in using it, and 14% (n=3) thought it should not be used. Among the respondents, 50% had created synthetic imaging data and 45% had used it in cardiovascular research. Advantages cited included privacy preservation, increased dataset size and diversity, improved data access, and reduced administrative burden. Concerns included potential biases, trust issues, privacy concerns, and the fact that the images were not real and may have limited diversity or quality. Conclusion Federated learning and synthetic data offer opportunities for advancing cardiovascular imaging research by addressing data privacy concerns and expanding data availability. However, challenges must be addressed to realise their full potential.

Kinematics of abdominal aortic Aneurysms.

A search in Scopus within "Article title, Abstract, Keywords" unveils 2,444 documents focused on the biomechanics of Abdominal Aortic Aneurysm (AAA), mostly on AAA wall stress. Only 24 documents investigated AAA kinematics, an important topic that could potentially offer significant insights into the biomechanics of AAA. In this paper, we present an image-based approach for patient-specific, in vivo, and non-invasive AAA kinematic analysis using patient's time-resolved 3D computed tomography angiography (4D-CTA) images, with an objective to measure wall displacement and strain during the cardiac cycle. Our approach relies on regularized deformable image registration for estimating wall displacement, estimation of the local wall strain as the ratio of its normal displacement to its local radius of curvature, and local surface fitting with non-deterministic outlier detection for estimating the wall radius of curvature. We verified our approach against synthetic ground truth image data created by warping a 3D-CTA image of AAA using a realistic displacement field obtained from a finite element biomechanical model. We applied our approach to assess AAA wall displacements and strains in ten patients. Our kinematic analysis results indicated that the 99th percentile of circumferential wall strain, among all patients, ranged from 2.62% to 5.54%, with an average of 4.45% and a standard deviation of 0.87%. We also observed that AAA wall strains are significantly lower than those of a healthy aorta. Our work demonstrates that the registration-based measurement of AAA wall displacements in the direction normal to the wall is sufficiently accurate to reliably estimate strain from these displacements.

Where, Why, and How is Bias Learned in Medical Image Analysis Models? A Study of Bias Encoding within Convolutional Networks using Synthetic Data

Understanding the mechanisms of algorithmic bias is highly challenging due to the complexity and uncertainty of how various unknown sources of bias impact deep learning models trained with medical images. This study aims to bridge this knowledge gap by studying where, why, and how biases from medical images are encoded in these models. We systematically studied layer-wise bias encoding in a convolutional neural network for disease classification using synthetic brain magnetic resonance imaging data with known disease and bias effects. We quantified the degree to which disease-related information, as well as morphology-based and intensity-based biases were represented within the learned features of the model. Although biases were encoded throughout the model, a stronger encoding did not necessarily lead to the model using these biases as a shortcut for disease classification. We also observed that intensity-based effects had a greater influence on shortcut learning compared to morphology-based effects when multiple biases were present. We believe that these results constitute an important first step towards a deeper understanding of algorithmic bias in deep learning models trained using medical imaging data. This study also showcases the benefits of utilising controlled, synthetic bias scenarios for objectively studying the mechanisms of shortcut learning. Alberta Innovates, Natural Sciences and Engineering Research Council of Canada, Killam Trusts, Parkinson Association of Alberta, River Fund at Calgary Foundation, Canada Research Chairs Program.

Generative adversarial network model to classify human induced pluripotent stem cell-cardiomyocytes based on maturation level

Our study develops a generative adversarial network (GAN)-based method that generates faithful synthetic image data of human cardiomyocytes at varying stages in their maturation process, as a tool to significantly enhance the classification accuracy of cells and ultimately assist the throughput of computational analysis of cellular structure and functions. Human induced pluripotent stem cell derived cardiomyocytes (hiPSC-CMs) were cultured on micropatterned collagen coated hydrogels of physiological stiffnesses to facilitate maturation and optical measurements were performed for their structural and functional analyses. Control groups were cultured on collagen coated glass well plates. These image recordings were used as the real data to train the GAN model. The results show the GAN approach is able to replicate true features from the real data, and inclusion of such synthetic data significantly improves the classification accuracy compared to usage of only real experimental data that is often limited in scale and diversity. The proposed model outperformed four conventional machine learning algorithms with respect to improved data generalization ability and data classification by incorporating synthetic data. This work demonstrates the importance of integrating synthetic data in situations where there are limited sample sizes and thus, effectively addresses the challenges imposed by data availability.

Addressing the Generalizability of AI in Radiology Using a Novel Data Augmentation Framework with Synthetic Patient Image Data: Proof-of-Concept and External Validation for Classification Tasks in Multiple Sclerosis.

Artificial intelligence (AI) models often face performance drops after deployment to external datasets. This study evaluated the potential of a novel data augmentation framework based on generative adversarial networks (GANs) that creates synthetic patient image data for model training to improve model generalizability. Model development and external testing were performed for a given classification task, namely the detection of new fluid-attenuated inversion recovery lesions at MRI during longitudinal follow-up of patients with multiple sclerosis (MS). An internal dataset of 669 patients with MS (n = 3083 examinations) was used to develop an attention-based network, trained both with and without the inclusion of the GAN-based synthetic data augmentation framework. External testing was performed on 134 patients with MS from a different institution, with MR images acquired using different scanners and protocols than images used during training. Models trained using synthetic data augmentation showed a significant performance improvement when applied on external data (area under the receiver operating characteristic curve [AUC], 83.6% without synthetic data vs 93.3% with synthetic data augmentation; P = .03), achieving comparable results to the internal test set (AUC, 95.0%; P = .53), whereas models without synthetic data augmentation demonstrated a performance drop upon external testing (AUC, 93.8% on internal dataset vs 83.6% on external data; P = .03). Data augmentation with synthetic patient data substantially improved performance of AI models on unseen MRI data and may be extended to other clinical conditions or tasks to mitigate domain shift, limit class imbalance, and enhance the robustness of AI applications in medical imaging. Keywords: Brain, Brain Stem, Multiple Sclerosis, Synthetic Data Augmentation, Generative Adversarial Network Supplemental material is available for this article. © RSNA, 2024.

Synthetic Image Data Generation via Rendering Techniques for Training AI-Based Instance Segmentation

Synthetic image data generation has gained popularity in computer vision and machine learning in recent years. The work introduces a technique for creating artificial image data by utilizing 3D files and rendering methods in Python and Blender. The technique employs BlenderProc, a rendering tool for generating artificial images, to efficiently create a substantial amount of data. The output of the method is saved in JSON format, containing COCO annotations of objects in the images, facilitating seamless integration with current machine-learning pipelines. The paper shows that the created synthetic data can be used to enhance object data during simulation. The method can enhance the accuracy and robustness of machine-learning models by modifying simulation parameters like lighting, camera position, and object orientation to create a variety of images. This is especially beneficial for applications that require significant amounts of labelled real-world data, which can be time-consuming and labour-intensive to obtain. The study addresses the constraints and potential prejudices of creating synthetic data, emphasizing the significance of verifying and assessing the generated data prior to its utilization in machine learning models. Synthetic data generation can be a valuable tool for improving the efficiency and effectiveness of machine learning and computer vision applications. However, it is crucial to thoroughly assess the potential limitations and biases of the generated data. This paper emphasizes the potential of synthetic data generation to enhance the accuracy and resilience of machine learning models, especially in scenarios with limited access to labelled real-world data. This paper introduces a method that efficiently produces substantial amounts of synthetic image data with COCO annotations, serving as a valuable resource for professionals in computer vision and machine learning.

Synthetic image data generation via rendering techniques for training AI-based Instance Segmentation

Within the scope of the development of the project, the acceleration and gyro signal, VR implementation, and User Control for VR were realized. The starting point was a list of MPU9250 data, which are acceleration and gyro data. The goal is to do the signal processing of the signal that is got from MPU9250 and implement the data with a specific format so that the LINAK LA36 actuator can be controlled in such a way that it performs as desired position. For this purpose, a multi-axis acceleration and position sensor is mounted on a surfing board which then collects the acceleration and Gyro data. A MATLAB Program had been written to convert the desired position into a CAN message. Before that, the raw acceleration data and gyro must be converted and filtered into actual data. First, the acceleration data was collected from another group. A signal processing method has been implemented. This includes gravitational force filter, noise filter, integration, and complementary filter method was used in order to get the accurate actual data that is able to be implemented into the system. For those methods, it was necessary to deal extensively with the characteristics of signal processing. VR application was implemented in this project to give the user a sensational surfing experience in the virtual world. User control has been implemented as well to allow users to control the video. Unity with Arduino work together so that both Arduino and Unity can communicate with each other, and signal messages will be sent to the Arduino from Unity and vice versa to achieve the synchronization of the video signal and Unity VR application.

Generating Synthetic Data for Medical Imaging.

Artificial intelligence (AI) models for medical imaging tasks, such as classification or segmentation, require large and diverse datasets of images. However, due to privacy and ethical issues, as well as data sharing infrastructure barriers, these datasets are scarce and difficult to assemble. Synthetic medical imaging data generated by AI from existing data could address this challenge by augmenting and anonymizing real imaging data. In addition, synthetic data enable new applications, including modality translation, contrast synthesis, and professional training for radiologists. However, the use of synthetic data also poses technical and ethical challenges. These challenges include ensuring the realism and diversity of the synthesized images while keeping data unidentifiable, evaluating the performance and generalizability of models trained on synthetic data, and high computational costs. Since existing regulations are not sufficient to guarantee the safe and ethical use of synthetic images, it becomes evident that updated laws and more rigorous oversight are needed. Regulatory bodies, physicians, and AI developers should collaborate to develop, maintain, and continually refine best practices for synthetic data. This review aims to provide an overview of the current knowledge of synthetic data in medical imaging and highlights current key challenges in the field to guide future research and development.

Synthetic Medical Imaging Generation with Generative Adversarial Networks for Plain Radiographs

In medical imaging, access to data is commonly limited due to patient privacy restrictions, and it can be difficult to acquire enough data in the case of rare diseases. The purpose of this investigation was to develop a reusable open-source synthetic image-generation pipeline, the GAN Image Synthesis Tool (GIST), that is easy to use as well as easy to deploy. The pipeline helps to improve and standardize AI algorithms in the digital health space by generating high quality synthetic image data that is not linked to specific patients. Its image generation capabilities include the ability to generate imaging of pathologies or injuries with low incidence rates. This improvement of digital health AI algorithms could improve diagnostic accuracy, aid in patient care, decrease medicolegal claims, and ultimately decrease the overall cost of healthcare. The pipeline builds on existing Generative Adversarial Networks (GANs) algorithms, and preprocessing and evaluation steps were included for completeness. For this work, we focused on ensuring the pipeline supports radiography, with a focus on synthetic knee and elbow X-ray images. In designing the pipeline, we evaluated the performance of current GAN architectures, studying the performance on available X-ray data. We show that the pipeline is capable of generating high-quality and clinically relevant images based on a lay person’s evaluation and the Fréchet Inception Distance (FID) metric.

Generating Synthetic MR Spectroscopic Imaging Data with Generative Adversarial Networks to Train Machine Learning Models.

To develop a new method to generate synthetic MR spectroscopic imaging (MRSI) data for training machine learning models. This study targeted routine MRI examination protocols with single voxel spectroscopy (SVS). A novel model derived from pix2pix generative adversarial networks was proposed to generate synthetic MRSI data using MRI and SVS data as inputs. T1- and T2-weighted, SVS, and reference MRSI data were acquired from healthy brains with clinically available sequences. The proposed model was trained to generate synthetic MRSI data. Quantitative evaluation involved the calculation of the mean squared error (MSE) against the reference and metabolite ratio value. The effect of the location of and the number of the SVS data on the quality of the synthetic MRSI data was investigated using the MSE. The synthetic MRSI data generated from the proposed model were visually closer to the reference. The 95% confidence interval (CI) of the metabolite ratio value of synthetic MRSI data overlapped with the reference for seven of eight metabolite ratios. The MSEs tended to be lower in the same location than in different locations. The MSEs among groups of numbers of SVS data were not significantly different. A new method was developed to generate MRSI data by integrating MRI and SVS data. Our method can potentially increase the volume of MRSI data training for other machine learning models by adding SVS acquisition to routine MRI examinations.

High-Resolution Two-Phase Velocimetry Of Aspherical Particles Using Wavelet-Based Optical Flow Velocimetry (WOFV) Benchmarked With Fully Resolved Direct Numerical Simulations

Several techniques exist to measure the carrier and dispersed phases in multi-phase flows. Particle image velocimetry (PIV) analyzes the carrier phase by cross-correlating particle images, but its resolution is limited by the size of the interrogation window, making it difficult to resolve fine-scale turbulent structures. Particle tracking algorithms capture translational motion of dispersed particles well, but struggle with rotational motion, especially for irregular and aspherical particles. Currently, there is no method that universally measures both the carrier phase velocity field and the translational and rotational motions of dispersed particles. This study evaluates wavelet-based optical flow velocimetry (wOFV) for motion estimation in multi-phase flows, focusing on dispersed ellipsoidal particles and their surrounding turbulent carrier flow using synthetic image data. The research proceeds in two phases: first, the rigid motion of ellipses, including translational and rotational components, is analyzed using wOFV-generated dense motion fields. The results highlight the critical role of the regularization weighting parameter λ. Higher λ values improve translational motion estimation, while an optimal λ avoids under-regularization and non-physical structures in rotational motion. wOFV maintains accuracy in combined motion scenarios with optimal λ values. In the second phase, wOFV captures the turbulent carrier flow around an aspherical particle, benchmarked against DNS data and compared to PIV. wOFV outperforms PIV in resolving finer structures near the particle surface and in accurately representing the wake region. Error analyses confirm wOFV’s superior performance, with optimal results within a specific λ range. In conclusion, wOFV is a highly effective tool for the analysis of multi-phase flow dynamics, providing greater resolution and accuracy than PIV, especially in complex scenarios involving aspherical particles.

TENSOR QUANTILE REGRESSION WITH LOW-RANK TENSOR TRAIN ESTIMATION.

Neuroimaging studies often involve predicting a scalar outcome from an array of images collectively called tensor. The use of magnetic resonance imaging (MRI) provides a unique opportunity to investigate the structures of the brain. To learn the association between MRI images and human intelligence, we formulate a scalar-on-image quantile regression framework. However, the high dimensionality of the tensor makes estimating the coefficients for all elements computationally challenging. To address this, we propose a low-rank coefficient array estimation algorithm based on tensor train (TT) decomposition which we demonstrate can effectively reduce the dimensionality of the coefficient tensor to a feasible level while ensuring adequacy to the data. Our method is more stable and efficient compared to the commonly used, Canonic Polyadic rank approximation-based method. We also propose a generalized Lasso penalty on the coefficient tensor to take advantage of the spatial structure of the tensor, further reduce the dimensionality of the coefficient tensor, and improve the interpretability of the model. The consistency and asymptotic normality of the TT estimator are established under some mild conditions on the covariates and random errors in quantile regression models. The rate of convergence is obtained with regularization under the total variation penalty. Extensive numerical studies, including both synthetic and real MRI imaging data, are conducted to examine the empirical performance of the proposed method and its competitors.

Cutout as augmentation in contrastive learning for detecting burn marks in plastic granules

Abstract. Plastic granules are a common delivery form for creating products in industries such as the plastic manufacturing, construction and automotive ones. In the corresponding sorting process of plastic granules, diverse defect types could appear. Burn marks, which potentially lead to weakened structural integrity of the plastic, are one of the most common types. Thus, plastic granules with burn marks should be filtered out during the sorting process. Artificial intelligence (AI)-based anomaly detection approaches are widely used in the field of visual-based sorting due to the higher accuracy and lower requirement of expert knowledge compared with classic rule-based algorithms (Chandola et al., 2009). In this contribution, a simple data augmentation strategy, cutout, is implemented as a way of simulating defects when combined with a contrastive learning-based methodology and is proven to improve the accuracy of the anomaly detection of burn marks. Different variants of cutout are also evaluated. Specifically, synthetic image data are used due to the lack of real data.

CycleGAN generated pneumonia chest x-ray images: Evaluation with vision transformer

The use of generative models in image synthesis has become increasingly prevalent. Synthetic medical imaging data is of paramount importance, primarily because medical imaging data is scarce, costly, and encumbered by legal considerations pertaining to patient confidentiality. Synthetic medical images offer a potential answer to these issues. The predominant approaches primarily assess the quality of images and the degree of resemblance between these images and the original ones employed for their generation.The central idea of the work can be summarized in the question: Do the performance metrics of Frechet Inception Distance(FID) and Inception Score(IS) in the Cycle-consistent Generative Adversarial Networks (CycleGAN) model are adequate to determine how real a generated chest x-ray pneumonia image is? In this study, a CycleGAN model was employed to produce artificial images depicting 3 classes of chest x-ray pneumonia images: general(any type), bacterial, and viral pneumonia. The quality of the images were evaluated assessing and contrasting 3 criteria: performance metric of CycleGAN model, clinical assessment of respiratory experts and the results of classification of a visual transformer(ViT). The overall results showed that the evaluation metrics of the CycleGAN are insufficient to establish realism in generated medical images.

Spatializing soil elemental concentration as measured by X-ray fluorescence analysis using remote sensing data

Spatial soil information has significantly contributed to public policy and planning. The use of X-ray fluorescence spectrometry (XRF) in soil science, primarily in laboratory settings or for isolated point analyses, has been well documented. This study broadens the application of XRF data by integrating it with remote sensing data for a comprehensive understanding of soil patterns and processes. Focusing on a 2574 km2 area in Brazil, we utilized a lab-based XRF instrument to measure the total concentrations of key chemical elements in the soil, such as Al, Si, Ti, and Fe. These geochemical data were then spatialized using the digital soil mapping (DSM) framework. We employed a synthetic bare soil image and elevation data as covariates in the analysis. The spatial accuracy assessment yielded R2 values ranging from 0.65 to 0.81 for Fe, 0.62 to 0.78 for Ti, 0.52 to 0.63 for Si, and 0.48 to 0.63 for Al. The digital soil maps of geochemical elements aligned well with existing pedological, and geological maps as evaluated using multiple correspondence analysis. The integration of spatialized XRF data with DSM and remote sensing techniques shows significant promise in assessing soil variations across landscapes based on their chemical composition.

Denoising diffusion probabilistic models for generation of realistic fully-annotated microscopy image datasets.

Recent advances in computer vision have led to significant progress in the generation of realistic image data, with denoising diffusion probabilistic models proving to be a particularly effective method. In this study, we demonstrate that diffusion models can effectively generate fully-annotated microscopy image data sets through an unsupervised and intuitive approach, using rough sketches of desired structures as the starting point. The proposed pipeline helps to reduce the reliance on manual annotations when training deep learning-based segmentation approaches and enables the segmentation of diverse datasets without the need for human annotations. We demonstrate that segmentation models trained with a small set of synthetic image data reach accuracy levels comparable to those of generalist models trained with a large and diverse collection of manually annotated image data, thereby offering a streamlined and specialized application of segmentation models.

The application of artificial intelligence algorithms for the generation of synthetic medical imaging data

The application of artificial intelligence algorithms for the generation of synthetic medical imaging data

The Effects of Different Image Parameters on Human Action Recognition Models Trained on Real and Synthetic Image Data

The Effects of Different Image Parameters on Human Action Recognition Models Trained on Real and Synthetic Image Data

Super-Resolution Reconstruction of CT Images Based on Multi-scale Information Fused Generative Adversarial Networks.

The popularization and widespread use of computed tomography (CT) in the field of medicine evocated public attention to the potential radiation exposure endured by patients. Reducing the radiation dose may lead to scattering noise and low resolution, which can adversely affect the radiologists' judgment. Hence, this paper introduces a new network called PANet-UP-ESRGAN (PAUP-ESRGAN), specifically designed to obtain low-dose CT (LDCT) images with high peak signal-to-noise ratio (PSNR) and high resolution (HR). The model was trained on synthetic medical image data based on a Generative Adversarial Network (GAN). A degradation modeling process was introduced to accurately represent realistic degradation complexities. To reconstruct image edge textures, a pyramidal attention model call PANet was added before the middle of the multiple residual dense blocks (MRDB) in the generator to focus on high-frequency image information. The U-Net discriminator with spectral normalization was also designed to improve its efficiency and stabilize the training dynamics. The proposed PAUP-ESRGAN model was evaluated on the abdomen and lung image datasets, which demonstrated a significant improvement in terms of robustness of model and LDCT image detail reconstruction, compared to the latest real-esrgan network. Results showed that the mean PSNR increated by 19.1%, 25.05%, and 21.25%, the mean SSIM increated by 0.4%, 0.4%, and 0.4%, and the mean NRMSE decreated by 0.25%, 0.25%, and 0.35% at 2[Formula: see text], 4[Formula: see text], and 8[Formula: see text] super-resolution scales, respectively. Experimental results demonstrate that our method outperforms the state-of-the-art super-resolution methods on restoring CT images with respect to peak signal-to-noise ratio (PSNR), structural similarity (SSIM) and normalized root-mean-square error (NRMSE) indices.