Synthetic Dataset Generation Research Articles

Review and analysis of synthetic dataset generation methods and techniques for application in computer vision

Review and analysis of synthetic dataset generation methods and techniques for application in computer vision

Synthetic data in health care: A narrative review.

Data are central to research, public health, and in developing health information technology (IT) systems. Nevertheless, access to most data in health care is tightly controlled, which may limit innovation, development, and efficient implementation of new research, products, services, or systems. Using synthetic data is one of the many innovative ways that can allow organizations to share datasets with broader users. However, only a limited set of literature is available that explores its potentials and applications in health care. In this review paper, we examined existing literature to bridge the gap and highlight the utility of synthetic data in health care. We searched PubMed, Scopus, and Google Scholar to identify peer-reviewed articles, conference papers, reports, and thesis/dissertations articles related to the generation and use of synthetic datasets in health care. The review identified seven use cases of synthetic data in health care: a) simulation and prediction research, b) hypothesis, methods, and algorithm testing, c) epidemiology/public health research, d) health IT development, e) education and training, f) public release of datasets, and g) linking data. The review also identified readily and publicly accessible health care datasets, databases, and sandboxes containing synthetic data with varying degrees of utility for research, education, and software development. The review provided evidence that synthetic data are helpful in different aspects of health care and research. While the original real data remains the preferred choice, synthetic data hold possibilities in bridging data access gaps in research and evidence-based policymaking.

Generating of synthetic datasets using diffusion models for solving computer vision tasks in urban applications

The paper illustrates the process of generating synthetic data using diffusion neural networks for solving popular computer vision tasks in urban applications: traffic vehicles detection and classification. The result of this study demonstrated new approach for synthetic datasets creation using modern generative neural networks. To elaborate this approach, experiments were performed to generate synthetic dataset to solve the above basic computer vision problem in urban applications using the most advanced diffusion neural network models (Kandinsky 2.2). Experiments were performed on these datasets to train deep neural networks to solve the object detection problem (YOLOv5). The results of testing the trained detectors on a specially selected validation dataset showed the potential viability of a synthetic dataset generation approach using diffusion neural network models. However, full-fledged use of this approach to generate synthetic datasets that can be used in training deep neural networks of computer vision in production is accompanied by some difficulties, namely high difference between domains of real data and generated data, high labor costs of tuning trained diffusion models to achieve the highest quality of generation, etc.

What makes the unsupervised monocular depth estimation (UMDE) model training better

Current computer vision tasks based on deep learning require a huge amount of data with annotations for model training or testing, especially in some dense estimation tasks, such as optical flow segmentation and depth estimation. In practice, manual labeling for dense estimation tasks is very difficult or even impossible, and the scenes of the dataset are often restricted to a small range, which dramatically limits the development of the community. To overcome this deficiency, we propose a synthetic dataset generation method to obtain the expandable dataset without burdensome manual workforce. By this method, we construct a dataset called MineNavi containing video footages from first-perspective-view of the aircraft matched with accurate ground truth for depth estimation in aircraft navigation application. We also provide quantitative experiments to prove that pre-training via our MineNavi dataset can improve the performance of depth estimation model and speed up the convergence of the model on real scene data. Since the synthetic dataset has a similar effect to the real-world dataset in the training process of deep model, we finally conduct the experiments on MineNavi with unsupervised monocular depth estimation (UMDE) deep learning models to demonstrate the impact of various factors in our dataset such as lighting conditions and motion mode, aiming to explore what makes this kind of models training better.

Synthetic LiDAR-Labeled Data Set Generation to Train Deep Neural Networks for Object Classification in IoT at the Edge

Light detection and ranging (LiDAR) sensors are increasing in popularity due to the advantages they provide over 2-D sensors in IoT object detection and classification applications, because of their ability to provide very precise distances to objects. Deep learning algorithms need a huge amount of data during training to obtain high accuracy results. When using 2-D images, a vast quantity of data sets are publicly available, but this is not the case for LiDAR point clouds. Each LiDAR model generates a point cloud with unique properties, which causes the data sets not to be compatible between different LiDAR models. As a result, when using deep learning with LiDARs, it is necessary to generate the data sets manually. For this purpose, the data must be captured and then labeled one by one, which is a very time and cost-consuming process. To overcome this issue and to reduce the development time when using LiDAR sensors with deep learning algorithms, a methodology is proposed in this article to automatically generate point cloud data sets using a 3-D simulator for autonomous cars. In this regard, a data set can be generated for any LiDAR model by adding the specific LiDAR parameters to the simulator. Besides, custom scenarios can be designed and generated, based on the final deployment location, to provide a simulated solution very close to the final implementation. With the proposed methodology, a simulation can be performed to select the LiDAR that best fits certain application requirements, in contrast to the traditional approach where the LiDAR must first be purchased.

A genetically-optimised artificial life algorithm for complexity-based synthetic dataset generation

Algorithmic evaluation is a vital step in developing new approaches to machine learning and relies on the availability of existing datasets. However, real-world datasets often do not cover the necessary complexity space required to understand an algorithm’s domains of competence. As such, the generation of synthetic datasets to fill gaps in the complexity space has gained attention, offering a means of evaluating algorithms when data is unavailable. Existing approaches to complexity-focused data generation are limited in their ability to generate solutions that invoke similar classification behaviour to real data. The present work proposes a novel method (Sy:Boid) for complexity-based synthetic data generation, adapting and extending the Boid algorithm that was originally intended for computer graphics simulations. Sy:Boid embeds the modified Boid algorithm within an evolutionary multi-objective optimisation algorithm to generate synthetic datasets which satisfy predefined magnitudes of complexity measures. Sy:Boid is evaluated and compared to labelling-based and sampling-based approaches to data generation to understand its ability to generate a wide variety of realistic datasets. Results demonstrate Sy:Boid is capable of generating datasets across a greater portion of the complexity space than existing approaches. Furthermore, the produced datasets were observed to invoke very similar classification behaviours to that of real data.

MSP Patches Based Optimized EEG Source Localization and Validation in Visual Cortex of Human Brain

This paper is focused on optimizing EEG source localization of visual neural activities generated in the posterior lobe of the human brain. The visual and neural systems of the human brain process the captured images of faces or scenes into optical and chemical neurons in order to produce electrical potentials over the scalp surface, where EEG electrodes measure these signals to sense the underneath visual brain activity. However, it is categorically hard to localize the true neural sources in the human brain's visual cortex due to overlapping and interaction of other active areas of the lobes of the brain. Thus, a novel algorithm of MSP inversion-based Bayesian framework with varying patches for providing the optimal free energy and minimum location in the visual cortex is proposed to address this issue. This algorithm is integrated with a synthetic EEG dataset generation scheme to validate active neural sources. This proposed algorithm provides satisfactory results in terms of optimal free energy with minimum localization error and validates true active sources in the brain. The SPM12 Toolbox is applied in processing the visual EEG dataset in this research. The application of this proposed algorithm is beneficial in terms of localizing the optimum visual sources and identifying the visual disorders or diseases in the human brain.

Statistical characterization of experimental magnetized liner inertial fusion stagnation images using deep-learning-based fuel–background segmentation

Significant variety is observed in spherical crystal x-ray imager (SCXI) data for the stagnated fuel–liner system created in Magnetized Liner Inertial Fusion (MagLIF) experiments conducted at the Sandia National Laboratories Z-facility. As a result, image analysis tasks involving, e.g., region-of-interest selection (i.e. segmentation), background subtraction and image registration have generally required tedious manual treatment leading to increased risk of irreproducibility, lack of uncertainty quantification and smaller-scale studies using only a fraction of available data. We present a convolutional neural network (CNN)-based pipeline to automate much of the image processing workflow. This tool enabled batch preprocessing of an ensemble of $N_{\text {scans}} = 139$ SCXI images across $N_{\text {exp}} = 67$ different experiments for subsequent study. The pipeline begins by segmenting images into the stagnated fuel and background using a CNN trained on synthetic images generated from a geometric model of a physical three-dimensional plasma. The resulting segmentation allows for a rules-based registration. Our approach flexibly handles rarely occurring artifacts through minimal user input and avoids the need for extensive hand labelling and augmentation of our experimental dataset that would be needed to train an end-to-end pipeline. We also fit background pixels using low-degree polynomials, and perform a statistical assessment of the background and noise properties over the entire image database. Our results provide a guide for choices made in statistical inference models using stagnation image data and can be applied in the generation of synthetic datasets with realistic choices of noise statistics and background models used for machine learning tasks in MagLIF data analysis. We anticipate that the method may be readily extended to automate other MagLIF stagnation imaging applications.

Generation of synthetic datasets using weighted bayesian association rules in clinical world

Generation of synthetic datasets using weighted bayesian association rules in clinical world

Automatic generation of synthetic datasets from a city digital twin for use in the instance segmentation of building facades

Abstract The extraction and integration of building facade data are necessary for the development of information infrastructure for urban environments. However, existing methods for parsing building facades based on semantic segmentation have difficulties in distinguishing individual instances of connected buildings. Manually collecting and annotating instances of building facades in large datasets is time-consuming and labor-intensive. With the recent development and use of city digital twins (CDTs), massive high-quality digital assets of buildings have been created. These assets make it possible to generate high-quality and cost-effective synthetic datasets that can replace real-world ones as training sets for the supervised learning-based instance segmentation of building facades. In this study, we developed a novel framework that can automatically produce synthetic datasets from a CDT. An auto-generation system for synthetic street views was built by rendering city digital assets in a game engine, while the system auto-generated the instance annotations for building facades. The hybrid dataset HSRBFIA, along with various subsets containing different proportions of synthetic and real data, were used to train deep learning models for facade instance segmentation. In our experiments, two types of synthetic data (CDT-based and virtual-based) were compared, and the results showed that the CDT synthetic data were more effective in boosting deep learning training with real-world images compared with the virtual synthetic data (no real-world counterparts). By swapping a certain portion of the real data with the proposed CDT synthetic images, the performance could almost match what is achievable when using the real-world training set.

On the generation of realistic synthetic petrographic datasets using a style-based GAN

Deep learning architectures have transformed data analytics in geosciences, complementing traditional approaches to geological problems. Although deep learning applications in geosciences show encouraging signs, their potential remains untapped due to limited data availability and the required in-depth knowledge to provide a high-quality labeled dataset. We approached these issues by developing a novel style-based deep generative adversarial network (GAN) model, PetroGAN, to create the first realistic synthetic petrographic datasets across different rock types. PetroGAN adopts the architecture of StyleGAN2 with adaptive discriminator augmentation (ADA) to allow robust replication of statistical and esthetical characteristics and improve the internal variance of petrographic data. In this study, the training dataset consists of > 10,000 thin section images both under plane- and cross-polarized lights. Here, using our proposed novel approach, the model reached a state-of-the-art Fréchet Inception Distance (FID) score of 12.49 for petrographic images. We further observed that the FID values vary with lithology type and image resolution. The generated images were validated through a survey where the participants have various backgrounds and level of expertise in geosciences. The survey established that even a subject matter expert observed the generated images were indistinguishable from real images. This study highlights that GANs are a powerful method for generating realistic synthetic data in geosciences. Moreover, they are a future tool for image self-labeling, reducing the effort in producing big, high-quality labeled geoscience datasets. Furthermore, our study shows that PetroGAN can be applied to other geoscience datasets, opening new research horizons in the application of deep learning to various fields in geosciences, particularly with the presence of limited datasets.

Experimental methods to predict dynamic vision sensor event camera performance

Dynamic vision sensors (DVS) represent a promising new technology, offering low power consumption, sparse output, high temporal resolution, and wide dynamic range. These features make DVS attractive for new research areas including scientific and space-based applications; however, more precise understanding of how sensor input maps to output under real-world constraints is needed. Often, metrics used to characterize DVS report baseline performance by measuring observable limits but fail to characterize the physical processes at the root of those limits. To address this limitation, we describe step-by-step procedures to measure three important performance parameters: (1) temporal contrast threshold, (2) cutoff frequency, and (3) refractory period. Each procedure draws inspiration from previous work, but links measurements sequentially to infer physical phenomena at the root of measured behavior. Results are reported over a range of brightness levels and user-defined biases. The threshold measurement technique is validated with test-pixel node voltages, and a first-order low-pass approximation of photoreceptor response is shown to predict event cutoff temporal frequency to within 9% accuracy. The proposed method generates lab-measured parameters compatible with the event camera simulator v2e, allowing more accurate generation of synthetic datasets for innovative applications.

Generation of synthetic manufacturing datasets for machine learning using discrete-event simulation

ABSTRACT Recent advances in computing power have seen machine learning becoming an area of significant interest in manufacturing for scholars attempting to realise its full potential. Successful machine learning applications require a great amount of specific production data that is not easily nor publicly accessible. This study aims to develop a framework to use discrete-event-simulation (DES) to generate large datasets for training machine learning models. Three DES models were designed and executed to generate synthetic production data for different manufacturing scenarios. Inferences were made on the dependency between the time required to generate data and the complexity of the simulation model. The experimental results show that with the incremental changes in the simulation model, the time required to generate synthetic data tends to increase. The study revealed that DES is an effective tool for generating high-quality synthetic data which can be fed into machine learning models for training. The datasets generated by the simulations are made publicly available.

SyntEO: Synthetic dataset generation for earth observation and deep learning – Demonstrated for offshore wind farm detection

With the emergence of deep learning in the last years, new opportunities arose in Earth observation research. Nevertheless, they also brought with them new challenges. The data-hungry training processes of deep learning models demand large, resource expensive, annotated datasets and partly replaced knowledge-driven approaches so that model behaviour and the final prediction process became a black box. The proposed SyntEO approach enables Earth observation researchers to automatically generate large deep learning ready datasets by merging existing and procedural data. SyntEO does this by including expert knowledge in the data generation process in a highly structured manner to control the automatic image and label generation by employing an ontology. In this way, fully controllable experiment environments are set up, which support insights in the model training on the synthetic datasets. Thus, SyntEO makes the learning process approachable, which is an important cornerstone for explainable machine learning. We demonstrate the SyntEO approach by predicting offshore wind farms in Sentinel-1 images on two of the worlds largest offshore wind energy production sites. The largest generated dataset has 90,000 training examples. A basic convolutional neural network for object detection, that is only trained on this synthetic data, confidently detects offshore wind farms by minimising false detections in challenging environments. In addition, four sequential datasets are generated, demonstrating how the SyntEO approach can precisely define the dataset structure and influence the training process. SyntEO is thus a hybrid approach that creates an interface between expert knowledge and data-driven image analysis.

Vision based crown loss estimation for individual trees with remote aerial robots

With the capability of capturing high-resolution imagery data and the ease of accessing remote areas, aerial robots are becoming increasingly popular for forest health monitoring applications. For example, forestry tasks such as field surveys and foliar sampling which are generally manual and labour intensive can be automated with remotely controlled aerial robots. In this study, we propose two new online frameworks to quantify and rank the severity of individual tree crown loss. The real-time crown loss estimation (RTCLE) model localises and classifies individual trees into their respective crown loss percentage bins. Experiments are conducted to investigate if synthetically generated tree images can be used to train the RTCLE model as real images with diverse viewpoints are generally expensive to collect. Results have shown that synthetic data training helps to achieve a satisfactory baseline mean average precision (mAP) which can be further improved with just some additional real imagery data. We showed that the mAP can be increased approximately from 60% to 78% by mixing the real dataset with the generated synthetic data. For individual tree crown loss ranking, a two-step crown loss ranking (TSCLR) framework is developed to handle the inconsistently labelled crown loss data. The TSCLR framework detects individual trees before ranking them based on some relative crown loss severity measures. The tree detection model is trained with the combined dataset used in the RTCLE model training where we achieved an mAP of approximately 95% suggesting that the model generalises well to unseen datasets. The relative crown loss severity of each tree is estimated, with deep representation learning, by a probabilistic encoder from a fully trained variational autoencoder (VAE) model. The VAE is trained end-to-end to reconstruct tree images in a background agnostic way. Based on a conservative evaluation, the estimated crown loss severity from the probabilistic encoder generally showed moderate agreement with the expert’s estimation across all species of trees present in the dataset. All the software pipelines, the dataset, and the synthetic dataset generation can be found in the GitHub link.

Synthetic time series data generation for edge analytics

Background: Internet of Things (IoT) edge analytics enables data computation and storage to be available adjacent to the source of data generation at the IoT system. This method improves sensor data handling and speeds up analysis, prediction, and action. Using machine learning for analytics and task offloading in edge servers could minimise latency and energy usage. However, one of the key challenges in using machine learning in edge analytics is to find a real-world dataset to implement a more representative predictive model. This challenge has undeniably slowed down the adoption of machine learning methods in IoT edge analytics. Thus, the generation of realistic synthetic datasets can leverage the need to speed up methodological use of machine learning in edge analytics. Methods: We create synthetic data with features that are like data from IoT devices. We use an existing air quality dataset that includes temperature and gas sensor measurements. This real-time dataset includes component values for the Air Quality Index (AQI) and ppm concentrations for various polluting gases. We build a JavaScript Object Notation (JSON) model to capture the distribution of variables and the structure of this real dataset to generate the synthetic data. Based on the synthetic dataset and original dataset, we create a comparative predictive model. Results: Analysis of synthetic dataset predictive model shows that it can be successfully used for edge analytics purposes, replacing real-world datasets. There is no significant difference between the real-world dataset compared the synthetic dataset. The generated synthetic data requires no modification to suit the edge computing requirements. Conclusions: The framework can generate representative synthetic datasets based on JSON schema attributes. The accuracy, precision, and recall values for the real and synthetic datasets indicate that the logistic regression model is capable of successfully classifying data.

Cross-Domain Knowledge Transfer Using High Dynamic Range Imaging in Synthetic Datasets

Herein, we propose a framework for the generation of photorealistic synthetic datasets using High Dynamic Range Imaging (HDRI) that include all kinds of information computer vision algorithms need. Furthermore, by utilizing the proposed framework, we demonstrate cross-domain knowledge transfer in a semantic segmentation scenario. We found that deep neural networks trained with our synthetic images or with a mix of real and synthetic perform equal to or in cases better than those trained solely on real images. To our knowledge, this is the first work that uses HDRI to successfully transfer knowledge from the synthetic domain to the real world.

Systematic Design, Generation, and Application of Synthetic Datasets for Flow Cytometry.

Application of synthetic datasets in training and validation of analysis tools has led to improvements in many decision-making tasks in a range of domains from computer vision to digital pathology. Synthetic datasets overcome the constraints of real-world datasets, namely difficulties in collection and labeling, expense, time, and privacy concerns. In flow cytometry, real cell-based datasets are limited by properties such as size, number of parameters, distance between cell populations, and distributions and are often focused on a narrow range of disease or cell types. Researchers in some cases have designed these desired properties into synthetic datasets; however, operators have implemented them in inconsistent approaches, and there is a scarcity of publicly available, high-quality synthetic datasets. In this research, we propose a method to systematically design and generate flow cytometry synthetic datasets with highly controlled characteristics. We demonstrate the generation of two-cluster synthetic datasets with specific degrees of separation between cell populations, and of non-normal distributions with increasing levels of skewness and orientations of skew pairs. We apply our synthetic datasets to test the performance of a popular automated cell populations identification software, SPADE3, and define the region where the software performance decreases as the clusters get closer together. Application of the synthetic skewed dataset suggests the software is capable of processing non-normal data. We calculate the classification accuracy of SPADE3 with robustness not achievable with real-world datasets. Our approach aims to advance research toward generation of high-quality synthetic flow cytometry datasets and to increase their awareness among the community. The synthetic datasets can be used in benchmarking studies that critically evaluate cell population identification tools and help illustrate potential digital platform inconsistencies. These datasets have the potential to improve cell characterization workflows that integrate automated analysis in clinical diagnostics and cell therapy manufacturing.

Towards Generation of Synthetic Data Sets for Hybrid Conflict Modelling

Design proposal for an AI-driven military situational awareness application. Current events in Ukraine have emphasized the importance of the Information Environment in supporting military operations. Activities in the physical domain are processed in the virtual domain of computers and networks before being interpreted by the human cognitive domain where decisions are made. Commanders at all levels have an ever increasing amount of information of varying latency and reliability available to them from a variety of sources. These range from highly sophisticated and complex bespoke surveillance systems to individuals equipped with a smartphone and Internet connection. An effective commander must be able to assimilate all the information sources available to them and be able to visualise the battlespace in an accurate and timely manner. To assist them, there are already a wide variety of software applications capable of receiving multiple inputs and displaying the disposition of military forces within a mapping environment. However, these tools place the interpretation and analysis responsibilities with the human operator. This paper proposes an AI driven process in which the initial analysis and correlation function is conducted in real time and in response to data inputs from multiple sources. This presents the decision maker with a fused, correlated and predictive Common Operational Picture providing a clear information advantage.

A method for machine learning generation of realistic synthetic datasets for validating healthcare applications.

Digital health applications can improve quality and effectiveness of healthcare, by offering a number of new tools to users, which are often considered a medical device. Assuring their safe operation requires, amongst others, clinical validation, needing large datasets to test them in realistic clinical scenarios. Access to datasets is challenging, due to patient privacy concerns. Development of synthetic datasets is seen as a potential alternative. The objective of the paper is the development of a method for the generation of realistic synthetic datasets, statistically equivalent to real clinical datasets, and demonstrate that the Generative Adversarial Network (GAN) based approach is fit for purpose. A generative adversarial network was implemented and trained, in a series of six experiments, using numerical and categorical variables, including ICD-9 and laboratory codes, from three clinically relevant datasets. A number of contextual steps provided the success criteria for the synthetic dataset. A synthetic dataset that exhibits very similar statistical characteristics with the real dataset was generated. Pairwise association of variables is very similar. A high degree of Jaccard similarity and a successful K-S test further support this. The proof of concept of generating realistic synthetic datasets was successful, with the approach showing promise for further work.