Synthetic Data Distributions Research Articles

Adaptive Learning of the Latent Space of Wasserstein Generative Adversarial Networks

Generative models based on latent variables, such as generative adversarial networks (GANs) and variational auto-encoders (VAEs), have gained lots of interests due to their impressive performance in many fields. However, many data such as natural images usually do not populate the ambient Euclidean space but instead reside in a lower-dimensional manifold. Thus an inappropriate choice of the latent dimension fails to uncover the structure of the data, possibly resulting in mismatch of latent representations and poor generative qualities. Towards addressing these problems, we propose a novel framework called the latent Wasserstein GAN (LWGAN) that fuses the Wasserstein auto-encoder and the Wasserstein GAN so that the intrinsic dimension of the data manifold can be adaptively learned by a modified informative latent distribution. We prove that there exist an encoder network and a generator network in such a way that the intrinsic dimension of the learned encoding distribution is equal to the dimension of the data manifold. We theoretically establish that our estimated intrinsic dimension is a consistent estimate of the true dimension of the data manifold. Meanwhile, we provide an upper bound on the generalization error of LWGAN, implying that we force the synthetic data distribution to be similar to the real data distribution from a population perspective. Comprehensive empirical experiments verify our framework and show that LWGAN is able to identify the correct intrinsic dimension under several scenarios, and simultaneously generate high-quality synthetic data by sampling from the learned latent distribution.

Integrating Machine Learning Utility in Tabular Data Synthesizer Training using Loss Function Learning

Machine learning (ML) utility has been the main evaluation metrics for data synthesizers. However, because ML utility cannot be simply calculated, none of the previous synthesizers were trained to reach the same level of ML utility as a training objective. This study aims to integrate ML utility into data synthesizer training using a transformer-based model as a learned loss function. The transformer was trained to estimate ML utility of synthetic datasets, then it’s integrated by backpropagating the difference between estimated and expected value. The integration has significantly improved the average ML utility of LCT-GAN and Realtabformer. The ML utility of LCT-GAN improved by 0.0158 for Contraceptive dataset, 0.031 for Insurance dataset, and 0.0561 for Treatment dataset. The ML utility of Realtabformer improved by 0.02 for Contraceptive dataset and 0.0024 for Insurance dataset. The increase affects the dataset distribution, correlation between features, and privacy, but the direction varies. Correlation coefficients indicate that synthetic data distribution gets closer to real data as ML utility improves. In addition to ML utility integration, this study has also shown that patterns between rows in a dataset can be learned, so better synthesizers can be developed based on them.

Data-Driven Generative Model Aimed to Create Synthetic Data for the Long-Term Forecast of Gas Turbine Operation

Abstract The prediction of gas turbine (GT) future health state plays a strategic role in the current energy sector. However, training an accurate prognostic model is challenging in case of limited historical data (e.g., new installation). Thus, this paper develops a Generative Adversarial Network (GAN) model aimed to generate synthetic data that can be used for data augmentation. The GAN model includes two neural networks, i.e., a generator and a discriminator. The generator aims to generate synthetic data that mimic the real data. The discriminator is a binary classification network. During the training process, the generator is optimized to fool the discriminator in distinguishing between real and synthetic data. The real data employed in this paper were taken from the literature, gathered from three GTs, and refer to two quantities, i.e., corrected power output and compressor efficiency, which are tracked during several years. Three different analyses are presented to validate the reliability of the synthetic dataset. First, a visual comparison of real and synthetic data is performed. Then, two metrics are employed to quantitively evaluate the similarity between real and synthetic data distributions. Finally, a prognostic model is trained by only using synthetic data and then employed to predict real data. The results prove the high reliability of the synthetic data, which can be thus exploited to train a prognostic model. In fact, the prediction error of the prognostic model on the real data is lower than 2.5% even in the case of long-term prediction.

Constraint-Aware Learning for Fractional Flow Reserve Pullback Curve Estimation from Invasive Coronary Imaging.

Estimation of the fractional flow reserve (FFR) pullback curve from invasive coronary imaging is important for the intraoperative guidance of coronary intervention. Machine/deep learning has been proven effective in FFR pullback curve estimation. However, the existing methods suffer from inadequate incorporation of intrinsic geometry associations and physics knowledge. In this paper, we propose a constraint-aware learning framework to improve the estimation of the FFR pullback curve from invasive coronary imaging. It incorporates both geometrical and physical constraints to approximate the relationships between the geometric structure and FFR values along the coronary artery centerline. Our method also leverages the power of synthetic data in model training to reduce the collection costs of clinical data. Moreover, to bridge the domain gap between synthetic and real data distributions when testing on real-world imaging data, we also employ a diffusion-driven test-time data adaptation method that preserves the knowledge learned in synthetic data. Specifically, this method learns a diffusion model of the synthetic data distribution and then projects real data to the synthetic data distribution at test time. Extensive experimental studies on a synthetic dataset and a real-world dataset of 382 patients covering three imaging modalities have shown the better performance of our method for FFR estimation of stenotic coronary arteries, compared with other machine/deep learning-based FFR estimation models and computational fluid dynamics-based model. The results also provide high agreement and correlation between the FFR predictions of our method and the invasively measured FFR values. The plausibility of FFR predictions along the coronary artery centerline is also validated.

A Comparative Analysis of Data Synthesis Techniques to Improve Classification Accuracy of Raman Spectroscopy Data.

Raman spectra are examples of high dimensional data that can often be limited in the number of samples. This is a primary concern when Deep Learning frameworks are developed for tasks such as chemical species identification, quantification, and diagnostics. Open-source data are difficult to obtain and often sparse; furthermore, the collecting and curating of new spectra require expertise and resources. Deep generative modeling utilizes Deep Learning architectures to approximate high dimensional distributions and aims to generate realistic synthetic data. The evaluation of the data and the performance of the deep models is usually conducted on a per-task basis and provides no indication of an increase to robustness, or generalization, on a wider scale. In this study, we compare the benefits and limitations of a standard statistical approach to data synthesis (weighted blending) with a popular deep generative model, the Variational Autoencoder. Two binary data sets are divided into 3-fold to simulate small, limited samples. Synthetic data distributions are created per fold using the two methods and then augmented into the training of two Deep Learning algorithms, a Convolutional Neural Network and a Fully-Connected Neural Network. The goal of this study is to observe the trends in learning as synthetic data are continually augmented to the training data in increasing batches. To determine the impact of each synthetic method, Principal Component Analysis and the discrete Fréchet distance are implemented to visualize and measure the distance between the source and synthetic distributions along with the Machine Learning metric balanced accuracy for evaluating performance on imbalanced data.

Protecting query privacy with differentially private k-anonymity in location-based services

Nowadays, location-based services (LBS) are facilitating people in daily life through answering LBS queries. However, privacy issues including locationprivacy and queryprivacy arise at the same time. Existing works for protecting queryprivacy either work on trusted servers or fail to provide sufficient privacy guarantee. This paper combines the concepts of differential privacy and k-anonymity to propose the notion of differentially private k-anonymity (DPkA) for queryprivacy in LBS. We recognize the sufficient and necessary condition for the availability of 0-DPkA and present how to achieve it. For cases where 0-DPkA is not achievable, we propose an algorithm to achieve ??-DPkA with minimized ??. Extensive simulations are conducted to validate the proposed mechanisms based on real-life datasets and synthetic data distributions.

Performance reproducibility index for classification

A common practice in biomarker discovery is to decide whether a large laboratory experiment should be carried out based on the results of a preliminary study on a small set of specimens. Consideration of the efficacy of this approach motivates the introduction of a probabilistic measure, for whether a classifier showing promising results in a small-sample preliminary study will perform similarly on a large independent sample. Given the error estimate from the preliminary study, if the probability of reproducible error is low, then there is really no purpose in substantially allocating more resources to a large follow-on study. Indeed, if the probability of the preliminary study providing likely reproducible results is small, then why even perform the preliminary study? This article introduces a reproducibility index for classification, measuring the probability that a sufficiently small error estimate on a small sample will motivate a large follow-on study. We provide a simulation study based on synthetic distribution models that possess known intrinsic classification difficulties and emulate real-world scenarios. We also set up similar simulations on four real datasets to show the consistency of results. The reproducibility indices for different distributional models, real datasets and classification schemes are empirically calculated. The effects of reporting and multiple-rule biases on the reproducibility index are also analyzed. We have implemented in C code the synthetic data distribution model, classification rules, feature selection routine and error estimation methods. The source code is available at http://gsp.tamu.edu/Publications/supplementary/yousefi12a/.

Multiple-rule bias in the comparison of classification rules

There is growing discussion in the bioinformatics community concerning overoptimism of reported results. Two approaches contributing to overoptimism in classification are (i) the reporting of results on datasets for which a proposed classification rule performs well and (ii) the comparison of multiple classification rules on a single dataset that purports to show the advantage of a certain rule. This article provides a careful probabilistic analysis of the second issue and the 'multiple-rule bias', resulting from choosing a classification rule having minimum estimated error on the dataset. It quantifies this bias corresponding to estimating the expected true error of the classification rule possessing minimum estimated error and it characterizes the bias from estimating the true comparative advantage of the chosen classification rule relative to the others by the estimated comparative advantage on the dataset. The analysis is applied to both synthetic and real data using a number of classification rules and error estimators. We have implemented in C code the synthetic data distribution model, classification rules, feature selection routines and error estimation methods. The code for multiple-rule analysis is implemented in MATLAB. The source code is available at http://gsp.tamu.edu/Publications/supplementary/yousefi11a/. Supplementary simulation results are also included.

A boundary method for outlier detection based on support vector domain description

The support vector domain description (SVDD) is a popular kernel method for outlier detection, which tries to fit a class of data with a sphere and uses a few target objects to support its decision boundary. The problem is that even with a flexible Gaussian kernel function, the SVDD could sometimes generate such a loose decision boundary that the discrimination ability becomes poor. Therefore, a computationally intensive procedure called kernel whitening is often required to improve the performance. In this paper, we propose a simple post-processing method which tries to modify the SVDD boundary in order to achieve a tight data description with no need of kernel whitening. With the derivation of the distance between an object and its nearest boundary point in input space, the proposed method can efficiently construct a new decision boundary based on the SVDD boundary. The improvement from the proposed method is demonstrated with synthetic and real-world datasets. The results show that the proposed decision boundary can fit the shape of synthetic data distribution closely and achieves better or comparable classification performance on real-world datasets.

On the bogussing of tropical cyclones in numerical models: A comparison of vortex profiles

At the resolutions currently in use, and with the sparse oceanic data coverage, numerical analyses cannot adequately represent tropical cyclone circulations for use in numerical weather prediction models. In many cases there is no circulation present at all. Most numerical weather prediction centers therefore employ a “bogussing” scheme to force a tropical cyclone vortex into the numerical analysis. The standard procedure is to define a synthetic data distribution based on an analytically prescribed vortex, which is passed to the analysis scheme as a set of high quality observations.