Synthetic Data Research Articles

Anonymization: The imperfect science of using data while preserving privacy.

Information about us, our actions, and our preferences is created at scale through surveys or scientific studies or as a result of our interaction with digital devices such as smartphones and fitness trackers. The ability to safely share and analyze such data is key for scientific and societal progress. Anonymization is considered by scientists and policy-makers as one of the main ways to share data while minimizing privacy risks. In this review, we offer a pragmatic perspective on the modern literature on privacy attacks and anonymization techniques. We discuss traditional de-identification techniques and their strong limitations in the age of big data. We then turn our attention to modern approaches to share anonymous aggregate data, such as data query systems, synthetic data, and differential privacy. We find that, although no perfect solution exists, applying modern techniques while auditing their guarantees against attacks is the best approach to safely use and share data today.

Variational autoencoders and transformers for multivariate time-series generative modeling and forecasting: Applications to vortex-induced vibrations

This study employs a data-driven approach to studying physical system vibrations, focusing on two main aspects: using variational autoencoders (VAEs) to generate physical data (i.e. data “similar” to those obtained via real-world processes) and using transformers in order to continuously forecast flexible body nonstationary vibrations (2D time-series) in time–space using information from sparse sensors on the body (observers). A VAE is trained on vortex-induced vibrations (VIV) data collected from experiments conducted by the authors and is then tasked with generating synthetic VIV data similar to the experimental. The synthetic data are then used to train a transformer architecture whose objective is to continuously forecast the vibrations in time–space using sparse observations. The transformer (which has never seen real data) is tested against real experiments and its performance is compared to that of the same architecture trained on real data. In doing so, the ability of VAEs to generate data which preserve their training data’s intrinsic properties (i.e. physicality) is evaluated. Finally a comparison between the forecasting performance of the transformer architecture, an LSTM, and a DNN is presented.

Predictive modelling of freeway space utilising clinical history, normalised muscle activity, dental occlusion, and mandibular movement analysis

This study aimed to predict dental freeway space by examining the clinical history, habits, occlusal parameters, mandibular hard tissue movement, soft tissue motion, muscle activity, and temporomandibular joint function of 66 participants. Data collection involved video-based facial landmark tracking, mandibular electrognathography, surface electromyography of mandibular range of motion, freeway space, chewing tasks, phonetic expressions, joint vibration analysis, and 3D jaw scans of occlusion. This resulted in a dataset of 121 predictor features, with freeway space as the target variable. Six models were trained on synthetic data ranging from 500 to 25,000 observations, with 65 original observations reserved for testing: Linear Regression, Random Forest, CatBoost Regressor, XGBoost Regressor, Multilayer Perceptron Neural Network (MPNN), and TabNet. Explainable AI indicated that key predictors of freeway space included phonetics, resting temporalis muscle activity, mandibular muscle activity during clenching, body weight, mandibular hard tissue lateral displacements, and dental arch parameters. CatBoost excelled with a test error of 0.65 mm using 5000 synthetic data points, while a refined MPNN achieved the best performance with 25,000 synthetic data points and 121 unique predictors, yielding an absolute error of 0.43 mm on the 65 original observations.

An attention-based deep learning model for phase-resolved wave prediction

Abstract Phase-resolved wave prediction capability, even if only over two wave periods in advance, is of value for optimal control of wave energy converters (WECs), resulting in a dramatic increase in power generation efficiency. Previous studies on wave-by-wave predictions have shown that an Artificial Neural Network (ANN) model can outperform the traditional linear wave theory-based model in terms of both prediction accuracy and prediction horizon when using synthetic wave data. However, the prediction performance of ANN models is significantly reduced by the varying wave conditions and buoy positions that occur in the field. To overcome these limitations, a novel wave prediction method is developed based on the neural network with an attention mechanism. This study validates the new model using wave data measured at sea. The model utilizes past time histories of three Sofar Spotter wave buoys at upwave locations to predict the vertical motion of a Datawell Waverider-4 at a downwave location. The results show that the attention-based neural network model is capable of capturing the slow variation in the displacement of the buoys, which reduces the prediction error compared to a standard ANN and Long Short-Term Memory (LSTM) model.

Neural patient-specific 3D-2D registration in laparoscopic liver resection.

Augmented reality guidance in laparoscopic liver resection requires the registration of a preoperative 3D model to the intraoperative 2D image. However, 3D-2D liver registration poses challenges owing to the liver's flexibility, particularly in the limited visibility conditions of laparoscopy. Although promising, the current registration methods are computationally expensive and often necessitate manual initialisation. The first neural model predicting the registration (NM) is proposed, represented as 3D model deformation coefficients, from image landmarks. The strategy consists in training a patient-specific model based on synthetic data generated automatically from the patient's preoperative model. A liver shape modelling technique, which further reduces time complexity, is also proposed. The NM method was evaluated using the target registration error measure, showing an accuracy on par with existing methods, all based on numerical optimisation. Notably, NM runs much faster, offering the possibility of achieving real-time inference, a significant step ahead in this field. The proposed method represents the first neural method for 3D-2D liver registration. Preliminary experimental findings show comparable performance to existing methods, with superior computational efficiency. These results suggest a potential to deeply impact liver registration techniques.

Technical Note—Near-Optimal Bayesian Online Assortment of Reusable Resources

Near-Optimal Bayesian Online Assortment of Reusable Resources Motivated by rental services in e-commerce, we consider revenue maximization in the online assortment of reusable resources for different types of arriving consumers. We design competitive online algorithms compared with the optimal online policy in the Bayesian setting, where consumer types are drawn independently from known heterogeneous distributions over time. In scenarios with large initial inventories, our main result is a near-optimal competitive algorithm for reusable resources. Our algorithm relies on an expected linear programming (LP) benchmark, solves this LP, and simulates the solution through independent randomized rounding. The main challenge is achieving inventory feasibility efficiently using these simulation-based algorithms. To address this, we design discarding policies for each resource, balancing inventory feasibility and revenue loss. Discarding a unit of a resource impacts future consumption of other resources, so we introduce postprocessing assortment procedures to design and analyze our discarding policies. Additionally, we present an improved competitive algorithm for nonreusable resources and evaluate our algorithms using numerical simulations on synthetic data.

Real-Time Dual-Parameter Full-Waveform Inversion of GPR Data Based on Robust Deep Learning

Summary Ground penetrating radar (GPR) is becoming an increasingly important tool for understanding the shallow electrical structures of the earth and planets due to its adaptability to harsh detection environments, efficient data acquisition and accurate detection results. GPR full-waveform can simultaneously constrain the permittivity and resistivity of the medium, providing more comprehensive geophysical information and reducing the non-uniqueness of inversion. However, given the highly non-linear inverse problem and the massive data resulted from high temporal and spatial samplings, traditional full-waveform inversion algorithms are prohibitively costly. Inspired by Google's vision semantic segmentation system, we develop a robust deep learning-guided network that integrates geology and geophysics knowledge to support the real-time translation of zero-offset GPR data into dual-parameter electrical structures. We test our proposed network using synthetic data, which demonstrates that the algorithm can provide an accurate dual-parameter electrical model from a GPR sounding in milliseconds on a common laptop PC, exhibiting high robustness and adaptability to noise interference and extreme values of model parameters. We also apply our network to field data gathered for pollutant investigation in the US. The resulting dual-parameter structure provides a more comprehensive and realistic depiction of subsurface electrical properties and reveals the migration and aging of pollutants. Our algorithm's real-time and accurate advantages are expected to further unleash the potential of GPR technology and enable it to play a more significant role in earth and planetary exploration.

Quantum‐Noise‐Driven Generative Diffusion Models

AbstractGenerative models realized with Machine Learning (ML) techniques are powerful tools to infer complex and unknown data distributions from a finite number of training samples in order to produce new synthetic data. Diffusion Models (DMs) are an emerging framework that have recently overcome Generative Adversarial Networks (GANs) in creating high‐quality images. Here, is proposed and discussed the quantum generalization of DMs, i.e., three Quantum‐Noise‐Driven Generative Diffusion Models (QNDGDMs) that could be experimentally tested on real quantum systems. The idea is to harness unique quantum features, in particular the non‐trivial interplay among coherence, entanglement, and noise that the currently available noisy quantum processors do unavoidably suffer from, in order to overcome the main computational burdens of classical diffusion models during inference. Hence, the suggestion is to exploit quantum noise not as an issue to be detected and solved but instead as a beneficial key ingredient to generate complex probability distributions from which a quantum processor might sample more efficiently than a classical one. Three examples of the numerical simulations are also included for the proposed approaches. The results are expected to pave the way for new quantum‐inspired or quantum‐based generative diffusion algorithms addressing tasks as data generation with widespread real‐world applications.

Identifying topologically associating domains using differential kernels.

Chromatin is a polymer complex of DNA and proteins that regulates gene expression. The three-dimensional (3D) structure and organization of chromatin controls DNA transcription and replication. High-throughput chromatin conformation capture techniques generate Hi-C maps that can provide insight into the 3D structure of chromatin. Hi-C maps can be represented as a symmetric matrix [Formula: see text], where each element represents the average contact probability or number of contacts between chromatin loci i and j. Previous studies have detected topologically associating domains (TADs), or self-interacting regions in [Formula: see text] within which the contact probability is greater than that outside the region. Many algorithms have been developed to identify TADs within Hi-C maps. However, most TAD identification algorithms are unable to identify nested or overlapping TADs and for a given Hi-C map there is significant variation in the location and number of TADs identified by different methods. We develop a novel method to identify TADs, KerTAD, using a kernel-based technique from computer vision and image processing that is able to accurately identify nested and overlapping TADs. We benchmark this method against state-of-the-art TAD identification methods on both synthetic and experimental data sets. We find that the new method consistently has higher true positive rates (TPR) and lower false discovery rates (FDR) than all tested methods for both synthetic and manually annotated experimental Hi-C maps. The TPR for KerTAD is also largely insensitive to increasing noise and sparsity, in contrast to the other methods. We also find that KerTAD is consistent in the number and size of TADs identified across replicate experimental Hi-C maps for several organisms. Thus, KerTAD will improve automated TAD identification and enable researchers to better correlate changes in TADs to biological phenomena, such as enhancer-promoter interactions and disease states.

A novel approach to brain tumor detection using K-Means++, SGLDM, ResNet50, and synthetic data augmentation

Introduction: Brain tumors are abnormal cell growths in the brain, posing significant treatment challenges. Accurate early detection using non-invasive methods is crucial for effective treatment. This research focuses on improving the early detection of brain tumors in MRI images through advanced deep-learning techniques. The primary goal is to identify the most effective deep-learning model for classifying brain tumors from MRI data, enhancing diagnostic accuracy and reliability.Methods: The proposed method for brain tumor classification integrates segmentation using K-means++, feature extraction from the Spatial Gray Level Dependence Matrix (SGLDM), and classification with ResNet50, along with synthetic data augmentation to enhance model robustness. Segmentation isolates tumor regions, while SGLDM captures critical texture information. The ResNet50 model then classifies the tumors accurately. To further improve the interpretability of the classification results, Grad-CAM is employed, providing visual explanations by highlighting influential regions in the MRI images.Result: In terms of accuracy, sensitivity, and specificity, the evaluation on the Br35H::BrainTumorDetection2020 dataset showed superior performance of the suggested method compared to existing state-of-the-art approaches. This indicates its effectiveness in achieving higher precision in identifying and classifying brain tumors from MRI data, showcasing advancements in diagnostic reliability and efficacy.Discussion: The superior performance of the suggested method indicates its robustness in accurately classifying brain tumors from MRI images, achieving higher accuracy, sensitivity, and specificity compared to existing methods. The method's enhanced sensitivity ensures a greater detection rate of true positive cases, while its improved specificity reduces false positives, thereby optimizing clinical decision-making and patient care in neuro-oncology.

A multi-domain adversarial transfer network for cross domain fault diagnosis under imbalanced data

In the intelligent fault diagnosis of rolling bearings, transfer learning methods extend the applicability of models to diverse working scenarios. However, real-world scenarios often suffer from data imbalance, which reduces diagnostic accuracy. To address this issue, this paper proposes a multi-domain adversarial transfer (MDAT) framework to enhance cross-domain fault diagnosis accuracy for rolling bearings under imbalanced data conditions. First, an enhanced information generation method is introduced to produce realistic and useable synthetic data to mitigate data imbalance. Subsequently, an adversarial multi-domain adaptation module is designed to learn invariant features across multiple domains. Finally, a domain reweighting method is proposed to improve domain alignment and enhance domain confusion. The effectiveness of the proposed method was validated through two case studies on rolling bearing fault diagnosis. The results demonstrated that MDAT achieved cross-domain diagnosis accuracies of 89.2% and 99.0% under imbalanced data conditions, confirming the effectiveness and superiority of the MDAT framework.

Robust unsupervised 5D seismic data reconstruction on both regular and irregular grid

Seismic data reconstruction in five dimensions (5D) has become a central focus in seismic data processing, addressing challenges posed by irregular sampling due to physical and budgetary constraints. Most traditional high-dimensional reconstruction methods commonly utilize the fast Fourier transform (FFT), requiring regular grids and preliminary 4D binning before 5D interpolation. Discrete Fourier transform and non-equidistant FFT can honour original irregular coordinates. However, when using exact locations, these methods become computationally expensive. This study introduces an unsupervised deep-learning methodology to learn a continuous function across sampling points in seismic data, facilitating reconstruction on both regular and irregular grids. The network comprises a multilayer perceptron (MLP) with linear layers and element-wise periodic activation functions. It excels at mapping input coordinates to corresponding seismic data amplitudes without relying on external training sets. The network’s intrinsic low-frequency bias is crucial in prioritizing acquiring self-similar features over high-frequency, incoherent ones during training. This characteristic mitigates incoherent noise in seismic data, including random and erratic components. To assess the robustness of the unsupervised reconstruction technique, we conduct comprehensive evaluations using synthetic data examples sampled both regularly and irregularly, as well as field-data examples with and without binning. The findings demonstrate the efficacy of the proposed deep-learning framework in achieving resilient and accurate seismic data reconstruction across diverse sampling scenarios.

Poldw: a Python code to denoise 3C seismic data with a new threshold-free polarization technique#xD;

We present a Python code that implements a novel threshold-free polarization strategy for removing random noise from three-component (3C) linearly polarized seismic data. The code, which we refer to as poldw (polarization denoising through windowing), uses closed-form formulas along sliding windows that span the data to determine the optimal rotation angles that allow the transfer of most of the signal energy to a given component. The denoised 3C data is obtained after canceling out the other two components, which are assumed to contain predominantly noise, and rotating back. The method is simple and efficient because it only requires setting the sliding window length. Synthetic and microseismic field data examples show the method’s effectiveness, which significantly improves the signal-to-noise ratio without the need for threshold-based polarization filters. Even so, these filters can be pipelined in the rotation-based strategy for additional noise removal if necessary. When the dataset contains non-linearly polarized data or significant non-random noise, the method is likely to fail. For robustness against non-Gaussian noise and outliers, poldw allows for the use of alternative norms like the L1- or L p-norms instead of the energy. In addition to the code, we provide a Jupyter notebook to illustrate the method step by step and reproduce the results of the field data example.

A high-dimensional additive nonparametric model

Nonparametric additive models are garnering increasing attention in applied research across fields like statistics and economics, attributed to their distinct interpretability, versatility, and their adeptness at addressing the curse of dimensionality. This paper introduces a novel and efficient fully Bayesian method for estimating nonparametric additive models, employing a band matrix smoothness prior. Our methodology leverages unobserved binary indicator parameters, promoting linearity in each additive component while allowing for deviations from it. We validate the efficacy of our approach through experiments on synthetic data derived from ten-component additive models, encompassing diverse configurations of linear, nonlinear, and zero function components. Additionally, the robustness of our algorithm is tested on high-dimensional models featuring up to one hundred components, and models correlated components. The practical utility and computational efficiency of our technique are further underscored by its application to two real-world datasets, showcasing its broad applicability and effectiveness in various scenarios.

Using early rejection Markov chain Monte Carlo and Gaussian processes to accelerate ABC methods

Approximate Bayesian computation (ABC) is a class of Bayesian inference algorithms that targets problems with intractable or unavailable likelihood functions. It uses synthetic data drawn from the simulation model to approximate the posterior distribution. However, ABC is computationally intensive for complex models in which simulating synthetic data is very expensive. In this article, we propose an early rejection Markov chain Monte Carlo (ejMCMC) sampler based on Gaussian processes to accelerate inference speed. We early reject samples in the first stage of the kernel using a discrepancy model, in which the discrepancy between the simulated and observed data is modeled by Gaussian process (GP). Hence, synthetic data is generated only if the parameter space is worth exploring. We demonstrate through theory, simulation experiments, and real data analysis that the new algorithm significantly improves inference efficiency compared to existing early-rejection MCMC algorithms. In addition, we employ our proposed method within an ABC sequential Monte Carlo (SMC) sampler. In our numerical experiments, we use examples of ordinary differential equations, stochastic differential equations, and delay differential equations to demonstrate the effectiveness of the proposed algorithm. We develop an R package that is available at https://github.com/caofff/ejMCMC.

Generating Synthetic MR Spectroscopic Imaging Data with Generative Adversarial Networks to Train Machine Learning Models.

To develop a new method to generate synthetic MR spectroscopic imaging (MRSI) data for training machine learning models. This study targeted routine MRI examination protocols with single voxel spectroscopy (SVS). A novel model derived from pix2pix generative adversarial networks was proposed to generate synthetic MRSI data using MRI and SVS data as inputs. T1- and T2-weighted, SVS, and reference MRSI data were acquired from healthy brains with clinically available sequences. The proposed model was trained to generate synthetic MRSI data. Quantitative evaluation involved the calculation of the mean squared error (MSE) against the reference and metabolite ratio value. The effect of the location of and the number of the SVS data on the quality of the synthetic MRSI data was investigated using the MSE. The synthetic MRSI data generated from the proposed model were visually closer to the reference. The 95% confidence interval (CI) of the metabolite ratio value of synthetic MRSI data overlapped with the reference for seven of eight metabolite ratios. The MSEs tended to be lower in the same location than in different locations. The MSEs among groups of numbers of SVS data were not significantly different. A new method was developed to generate MRSI data by integrating MRI and SVS data. Our method can potentially increase the volume of MRSI data training for other machine learning models by adding SVS acquisition to routine MRI examinations.

Generating synthetic signaling networks for in silico modeling studies

Predictive models of signaling pathways have proven to be difficult to develop. Traditional approaches to developing mechanistic models rely on collecting experimental data and fitting a single model to that data. This approach works for simple systems but has proven unreliable for complex systems such as biological signaling networks. Thus, there is a need to develop new approaches to create predictive mechanistic models of complex systems. To meet this need, we developed a method for generating artificial signaling networks that were reasonably realistic and thus could be treated as ground truth models. These synthetic models could then be used to generate synthetic data for developing and testing algorithms designed to recover the underlying network topology and associated parameters. We defined the reaction degree and reaction distance to measure the topology of reaction networks, especially to consider enzymes. To determine whether our generated signaling networks displayed meaningful behavior, we compared them with signaling networks from the BioModels Database. This comparison indicated that our generated signaling networks had high topological similarities with BioModels signaling networks with respect to the reaction degree and distance distributions. In addition, our synthetic signaling networks had similar behavioral dynamics with respect to both steady states and oscillations, suggesting that our method generated synthetic signaling networks comparable with BioModels and thus could be useful for building network evaluation tools.

Parameter estimation in a whole-brain network model of epilepsy: Comparison of parallel global optimization solvers.

The Virtual Epileptic Patient (VEP) refers to a computer-based representation of a patient with epilepsy that combines personalized anatomical data with dynamical models of abnormal brain activities. It is capable of generating spatio-temporal seizure patterns that resemble those recorded with invasive methods such as stereoelectro EEG data, allowing for the evaluation of clinical hypotheses before planning surgery. This study highlights the effectiveness of calibrating VEP models using a global optimization approach. The approach utilizes SaCeSS, a cooperative metaheuristic capable of parallel computation, to yield high-quality solutions without requiring excessive computational time. Through extensive benchmarking on synthetic data, our proposal successfully solved a set of different configurations of VEP models, demonstrating better scalability and superior performance against other parallel solvers. These results were further enhanced using a Bayesian optimization framework for hyperparameter tuning, with significant gains in terms of both accuracy and computational cost. Additionally, we added a scalable uncertainty quantification phase after model calibration, and used it to assess the variability in estimated parameters across different problems. Overall, this study has the potential to improve the estimation of pathological brain areas in drug-resistant epilepsy, thereby to inform the clinical decision-making process.

Detecting rhythmic spiking through the power spectra of point process model residuals.

Objective.Oscillations figure prominently as neurological disease hallmarks and neuromodulation targets. To detect oscillations in a neuron's spiking, one might attempt to seek peaks in the spike train's power spectral density (PSD) which exceed a flat baseline. Yet for a non-oscillating neuron, the PSD is not flat: The recovery period ("RP", the post-spike drop in spike probability, starting with the refractory period) introduces global spectral distortion. An established "shuffling" procedure corrects for RP distortion by removing the spectral component explained by the inter-spike interval (ISI) distribution. However, this procedure sacrifices oscillation-related information present in the ISIs, and therefore in the PSD. We asked whether point process models (PPMs) might achieve more selective RP distortion removal, thereby enabling improved oscillation detection.Approach.In a novel "residuals" method, we first estimate the RP duration (nr) from the ISI distribution. We then fit the spike train with a PPM that predicts spike likelihood based on the time elapsed since the most recent of any spikes falling within the preceding nrmilliseconds. Finally, we compute the PSD of the model's residuals.Main results.We compared the residuals and shuffling methods' ability to enable accurate oscillation detection with flat baseline-assuming tests. Over synthetic data, the residuals method generally outperformed the shuffling method in classification of true- versus false-positive oscillatory power, principally due to enhanced sensitivity in sparse spike trains. In single-unit data from the internal globus pallidus (GPi) and ventrolateral anterior thalamus (VLa) of a parkinsonian monkey -- in which alpha-beta oscillations (8-30 Hz) were anticipated -- the residuals method reported the greatest incidence of significant alpha-beta power, with low firing rates predicting residuals-selective oscillation detection.Significance.These results encourage continued development of the residuals approach, to support more accurate oscillation detection. Improved identification of oscillations could promote improved disease models and therapeutic technologies.

Explainable decomposition of nested dense subgraphs

AbstractDiscovering dense regions in a graph is a popular tool for analyzing graphs. While useful, analyzing such decompositions may be difficult without additional information. Fortunately, many real-world networks have additional information, namely node labels. In this paper we focus on finding decompositions that have dense inner subgraphs and that can be explained using labels. More formally, we construct a binary tree T with labels on non-leaves that we use to partition the nodes in the input graph. To measure the quality of the tree, we model the edges in the shell and the cross edges to the inner shells as a Bernoulli variable. We reward the decompositions with the dense regions by requiring that the model parameters are non-increasing. We show that our problem is NP-hard, even inapproximable if we constrain the size of the tree. Consequently, we propose a greedy algorithm that iteratively finds the best split and applies it to the current tree. We demonstrate how we can efficiently compute the best split by maintaining certain counters. Our experiments show that our algorithm can process networks with over million edges in few minutes. Moreover, we show that the algorithm can find the ground truth in synthetic data and produces interpretable decompositions when applied to real world networks.