Synthetic Data Research Articles

Synthetic Data in Human Analysis: A Survey.

Deep neural networks have become prevalent in human analysis, boosting the performance of applications, such as biometric recognition, action recognition, as well as person re-identification. However, the performance of such networks scales with the available training data. In human analysis, the demand for large-scale datasets poses a severe challenge, as data collection is tedious, time-expensive, costly and must comply with data protection laws. Current research investigates the generation of synthetic data as an efficient and privacy-ensuring alternative to collecting real data in the field. This survey introduces the basic definitions and methodologies, essential when generating and employing synthetic data for human analysis. We summarise current state-of-the-art methods and the main benefits of using synthetic data. We also provide an overview of publicly available synthetic datasets and generation models. Finally, we discuss limitations, as well as open research problems in this field. This survey is intended for researchers and practitioners in the field of human analysis.

Learning Keypoints for Robotic Cloth Manipulation Using Synthetic Data

Learning Keypoints for Robotic Cloth Manipulation Using Synthetic Data

Hilbert Curve Projection Distance for Distribution Comparison.

Distribution comparison plays a central role in many machine learning tasks like data classification and generative modeling. In this study, we propose a novel metric, called Hilbert curve projection (HCP) distance, to measure the distance between two probability distributions with low complexity. In particular, we first project two high-dimensional probability distributions using Hilbert curve to obtain a coupling between them, and then calculate the transport distance between these two distributions in the original space, according to the coupling. We show that HCP distance is a proper metric and is well-defined for probability measures with bounded supports. Furthermore, we demonstrate that the modified empirical HCP distance with the Lp cost in the d-dimensional space converges to its population counterpart at a rate of no more than O(n-1/2max{d,p}). To suppress the curse-of-dimensionality, we also develop two variants of the HCP distance using (learnable) subspace projections. Experiments on both synthetic and real-world data show that our HCP distance works as an effective surrogate of the Wasserstein distance with low complexity and overcomes the drawbacks of the sliced Wasserstein distance.

Unsupervised Test-Time Adaptation Learning for Effective Hyperspectral Image Super-Resolution With Unknown Degeneration.

Fusing a low-resolution hyperspectral image (HSI) with a high-resolution (HR) multi-spectral image has provided an effective way for HSI super-resolution (SR). The key lies on inferring the posteriori of the latent (i.e., HR) HSI using an appropriate image prior and the likelihood determined by the degeneration between the latent HSI and the observed images. However, in scenarios with complex imaging environments and various imaging scenes, the prior of HSIs can be prohibitively complicated and the degeneration is often unknown, which causes it difficult to accurately infer the posteriori of each latent HSI. To tackle this problem, we present an unsupervised test-time adaptation learning (UTAL) framework for HSI SR under unknown degeneration. Instead of directly modeling the complicated image prior, it first implicitly learns a content-agnostic prior shared across different images through supervisedly pre-training a mutual-guiding fusion module on extensive synthetic data. Then, it adapts the shared prior to those private characteristics in the latent HSI for posteriori inference through unsupervisedly learning a self-guiding adaptation module and a degeneration estimation network on two observed images in the test phase. Such a two-stage learning scheme models the complicated image prior in a divide-and-conquer manner, which eases the modeling difficulty and improves the prior accuracy. Moreover, the unknown degeneration can be estimated properly. Both of these two advantages empower us to accurately infer the posteriori of the latent HSI, thereby increasing the generalization performance in real applications. Additionally, in order to further mitigate the over-fitting in coping with more challenging cases (e.g., degenerations in both spectral and spatial domains are unknown) and speed up, we propose to meta-train UTAL on extensive synthetic SR tasks and solve it using an alternative optimization strategy such that UTAL learns to produce good generalization performance in real challenging cases with a small number of gradient descent steps. To verify the efficacy of UTAL, we evaluate it on HSI SR tasks with different unknown degenerations as well as some other HSI restoration tasks (e.g., compressive sensing), and report strong results superior to that of existing competitors.

Thinking in Categories: A Survey on Assessing the Quality for Time Series Synthesis

Time series data are widely used and provide a wealth of information for countless applications. However, some applications are faced with a limited amount of data, or the data cannot be used due to confidentiality concerns. To overcome these obstacles, time series can be generated synthetically. For example, electrocardiograms can be synthesized to make them available for building models to predict conditions such as cardiac arrhythmia without leaking patient information. Although many different approaches to time series synthesis have been proposed, evaluating the quality of synthetic time series data poses unique challenges and remains an open problem, as there is a lack of a clear definition of what constitutes a “good” synthesis. To this end, we present a comprehensive literature survey to identify different aspects of synthesis quality and their relationships. Based on this, we propose a definition of synthesis quality and a systematic evaluation procedure for assessing it. With this work, we aim to provide a common language and criteria for evaluating synthetic time series data. Our goal is to promote more rigorous and reproducible research in time series synthesis by enabling researchers and practitioners to generate high-quality synthetic time series data.

Deep-learning viscoelastic seismic inversion for mapping subsea permafrost

Marine seismic surveys can be used to map ice-bearing subsea permafrost on a large scale. However, current seismic processing technologies have limited capacity to image permafrost distribution at depth, mainly due to the low sensitivity of primary reflections and refractions to the velocity inversion found at the base of permafrost. Guided waves and multiples are more sensitive to the velocity variations below the top of permafrost, but they remain challenging to use in physics-based inversion approaches. A deep-learning-based seismic inversion has the potential to improve seismic imaging below the top of permafrost by automatically extracting information from all wave modes. We develop a multi-input neural network (NN) to estimate seismic velocities from marine seismic data. The network is trained on synthetic data generated from the representative distributions of the seismic properties of subsea permafrost. We find that our network can image large velocity contrasts and reversals in depth, typical of subsea permafrost. We use our network to estimate the P- and S-wave velocity and Q-factor models from a seismic line in the Beaufort Sea. The NN indicates highly discontinuous subsea permafrost with variable thickness in the area. Our work shows that deep-learning-based seismic inversion could become a cost-effective technology to map the distribution of subsea permafrost on a large scale and, more generally, high-velocity geologic layers located in shallow waters.

PINNslope: Seismic data interpolation and local slope estimation with physics informed neural networks

Interpolation of aliased seismic data constitutes a key step in a seismic processing workflow to obtain high-quality velocity models and seismic images. Building on the idea of describing seismic wavefields as a superposition of local plane waves, we propose to interpolate seismic data by using a physics informed neural network (PINN). In the proposed framework, two feed-forward neural networks are jointly trained using the local plane wave differential equation as well as the available data as two terms in the objective function: a primary network assisted by positional encoding is tasked with reconstructing the seismic data, whereas an auxiliary, smaller network estimates the associated local slopes. Results on synthetic and field data validate the effectiveness of the proposed method in handling aliased (coarsely sampled) data and data with large gaps. Our method compares favorably against a classic least-squares inversion approach regularized by the local plane-wave equation as well as a PINN-based approach with a single network and precomputed local slopes. We find that introducing a second network to estimate the local slopes, whereas at the same time interpolating the aliased data enhances the overall reconstruction capabilities and convergence behavior of the primary network. Moreover, an additional positional encoding layer embedded as the first layer of the wavefield network confers to the network the ability to converge faster, improving the accuracy of the data term.

Influence of strong directional sources in ambient seismic imaging using traffic-induced noise

Continuously moving seismic sources, such as vehicles and train cars, play a crucial role as passive sources for nondestructive exploration in urban areas. Seismic interferometry is commonly used in field data acquisition and processing using linear arrays. However, the simultaneous recording of high-energy noise, excited by buildings and factories, cannot be overlooked. We simulate moving-source seismic records with strong noises using orthogonal arrays. When strong noise originates from specific angles, the energy-phase velocity curves of arrays in different directions significantly diverge. We determine that an L-shaped array, formed by orthogonal arrays, can effectively mitigate this effect, yielding more consistent results. Nonetheless, spatial constraints often preclude the deployment of L-shaped arrays. To mitigate this issue, we develop a new parallel observation system. Field tests conducted on a main road validate that the new array is comparable with the L-shaped array in terms of dispersion extraction. Similar to synthetic data, the phase velocity extracted from the linear array field data is found to be unreliable. Drilling data align well with the inversion results of the parallel observation system. Given the challenges of urban traffic-induced signal acquisition, deploying multiazimuth arrays to minimize noise impact is essential. Considering the spatial limitations, the convenient parallel observation system emerges as a good choice.

Sparse Generalized Canonical Correlation Analysis: Distributed Alternating Iteration-Based Approach.

Sparse canonical correlation analysis (CCA) is a useful statistical tool to detect latent information with sparse structures. However, sparse CCA, where the sparsity could be considered as a Laplace prior on the canonical variates, works only for two data sets, that is, there are only two views or two distinct objects. To overcome this limitation, we propose a sparse generalized canonical correlation analysis (GCCA), which could detect the latent relations of multiview data with sparse structures. Specifically, we convert the GCCA into a linear system of equationsand impose $\ell _1$ minimization penalty to pursue sparsity. This results in a nonconvex problem on the Stiefel manifold. Based on consensus optimization, a distributed alternating iteration approach is developed, and consistency is investigated elaborately under mild conditions. Experiments on several synthetic and real-world data sets demonstrate the effectiveness of the proposed algorithm.

Identification of Reaction Network Hypotheses for Complex Feedstocks from Spectroscopic Measurements with Minimal Human Intervention.

In this work, we detail an automated reaction network hypothesis generation protocol for processes involving complex feedstocks where information about the species and reactions involved is unknown. Our methodology is process agnostic and can be utilized in any reactive process with spectroscopic measurements that provide information on the evolution of the components in the mixture. We decompose the mixture spectra to obtain spectroscopic signatures of the individual components and use a 1-D convolutional neural network to automatically identify functional groups indicated by them. We employ atom-atom mapping to automatically recover reaction rules that are applied on candidate molecules identified from chemistry databases through fingerprint similarity. The method is tested on synthetic data and on spectroscopic measurements of lab-scale batch hydrothermal liquefaction (HTL) of biomass to determine the accuracy of prediction across datasets of varying complexities. Our methodology is able to identify reaction network hypotheses containing reaction networks close to the ground truth in the case of synthetic data, and we are also able to recover candidate molecules and reaction networks close to the ones reported in the previous literature studies for biomass pyrolysis.

Inversion of shear wave slowness and attenuation from dipole acoustic logging data based on an equivalent tool theory

The presence of a sonic logging tool affects the propagation characteristics of dipole-flexural waves in a fluid-filled borehole. As a result, the tool effect should be considered during the processing and inversion of the dipole-flexural waves for the acoustic properties of a shear wave (S wave). We show that the presence of the tool not only changes the slowness dispersion of the flexural wave but also results in a shift of its frequency-dependent attenuation toward lower frequencies. The equivalent tool theory (ETT) can realistically model the effect of an acoustic tool on guided wave propagation. With the ETT framework, we develop a unified workflow for estimating the S-wave slowness and attenuation from dipole array acoustic waveforms. The workflow is an extension of the model-based dispersive processing of dipole logging data. Our workflow involves two inversion procedures. By matching the actual dispersion data with the theoretical dispersion predicted by the ETT, we can invert the S-wave slowness and tool parameters. With the estimated S-wave slowness and other borehole parameters, we further calculate the partition coefficients of flexural waves using the ETT. Based on the concept of energy partitioning, we formulate a linear optimization problem and invert for the S-wave attenuation from the frequency-dependent dipole attenuation. We use synthetic and field data to demonstrate the effectiveness and applicability of the developed method. The results show that it is important to incorporate the tool effect when processing dipole acoustic logging data for S-wave slowness and attenuation, especially in a fast formation.

Significance of Dynamic Axial Stretching on Estimating Biomechanical Behavior and Properties of the Human Ascending Aorta.

Contrary to most vessels, the ascending thoracic aorta (ATA) not only distends but also elongates in the axial direction. The purpose of this study is to investigate the biomechanical behavior of the ascending thoracic aorta (ATA) in response to dynamic axial stretching during the cardiac cycle. In addition, the implications of neglecting this dynamic axial stretching when estimating the constitutive model parameters of the ATA are investigated. The investigations were performed through in silico simulations by assuming a Gasser-Ogden-Holzapfel (GOH) constitutive model representative of ATA tissue material. The GOH model parameters were obtained from biaxial tests performed on four human ATA tissues in a previous study. Pressure-diameter curves were simulated as synthetic data to assess theeffect of neglecting dynamic axial stretching on estimating constitutive model parameters. Our findings reveal a significant increase in axial stress (~ 16%) and stored strain energy (~ 18%) in the vessel when dynamic axial stretching is considered, as opposed to assuming a fixed axial stretch. All but one artery showed increased volume compliance while considering a dynamic axial stretching condition. Furthermore, we observe a notable difference in the estimated constitutive model parameters when dynamic axial stretching of the ATA is neglected, compared to the ground truth model parameters. These results underscore the critical importance of accounting for axial deformations when conducting in vivo biomechanical characterization of the ascending thoracic aorta.

Swarm intelligence for full Stokes dynamic imaging reconstruction of interferometric data

In very long baseline interferometry (VLBI), the combination of multiple antennas permits the synthesis of a virtual telescope with a larger diameter and consequently higher resolution than the individual antennas . However, due to the sparse nature of the array recovering an image from the observed data is a challenging ill-posed inverse problem. The VLBI community is interested in not only recovering an image in total intensity from interferometric data, but also in obtaining results in the polarimetric and the temporal domain. Only a few algorithms are able to work in all these domains simultaneously . In particular, the algorithms based on optimization that consider various penalty terms specific to static total intensity imaging, time-variability and polarimetry are restricted to grids in the domain of the objective function. In this work we present a novel algorithm, multiobjective particle swarm optimization (MO-PSO), that is able to recover the optimal weights without any space-gridding, and to obtain the marginal contribution of each of the playing terms. To this end, we utilized multiobjective optimization together with particle swarm metaheuristics. We let the swarm of weights converge to the best position. We evaluate our algorithm with synthetic data sets that are representative for the main science targets and instrumental configuration of the Event Horizon Telescope Collaboration (EHTC) and its planned successors. We successfully recover the polarimetric, static, and time-dynamic signature of the ground truth movie, even with relative sparsity, and a set of realistic data corruptions. We have built a novel, fast, hyperparameter space gridding-free algorithm that successfully recovers static and dynamic polarimetric reconstructions. Compared to regularized maximum likelihood (RML) methods, it avoids the need for parameter surveys, and it is not limited to the number of pixels, unlike recently proposed multiobjective imaging algorithms. Hence, this technique is a novel useful alternative tool to characterize full Stokes time-(in)dependent signatures in a VLBI data set robustly with a minimal set of user-based choices.

A Seq2Seq transformation strategy for generalizing a pre-trained model in anomaly detection of rolling element bearings

The monitoring of the health of Rolling Element Bearings (REBs) in the rolling mill process was recently automated through signal processing and machine learning technologies. However, large data sets are required, and existing methods require precise labels. Moreover, the scarcity of equipment anomalies challenges their practical identification. In addition, machine learning approaches require human resources to manage machine learning models because all REBs in the production line are not identical. Thus, we propose herein a Seq2Seq Transformation Strategy inspired by the accurate performance of such the methods in Natural Language Processing (NLP) and Computer Visioning (CV). Our strategy aims to use an anomaly detection model trained on one signal to detect anomalies in other signals, reducing the need for model management in practical scenarios. Specifically, we considered time series vibration signals collected from the REBs to be analogous to linguistic data, which are sequential and time variant. In essence, our strategy generates synthetic data from a vibration signal represented as the target domain by using a transformer. Subsequently, a model trained by another signal called the source domain is employed to detect the conditions of the target domain. The goal of transformation is to minimize the differences between the distributions of the synthetic and source data. We assumed that the target conditions could be identified from the synthetic data by the source model if the detection model trained on the source domain was precise. Our strategy was experimentally evaluated on six large, unlabeled data sets collected over 1 year from a rolling mill production line in a steel factory. The strategy significantly improved the accuracy of anomaly detection, which had initially been unacceptable when the source model was directly applied to the detection of the target conditions.

Feasibility of acousto-electric tomography

In acousto-electric tomography (AET) the goal is to reconstruct the electric conductivity in a domain from electrostatic boundary measurements of corresponding currents and voltages, while the domain is perturbed by a time-dependent acoustic wave, thus taking advantage of the acousto-electric effect. We approach the AET reconstruction in two steps: First, the interior power density is obtained from boundary measurements by solving a linear inverse and ill-posed problem; second, the interior conductivity is reconstructed from the power density by solving a non-linear and well-posed problem. Mathematically these inverse problems are fairly well understood, and reconstruction methods work well on synthetic data. This is in contrast to experimental findings. An effect can indeed be observed and data can be collected. However, the acousto-electric coupling is very weak, and consequently, the change in the measured voltage due to the acoustic perturbation might be too small compared to the background noise for viable reconstructions. In this paper, we take one step towards understanding the feasibility of AET. We provide an in-silico model of the coupled physics scenario based on standard models for the individual phenomena. Moreover, we formulate and implement numerically a full reconstruction method for the inverse problem via the two steps. We perform computational experiments with realistically chosen parameters from the context of medical imaging. The focus is on understanding the role of the acousto-electric coupling parameter and the signal-to-noise ratio (SNR). The critical signal strength is analyzed and the omnipresent Johnson–Nyquist noise is estimated. We obtain both positive and negative findings; we can reconstruct features even under severe noise conditions, but we also find that the SNR one is likely to face in practice is too low to obtain useful reconstructions.

The utility of TerraSAR-X, TanDEM-X, and PAZ for studying global volcanic activity: Successes, challenges, and future prospects

TerraSAR-X (TSX), TanDEM-X (TDX), and PAZ Synthetic Aperture Radar data have been used at over 120 volcanoes to assess surface characteristics and change over time. We examine previous work, adding additional examples to understand where and when these data are most useful for volcanology. We focus on volcanoes as part of the Committee on Earth Observation Satellites (CEOS) Volcano Demonstrator Project. TSX/TDX/PAZ data provide a valuable means of detecting small surface changes from amplitude images and topographic changes from bistatic TSX/TDX data. For short temporal and perpendicular baselines, TDX/TSX/PAZ can also provide useful deformation data, even in presence of vegetation. No global background mission currently acquires TSX/TDX/PAZ data at volcanoes: 70 % high spatial resolution data, limiting their suitability for studying pre-eruptive unrest. Coordinated targeting by SAR constellations of priority volcanoes would provide data and insights valuable for forecasting eruptions and associated hazards.

Solar wind data analysis aided by synthetic modeling: A better understanding of plasma frame variations from temporal data

In situ measurements of the solar wind, a turbulent and anisotropic plasma flow originating at the Sun, are mostly carried out by single spacecraft, resulting in one-dimensional time series. The conversion of these measurements to the spatial frame of the plasma is a great challenge, but it is required for direct comparison of the measurements with magnetohydrodynamic turbulence theories. We present a tool kit based on the synthetic modeling of solar wind fluctuations as two-dimensional noise maps with adjustable spectral and power anisotropy that can help with the temporal-spatial conversion of real data. Specifically, by following the spacecraft trajectory through a noise map (relative velocity and angle relative to some mean magnetic field) with properties tuned to mimic those of the solar wind, the likelihood that the temporal data fluctuations represent parallel or perpendicular fluctuations in the plasma frame can be quantified by correlating structure functions of the noise map. Synthetic temporal data can also be generated, which can provide a testing ground for analysis applied to the solar wind data. We demonstrate this tool by investigating Parker Solar Probe's seventh encounter trajectory and data, and we showcase several possible ways in which it can be used. We find that whether temporal variations in the spacecraft frame come from parallel or perpendicular variations in the plasma frame strongly depends on the spectral and power anisotropy of the measured wind. Data analysis assisted by such underlying synthetic models as presented here could open up new ways to interpret measurements in the future, specifically in the more reliable determination of plasma frame quantities from temporal measurements.

Submarine cable positioning using a residual convolutional neural network based on magnetic features

Accurate positioning is important for improving the efficiency of repairing submarine cables and reducing the related repair costs. The magnetic anomaly produced by a submarine cable can be used to estimate its vertical and horizontal positions. A novel approach using magnetic data for estimating the position of submarine cables based on the 1D residual convolutional neural network (RCNN) is investigated. Infinitely long ferromagnetic cylinder models with different parameters are used to generate data sets for model training and testing. Tests on noisy synthetic data sets show that the developed 1D RCNN method can capture detailed features related to the magnetic source position information, which is more accurate than the conventional Euler method in estimating the position of submarine cables. The developed 1D RCNN method has also been successfully applied to processing field data. Furthermore, the processing workflow of our 1D RCNN method is less noise-sensitive compared with the conventional Euler method. The proposed 1D RCNN method and its workflow open a new window for estimating the position of submarine cables using magnetic data.

Bayesian linearized amplitude variation with offset and azimuth inversion and uncertainty analysis in horizontal transversely isotropic media

AbstractThe stratum can be modelled as a horizontal transversely isotropic medium when a single set of vertically parallel fractures embedded in an isotropic background medium, which facilitates efficient study for fractured reservoirs. Elastic parameters and fracture weaknesses are important parameters to describe the characteristics of fractured reservoirs, and seismic inversion plays a significant role in parameters estimation. The commonly used deterministic inversion methods do not fully utilize the prior information and fails to present the uncertainty analysis of inversion results. To address these shortcomings, we propose a Bayesian linearized amplitude variation with offset and azimuth inversion method tailored for horizontal transversely isotropic media, enabling a more robust analysis of uncertainty. Within the framework of Bayesian inversion, the proposed method successfully derives analytical expressions for the posterior mean and covariance of both elastic parameters and fracture weaknesses. The response characteristics of the anisotropic reflection coefficient are analysed, and it is found that the perturbations of elastic parameters have a greater effect on reflection coefficient compared to fracture weaknesses. Synthetic data examples confirm that the accuracy of estimated P‐ and S‐wave velocities and density surpasses that of fracture weaknesses, and the proposed method still performs well for the case of moderate noise. A field data example demonstrates that the inverted profiles agree well with the logging curve, and the estimated fracture weaknesses display significantly high values in the reservoir area. The estimated reservoir parameters not only contribute to a more accurate representation of the fractured gas‐bearing reservoir but also provide insights into the target gas reservoir through its posterior distribution. Both synthetic and field data examples demonstrate the stability and reliability of the proposed method in characterizing fractured reservoirs. We determine that the proposed method provides an available tool for nuanced evaluation of uncertainty for the inversion results, and it is helpful for the fine description of fractured hydrocarbon‐bearing reservoirs.

Assessing the Efficacy of Synthetic Optic Disc Images for Detecting Glaucomatous Optic Neuropathy Using Deep Learning.

Deep learning architectures can automatically learn complex features and patterns associated with glaucomatous optic neuropathy (GON). However, developing robust algorithms requires a large number of data sets. We sought to train an adversarial model for generating high-quality optic disc images from a large, diverse data set and then assessed the performance of models on generated synthetic images for detecting GON. A total of 17,060 (6874 glaucomatous and 10,186 healthy) fundus images were used to train deep convolutional generative adversarial networks (DCGANs) for synthesizing disc images for both classes. We then trained two models to detect GON, one solely on these synthetic images and another on a mixed data set (synthetic and real clinical images). Both the models were externally validated on a data set not used for training. The multiple classification metrics were evaluated with 95% confidence intervals. Models' decision-making processes were assessed using gradient-weighted class activation mapping (Grad-CAM) techniques. Following receiver operating characteristic curve analysis, an optimal cup-to-disc ratio threshold for detecting GON from the training data was found to be 0.619. DCGANs generated high-quality synthetic disc images for healthy and glaucomatous eyes. When trained on a mixed data set, the model's area under the receiver operating characteristic curve attained 99.85% on internal validation and 86.45% on external validation. Grad-CAM saliency maps were primarily centered on the optic nerve head, indicating a more precise and clinically relevant attention area of the fundus image. Although our model performed well on synthetic data, training on a mixed data set demonstrated better performance and generalization. Integrating synthetic and real clinical images can optimize the performance of a deep learning model in glaucoma detection. Optimizing deep learning models for glaucoma detection through integrating DCGAN-generated synthetic and real-world clinical data can be improved and generalized in clinical practice.