DAGGER: Data AuGmentation GEneRative Framework for Time-Series Data in Data-Driven Smart Manufacturing Systems

Abstract

As industries transition into the Industry 4.0 paradigm, the relevance and interest in concepts like Digital Twin (DT) are at an all-time high. DTs offer direct avenues for industries to make more accurate predictions, rational decisions, and informed plans, ultimately reducing costs, increasing performance and productivity. Adequate operation of DTs in the context of smart manufacturing relies on an evolving data-set relating to the real-life object or process, and a means of dynamically updating the computational model to better conform to the data. This reliance on data is made more explicit when physics-based computational models are not available or difficult to obtain in practice, as it's the case in most modern manufacturing scenarios. For data-based model surrogates to "adequately" represent the underlying physics, the number of training data points must keep pace with the number of degrees of freedom in the model, which can be on the order of thousands. However, in niche industrial scenarios like the one in manufacturing applications, the availability of data is limited (on the order of a few hundred data points, at best), mainly because a manual measuring process typically must take place for a few of the relevant quantities, e.g., level of wear of a tool. In other words, notwithstanding the popular notion of big-data, there is still a stark shortage of ground-truth data when examining, for instance, a complex system's path to failure. In this work we present a framework to alleviate this problem via modern machine learning tools, where we show a robust, efficient and reliable pathway to augment the available data to train the data-based computational models.
 Small sample size data is a key limitation in performance in machine learning, in particular with very high dimensional data. Current efforts for synthetic data generation typically involve either Generative Adversarial Networks (GANs) or Variational AutoEncoders (VAEs). These, however, are are tightly related to image processing and synthesis, and are generally not suited for sensor data generation, which is the type of data that manufacturing applications produce. Additionally, GAN models are susceptible to mode collapse, training instability, and high computational costs when used for high dimensional data creation. Alternatively, the encoding of VAEs greatly reduces dimensional complexity of data and can effectively regularize the latent space, but often produces poor representational synthetic samples. Our proposed method thus incorporates the learned latent space from an AutoEncoder (AE) architecture into the training of the generation network in a GAN. The advantages of such scheme are twofold: \textbf{(\textit{i})} the latent space representation created by the AE reduces the complexity of the distribution the generator must learn, allowing for quicker discriminator convergence, and \textbf{(\textit{ii})} the structure in the sensor data is better captured in the transition from the original space to the latent space. Through time statistics (up to the fifth moment), ARIMA coefficients and Fourier series coefficients, we compare the synthetic data from our proposed AE+GAN model with the original sensor data. We also show that the performance of our proposed method is at least comparable with that of the Riemannian Hamiltonian VAE, which is a recently published data augmentation framework specifically designed to handle very small high dimensional data sets.