Time Series Imputation Research Articles

RAEF: An Imputation Framework Based on a Gated Regulator Autoencoder for Incomplete IIoT Time-Series Data

The number of intelligent applications available for IIoT environments is growing, but when the time-series data these applications rely on are incomplete, their performance suffers. Unfortunately, incomplete data are all too frequent to a phenomenon in the world of IIoT. A common workaround is to use imputation. However, the current methods are largely designed to reconstruct a single missing pattern, where a robust and flexible imputation framework would be able to handle many different missing patterns. Hence, the framework presented in this study, RAEF, is capable of processing multiple missing patterns. Based on a recurrent autoencoder, RAEF houses a novel neuron structure, called a gated regulator, which reduces the negative impact of different missing patterns. In a comparison of the state-of-the-art time-series imputation frameworks at a range of different missing rates, RAEF yielded fewer errors than all its counterparts.

A Stochastic Multivariate Irregularly Sampled Time Series Imputation Method for Electronic Health Records

Electronic health records (EHRs) can be very difficult to analyze since they usually contain many missing values. To build an efficient predictive model, a complete dataset is necessary. An EHR usually contains high-dimensional longitudinal time series data. Most commonly used imputation methods do not consider the importance of temporal information embedded in EHR data. Besides, most time-dependent neural networks such as recurrent neural networks (RNNs) inherently consider the time steps to be equal, which in many cases, is not appropriate. This study presents a method using the gated recurrent unit (GRU), neural ordinary differential equations (ODEs), and Bayesian estimation to incorporate the temporal information and impute sporadically observed time series measurements in high-dimensional EHR data.

MuSDRI: Multi-Seasonal Decomposition Based Recurrent Imputation for Time Series

Missing values are a common problem of time series such as sensor-generated data due to issues of data collection, transmission, and storage. To deal with this problem, various data imputation methods have been proposed. Among these methods, variants of recurrent neural networks (RNNs) have attracted a lot of attention as they provide a natural representation of temporal dynamics. Though such methods have been shown to achieve higher imputation accuracy, they mostly focus on the modeling of temporal dynamics from a short-term perspective while ignoring the long-term features such as trend and seasonal information embedded in time series data. In this paper, we investigate different paradigms of time series imputation methods to capture temporal dynamics from both long-term and short-term perspectives. In particular, we propose a new learning paradigm called MuSDRI which combines the time series decomposition method Seasonal Trend decomposition using Loess (STL) with RNNs. Specifically, STL is used to decompose long sequences of time series into trend, seasonal, and remainder components such that long-term dynamics can be revealed; RNNs are used to capture local patterns from the remainder component extracted by STL such that short-term dynamics can be refined. Moreover, MuSDRI is able to handle time series with multiple seasonal patterns by learning their contributions with respect to the imputation task. We evaluate the proposed imputation methods on three real-world datasets. Experimental results show that the imputation methods considering both the long-term and short-term dynamics achieve higher imputation accuracy, and among them MuSDRI in general achieves the best performance.

Generative Semi-supervised Learning for Multivariate Time Series Imputation

The missing values, widely existed in multivariate time series data, hinder the effective data analysis. Existing time series imputation methods do not make full use of the label information in real-life time series data. In this paper, we propose a novel semi-supervised generative adversarial network model, named SSGAN, for missing value imputation in multivariate time series data. It consists of three players, i.e., a generator, a discriminator, and a classifier. The classifier predicts labels of time series data, and thus it drives the generator to estimate the missing values (or components), conditioned on observed components and data labels at the same time. We introduce a temporal reminder matrix to help the discriminator better distinguish the observed components from the imputed ones. Moreover, we theoretically prove that, SSGAN using the temporal reminder matrix and the classifier does learn to estimate missing values converging to the true data distribution when the Nash equilibrium is achieved. Extensive experiments on three public real-world datasets demonstrate that, SSGAN yields a more than 15% gain in performance, compared with the state-of-the-art methods.

An unsupervised neural network approach for imputation of missing values in univariate time series data

SummaryHandling missing values in time series data plays a key role in predicting and forecasting, as complete and clean historical data help to achieve higher accuracy. Numerous research works are present in multivariate time series imputation, but imputation in univariate time series data is least considered due to correlated variables unavailability. This article aims to propose an iterative imputation algorithm by clustering univariate time series data, considering the trend, seasonality, cyclical, and residue features of the data. The proposed method uses a similarity based nearest neighbor imputation approach on each clusters for filling missing values. The proposed method is evaluated on publicly available data set from the data market repository and UCI repository by randomly simulating missing patterns under low, moderate, and high missingness rates throughout the data series. The proposed method's outcome is evaluated with the imputeTestbench package with root mean squared error as an error metric and validated through prediction accuracy and concordance correlation coefficient statistical test. Experimental results show that the proposed imputation technique produces closer values to the original time series data set, resulting in low error rates compared with other existing imputation methods.

Context-Aware Time Series Imputation for Multi-Analyte Clinical Data.

Clinical time series imputation is recognized as an essential task in clinical data analytics. Most models rely either on strong assumptions regarding the underlying data-generation process or on preservation of only local properties without effective consideration of global dependencies. To advance the state of the art in clinical time series imputation, we participated in the 2019 ICHI Data Analytics Challenge on Missing Data Imputation (DACMI). In this paper, we present our proposed model: Context-Aware Time Series Imputation (CATSI), a novel framework based on a bidirectional LSTM in which patients' health states are explicitly captured by learning a "global context vector" from the entire clinical time series. The imputations are then produced with reference to the global context vector. We also incorporate a cross-feature imputation component to explore the complex feature correlations. Empirical evaluations demonstrate that CATSI obtains a normalized root mean square deviation (nRMSD) of 0.1998, which is 10.6% better than that of state-of-the-art models. Further experiments on consecutive missing datasets also illustrate the effectiveness of incorporating the global context in the generation of accurate imputations.

A Sensor Fault-Tolerant Accident Diagnosis System.

Emergency situations in nuclear power plants are accompanied by an automatic reactor shutdown, which gives a big task burden to the plant operators under highly stressful conditions. Diagnosis of the occurred accident is an essential sequence for optimum mitigations; however, it is also a critical source of error because the results of accident identification determine the task flow connected to all subsequent tasks. To support accident identification in nuclear power plants, recurrent neural network (RNN)-based approaches have recently shown outstanding performances. Despite the achievements though, the robustness of RNN models is not promising because wrong inputs have been shown to degrade the performance of RNNs to a greater extent than other methods in some applications. In this research, an accident diagnosis system that is tolerant to sensor faults is developed based on an existing RNN model and tested with anticipated sensor errors. To find the optimum strategy to mitigate sensor error, Missforest, selected from among various imputation methods, and gated recurrent unit with decay (GRUD), developed for multivariate time series imputation based on the RNN model, are compared to examine the extent that they recover the diagnosis accuracies within a given threshold.

Adversarial Recurrent Time Series Imputation.

For the real-world time series analysis, data missing is a ubiquitously existing problem due to anomalies during data collecting and storage. If not treated properly, this problem will seriously hinder the classification, regression, or related tasks. Existing methods for time series imputation either impose too strong assumptions on the distribution of missing data or cannot fully exploit, even simply ignore, the informative temporal dependencies and feature correlations across different time steps. In this article, inspired by the idea of conditional generative adversarial networks, we propose a generative adversarial learning framework for time series imputation under the condition of observed data (as well as the labels, if possible). In our model, we employ a modified bidirectional RNN structure as the generator G, which is aimed at generating the missing values by taking advantage of the temporal and nontemporal information extracted from the observed time series. The discriminator D is designed to distinguish whether each value in a time series is generated or not so that it can help the generator to make an adjustment toward a more authentic imputation result. For an empirical verification of our model, we conduct imputation and classification experiments on several real-world time series data sets. The experimental results show an eminent improvement compared with state-of-the-art baseline models.

Recurrent Neural Networks for Meteorological Time Series Imputation

The aim of the work presented in this paper is to analyze the effectiveness of recurrent neural networks in imputation processes of meteorological time series, for this six different models based on recurrent neural networks such as Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) are implemented and it is experimented with hourly meteorological time series such as temperature, wind direction and wind velocity. The implemented models have architectures of 2, 3 and 4 sequential layers and their results are compared with each other, as well as with other imputation techniques for univariate time series mainly based on moving averages. The results show that for temperature time series on average the recurrent neural network achieve better results than the imputation techniques based on moving averages; in the case of wind direction time series, on average only one model based on RNN manages to exceed the models based on moving averages; and finally, for wind velocity time series on average, no RNN-based model manages to exceed the results achieved by moving averages-based models.

End-to-End Incomplete Time-Series Modeling From Linear Memory of Latent Variables.

Time series with missing values (incomplete time series) are ubiquitous in real life on account of noise or malfunctioning sensors. Time-series imputation (replacing missing data) remains a challenge due to the potential for nonlinear dependence on concurrent and previous values of the time series. In this paper, we propose a novel framework for modeling incomplete time series, called a linear memory vector recurrent neural network (LIME-RNN), a recurrent neural network (RNN) with a learned linear combination of previous history states. The technique bears some similarity to residual networks and graph-based temporal dependency imputation. In particular, we introduce a linear memory vector [called the residual sum vector (RSV)] that integrates over previous hidden states of the RNN, and is used to fill in missing values. A new loss function is developed to train our model with time series in the presence of missing values in an end-to-end way. Our framework can handle imputation of both missing-at-random and consecutive missing inputs. Moreover, when conducting time-series prediction with missing values, LIME-RNN allows imputation and prediction simultaneously. We demonstrate the efficacy of the model via extensive experimental evaluation on univariate and multivariate time series, achieving state-of-the-art performance on synthetic and real-world data. The statistical results show that our model is significantly better than most existing time-series univariate or multivariate imputation methods.

Time Series Imputation via Integration of Revealed Information Based on the Residual Shortcut Connection

Recovering missing values plays a significant role in time series tasks in practical applications. How to replace the missing data and build the dependency relations from the incomplete sample set is still a challenge. The previous research has found that residual network (ResNet) helps to form a deep network and cope with degradation problem by shortcut connection. Gated recurrent unit (GRU) can improve network model and reduce training parameters by update gate which takes the place of forgetting gate and output gate in long short-term memory (LSTM). Inspired by this finding, we observe that shortcut connection and mean of global revealed information can model the relationship among missing items, the previous and overall revealed information. Hence, we design an imputation network with decay factor for shortcut connection and mean of the global revealed information in GRU, called decay residual mean imputation GRU (DRMI-GRU). We introduce a decay residual mean unit (DRMU), which takes full advantage of the previous and global revealed information to model incomplete time series; and the decay factor is applied to balance the previous long-term dependencies and all non-missing values in the sample set. In addition, a mask unit is designed to check the missing data existing or not. An extensive body of empirical comparisons with other existing imputation algorithms over real-world data and public dataset with different ratio of missing data verifies the performance of our model.

Model for Time Series Imputation based on Average of Historical Vectors, Fitting and Smoothing

This paper presents a novel model for univariate time series imputation of meteorological data based on three algorithms: The first of them AHV (Average of Historical Vectors) estimates the set of NA values from historical vectors classified by seasonality; the second iNN (Interpolation to Nearest Neighbors) adjusts the curve predicted by AHV in such a way that it adequately fits to the prior and next value of the NAs gap; The third LANNf allows smoothing the curve interpolated by iNN in such a way that the accuracy of the predicted data can be improved. The results achieved by the model are very good, surpassing in several cases different algorithms with which it was compared.

Local Average of Nearest Neighbors: Univariate Time Series Imputation

The imputation of time series is one of the most important tasks in the homogenization process, the quality and precision of this process will directly influence the accuracy of the time series predictions. This paper proposes two simple algorithms, but quite powerful for univariate time series imputation process, which are based on the means of the nearest neighbors for the imputation of missing data. The first of them Local Average of Neighbors Neighbors (LANN) calculates the missing value from the average of the previous neighbor and the following neighbor to the missing value. The second Local Average of Neighbors Neighbors+ (LANN+), considers a threshold parameter, which allows to differentiate the calculation of the missing values according to the difference between the neighbors: for the differences less than or equal to the threshold the missing value is calculated through of LANN and for major differences the missing value is calculated from the average of the four closest neighbors, two previous and two subsequent to the missing value. Imputation results on different time series are promising.

Improving Long Short-Term Memory Predictions with Local Average of Nearest Neighbors

The study presented in this paper aims to improve the accuracy of meteorological time series predictions made with the recurrent neural network known as Long Short-Term Memory (LSTM). To reach this, instead of just making adjustments to the architecture of LSTM as seen in different related works, it is proposed to adjust the LSTM results using the univariate time series imputation algorithm known as Local Average of Nearest Neighbors (LANN) and LANNc which is a variation of LANN, that allows to avoid the bias towards the left of the synthetic data generated by LANN. The results obtained show that both LANN and LANNc allow to improve the accuracy of the predictions generated by LSTM, with LANN being superior to LANNc. Likewise, on average the best LANN and LANNc configurations make it possible to outperform the predictions reached by another recurrent neural network known as Gated Recurrent Unit (GRU).

Improving Gated Recurrent Unit Predictions with Univariate Time Series Imputation Techniques

The work presented in this paper has its main objective to improve the quality of the predictions made with the recurrent neural network known as Gated Recurrent Unit (GRU). For this, instead of making different adjustments to the architecture of the neural network in question, univariate time series imputation techniques such as Local Average of Nearest Neighbors (LANN) and Case Based Reasoning Imputation (CBRi) are used. It is experimented with different gap-sizes, from 1 to 11 consecutive NAs, resulting in the best gap-size of six consecutive NA values for LANN and for CBRi the gap-size of two NA values. The results show that both imputation techniques allow improving prediction quality of Gated Recurrent Unit, being LANN better than CBRi, thus the results of the best configurations of LANN and CBRi allowed to surpass the techniques with which they were compared.

Inductive Method of Geomagnetic Data Time Series Recovering

Today intensive development of systems and technologies for registration of the Earth’s magnetic field parameters causes an exponential increase of geomagnetic data quantity, mainly collected by the ground magnetic stations. Imperfection of applied equipment and enabled channels of information transfer leads to the presence of omissions in the registered data time series. Along with spatial anisotropy it creates a serious obstacle to the processing of geomagnetic data. Russian and foreign scientific organizations are used to recover missing geomagnetic data by the linear interpolation. The approach provides admissible results in conditions of a quiet magnetosphere, but significantly distorts time series when changing the surrounding magnetic environment. This fact causes a scientific and technical problem, which is concerned with the development of new approach to recovering geomagnetic data registered in unquiet magnetosphere with acceptable time series imputation quality metrics. The authors suggest the approach for time series recovering based on inductive method of machine learning. According to the approach each magnetic station operates its own knowledge base, which is formed during the registration of geomagnetic field and its variations parameters. The combination of the values of the series preceding and following the gap is supposed to be a characteristic description, which is used for searching the precedent in the magnetic station knowledge base. The result contains the required fragment of the time series, which replaces the missing values. The complexity of the information signal, caused by an unquiet magnetic environment, increases the accuracy of search by precedents. The greater the knowledge base of the magnetic station, the higher the effectiveness of the search. Analysis of the results obtained during gap recovering in geomagnetic data time series (registered in conditions of unquiet magnetosphere) demonstrated that the suggested inductive method of imputation allows increasing the accuracy of the missing values recovery by an average of 79.54% compared with the methods currently used. The approach will enhance the efficiency of geomagnetic data processing for solving applied problems.

Retail price time series imputation

A new method to fill in, or impute, missing prices in retail price time series datasets is proposed, called retail price time series imputation RPTSI. It is constructed from an ensemble of three existing methods: namely, price change lookup, central moving average, and polynomial interpolation. Four extended variations of RPTSI are also proposed by considering historical prices for similar products sold by the same retailer and equivalent products sold by competing retailers. Crowdsourced datasets from four North American cities over a year and a half period were used in experiments to evaluate the five RPTSI-based methods and to compare the results against those obtained using last value carried forward, mean imputation, moving average, polynomial interpolation, and multiple imputation. Accuracy was measured by using mean absolute imputation error. Experimental results showed that the RPTSI-based methods had significantly higher accuracy than the other methods on both univariate and multivariate time series datasets.

Model Estimation, Prediction, and Signal Extraction for Nonstationary Stock and Flow Time Series Observed at Mixed Frequencies

An important practical problem for statistical agencies and central banks that publish economic data is the seasonal adjustment of mixed frequency stock and flow time series. This may arise in practice due to changes in funding of a particular survey. Mathematically, the problem can be reduced to the need to compute imputations, forecasts, and backcasts from a given model of the highest available frequency data. The nonstationarity of the economic time series coupled with the alteration of sampling frequency makes the problem of model estimation and imputation challenging. For flow data the analysis cannot be recast as a missing value problem, so that time series imputation methods are ineffective. We provide explicit formulas and algorithms that allow one to compute the log Gaussian likelihood of the mixed sample, as well as any imputations and forecasts. Formulas for the relevant mean squared error are also derived. We evaluate the methodology through simulations, and illustrate the techniques on some economic time series.

Training energy-based models for time-series imputation

Imputing missing values in high dimensional time-series is a difficult problem. This paper presents a strategy for training energy-based graphical models for imputation directly, bypassing difficul...

Forecasting European Union politics: Real-time forecasts in political time series analysis

Forecasting plays an increasingly important role in the scientific study of European Union politics and in political science in general. This is because forecasts are not only indispensable for (political) actors who need to form expectations about future events, but can also be used to judge the validity of (competing) theoretical models. While the debate about whether political science should engage in forecasting is largely over, many questions about how this should be done in everyday research are still open. One of these is how forecasts of political time series can be derived from theoretical models. Using a practical example from European Union research, we start to address this question. We first show how forecasts of political time series can be derived from both theoretical and atheoretical models. Subsequently, we use an atheoretical time series (ARMA) imputation approach to demonstrate how they can be fruitfully integrated in order to overcome some of the limitations to making forecasts of political time series which are based on theoretical models.