End-to-End Incomplete Time-Series Modeling From Linear Memory of Latent Variables.

Abstract

Time series with missing values (incomplete time series) are ubiquitous in real life on account of noise or malfunctioning sensors. Time-series imputation (replacing missing data) remains a challenge due to the potential for nonlinear dependence on concurrent and previous values of the time series. In this paper, we propose a novel framework for modeling incomplete time series, called a linear memory vector recurrent neural network (LIME-RNN), a recurrent neural network (RNN) with a learned linear combination of previous history states. The technique bears some similarity to residual networks and graph-based temporal dependency imputation. In particular, we introduce a linear memory vector [called the residual sum vector (RSV)] that integrates over previous hidden states of the RNN, and is used to fill in missing values. A new loss function is developed to train our model with time series in the presence of missing values in an end-to-end way. Our framework can handle imputation of both missing-at-random and consecutive missing inputs. Moreover, when conducting time-series prediction with missing values, LIME-RNN allows imputation and prediction simultaneously. We demonstrate the efficacy of the model via extensive experimental evaluation on univariate and multivariate time series, achieving state-of-the-art performance on synthetic and real-world data. The statistical results show that our model is significantly better than most existing time-series univariate or multivariate imputation methods.