MBGAN: An improved generative adversarial network with multi-head self-attention and bidirectional RNN for time series imputation

Abstract

Time series data is of great value in data mining and analysis, but it often comes with the problem of data partly missing in many fields. So it is necessary to impute missing values from raw data to improve accuracy in the analysis of time series. Conventional methods based on interpolation ignore the temporal correlation of data. Recurrent Neural Networks (RNN) are good at capturing temporal relationships, while they have a limitation to obtain the potential correlations in multivariate time series. Based on Generative Adversarial Networks, this paper proposes a new model for time series imputation. The key contributions of the paper are: (i) A feature extraction module is designed to reduce the influence of irrelevant features in raw data. (ii) A bidirectional Gated Recurrent Unit (GRU) module is applied to capture the temporal relationships. A temporal attention mechanism is also designed to help capture important correlations in long sequences which will be neglected by conventional RNN. (iii) A new feature attention based on multi-head self-attention is proposed to extract the potential correlations within multivariate features. (iv) A temporal hint mechanism is added so that the discriminator can perform better in identifying fake data and the generator can learn the distribution of raw data better. The proposed model has been tested on 4 real-world datasets. Two metrics are applied to evaluate the results: Root Mean Square Error and Mean Absolute Error. The results illustrate that our model is superior to the other 10 state-of-the-art methods in most cases.