Partial Convolutional LSTM for Spatiotemporal Prediction of Incomplete Data

Abstract

Advanced data analysis techniques facilitate data-driven spatiotemporal prediction in various fields. However, in real-world data, missing values are inevitable, which causes the data incomplete and makes predictions more challenging. Although we can train complex spatiotemporal correlations with deep learning techniques, most deep learning networks require data without any missing values. In this paper, we propose a novel deep learning framework that manages missing values in the grid-based data structure. We design a partial convolutional long-short-term-memory (PConvLSTM) by combining partial convolution for inpainting and convolutional long-short-term-memory (ConvLSTM) for spatiotemporal prediction. We treat incomplete spatiotemporal data with the partial convolution and train spatiotemporal dependencies with the ConvLSTM structure. The trained PConvLSTM can predict continuous spatial data with missing regions in incomplete input data. We also train the network using incomplete spatiotemporal data without ground truth to enhance practicality. Existing deep learning networks to interpolate missing data are mostly trained by applying ground truth data without missing regions. We show that PConvLSTM achieves higher prediction accuracies compared to ConvLSTM for incomplete data without ground truth.