Adding attention to the neural ordinary differential equation for spatio-temporal prediction

Abstract

Explainable spatio-temporal prediction gains attraction in the development of geospatial artificial intelligence. The neural ordinal differential equation (NODE) emerges as a new solution for explainable spatio-temporal prediction. However, challenges still need to be solved in most existing NODE-based prediction models, such as difficulty modeling spatial data and mining long-term temporal dependencies in data. In this study, we propose a spatio-temporal attentional NODE (STA-ODE) to address the two challenges above. First, we define a spatio-temporal ordinary differential equation to predict a value at each time iteratively by a novel spatio-temporal derivative network. Second, we develop an attention mechanism to fuse multiple prediction values for capturing long-term temporal dependencies in data. To train the STA-ODE model, we design a loss function that aligns the prediction results in spatial dimension with prediction results in temporal dimension to calibrate the parameters of the model. The proposed model was validated with three real-world spatio-temporal datasets (traffic flow dataset, PM2.5 monitoring dataset, and temperature monitoring dataset). Experimental results showed that STA-ODE outperformed seven existing baselines regarding prediction accuracy. In addition, we used visualization to demonstrate the sound interpretability and prediction accuracy of the STA-ODE model.