Multi-scale spatial-temporal aware transformer for traffic prediction

Abstract

Traffic prediction is an important part of smart city management. Accurate traffic prediction can be deployed in urban applications such as congestion alerting and route planning, thus providing sustainable services to the public or relevant departments. Although some improvements have been made in existing traffic prediction methods, there are challenges due to the following: (1) Time series has multi-scale nature, that is, from different scale time ranges, traffic flow changes show different trends; (2) Spatial heterogeneity, meaning that traffic conditions in similar functional areas are usually similar. This task remains difficult. To address the above challenges, we propose a new spatial-temporal prediction method, namely Multi-Scale Spatial-Temporal Aware Transformer (MSSTAT), which is a Transformer architecture with multi-scale characteristics. Specifically, compared to the input of encoder, the input of different decoder layers has different scale information, MSSTAT synchronizes model the connection between time steps and scale information by a kind of Parallel Cross Multi-Head Attention, which gives each time step several times the perceived field while also being able to weaken the impact brought by anomaly point. In addition, to add connections between regions with similar functions, we map the traffic data of each node as a probability distribution and then measure the similarity between the nodes by the Wasserstein Distance, which leads to our proposed spatial-temporal aware adjacency matrix. Experimental results on four traffic flow datasets show that MSSTAT outperforms the state-of-the-art baseline.