Deep learning methods for atmospheric PM2.5 prediction: A comparative study of transformer and CNN-LSTM-attention

Abstract

A transformer-based method was firstly developed to predict the hourly PM2.5 concentration at 12 monitoring stations in Beijing. Convolutional neural network-long short-term memory-attention mechanism (CNN-LSTM-Attention) model were introduced to compare with Transformer model. Experiments on historical long time series data and future prediction were conducted to evaluate the model performance and generalization ability over a long period of time. The four metrics, namely, the explained variance score (EVS), R2, mean absolute error (MAE), and mean square error (MSE), were selected to evaluate the models. The results revealed that the EVS, MAE, MSE, and R2 values of Transformer model were 12%, 9%, 6%, and 30% higher than those of the CNN-LSTM-Attention model, respectively. The forecast results revealed that the Transformer model outperformed the CNN-LSTM-Attention model in terms of the goodness-of-fit R2 (94.4% vs. 83.6%). Transformer model can capture short-term pollution changes affected by abruptly changing meteorological conditions and long-term trends with significant seasonal changes, especially in autumn and winter when the pollution situation is more complex. Transformer model has obvious advantages in overcoming the interdependence problem of the influencing factors in long sequences, providing a new method for the long-term prediction of air quality.