Attention based long-term air temperature forecasting network: ALTF Net

Abstract

Air temperature is one of the most important meteorological parameters related with atmospheric and environmental research. In this context, accurate prediction and forecasting of temperature is crucial due to the current global climate change. Although, the short term temperature forecasting have been more or less conquered in the past by using predictive algorithms, the long-term temperature forecasting is still a challenging task. Long term temperature forecasting is previously attempted by deep learning methods like Recurrent Neural Network (RNN), Long Short Term Memory (LSTM), etc. However, the gradient explosion and gradient vanishing problems of the RNN based networks were the major roadblock in the path of long-term prediction. So, in this paper, an attention-based model called ALTF Net (Attention based Long term Temperature Forecasting Network) approaches this problem using an Encoder–Decoder orientation. The Encoder encodes the relative dependencies of the auto-regressive time-series into an attention tensor which is used by the Decoder to produce the prediction. The Encoder is augmented to incorporate a convolution block to recognize the seasonal patterns. The proposed model ALTF uses a Transformer with an augmented encoder to predict temperature up to 150 days with high accuracy, a feat which would be difficult using RNN and LSTM. The model has been trained with 25+ years of data from 5 cities around the globe and the performance have been rigorously evaluated in terms of RMSE, MAE, R2, and correlation values. It is observed that the proposed model dominated over several baselines (ARIMA, RFR, KNN, MLP, RNN, CNN, LSTM, and Transformer) for long term temperature forecasting.