Galformer: a transformer with generative decoding and a hybrid loss function for multi-step stock market index prediction

Abstract

The prediction of stock market fluctuations is crucial for decision-making in various financial fields. Deep learning algorithms have demonstrated outstanding performance in stock market index prediction. Recent research has also highlighted the potential of the Transformer model in enhancing prediction accuracy. However, the Transformer faces challenges in multi-step stock market forecasting, including limitations in inference speed for long sequence prediction and the inadequacy of traditional loss functions to capture the characteristics of noisy, nonlinear stock history data. To address these issues, we introduce an innovative transformer-based model with generative decoding and a hybrid loss function, named “Galformer,” tailored for the multi-step prediction of stock market indices. Galformer possesses two distinctive characteristics: (1) a novel generative style decoder that predicts long time-series sequences in a single forward operation, significantly boosting the speed of predicting long sequences; (2) a novel loss function that combines quantitative error and trend accuracy of the predicted results, providing feedback and optimizing the transformer-based model. Experimental results on four typical stock market indices, namely the CSI 300 Index, S&P 500 Index, Dow Jones Industrial Average Index (DJI), and Nasdaq Composite Index (IXIC), affirm that Galformer outperforms other classical methods, effectively optimizing the Transformer model for stock market prediction.