STELLM: Spatio-temporal enhanced pre-trained large language model for wind speed forecasting

Abstract

As a renewable and clean energy source, wind energy has caught worldwide attention. To ensure the reliability and stability of wind energy production, wind speed forecasting is of great importance. Although existing forecasting models have shown promising results, they remain relatively small in scale and are incapable of learning comprehensive sequential representations from large-scale data to reason intricate spatio-temporal patterns obscured in entangled wind series. Recently, pre-trained large models have demonstrated impressive performance in natural language processing (NLP) and computer vision (CV). However, large models are rarely applied to wind speed forecasting due to the lack of large-scale wind data for training. To resolve this issue, we propose Spatio-Temporal Enhanced Pre-Trained Large Language Model, namely STELLM, innovatively leveraging the powerful reasoning ability of Large Language Models (LLMs) for accurate wind speed forecasting. Specifically, we first perform the series decomposition to separate wind series data into seasonal and trend components, capturing the distinct properties from disentangled patterns. Then, to empower LLMs with the spatio-temporal modeling capability, we introduce the spatial and temporal prompts, which fuse the geographic distribution information and short-term temporal patterns of wind turbines to enrich the input wind series. Moreover, we leverage an autoregressive fine-tuning strategy to align LLMs with wind series data and learn patch-level representations. Afterwards these representations are fed into a localized spatial module to capture the spatial dependencies between wind turbines. Extensive experiments on four public wind datasets demonstrate the performance superiority of STELLM.