Crude oil future price forecasting using pretrained transformer model

Abstract

This study proposes a new crude oil futures price forecasting model based on Large Language Models. It takes advantage of the modeling capabilities of pretrained transformer models with attention mechanism in the mainstream Large Language Models. Results from empirical studies using INE future price showed that the introduction of LLM to the crude oil futures price forecasting model contributes to the improved forecasting accuracy. The forecasting accuracy of the pretrained transformer based crude oil futures price forecasting model is sensitive to the LLM model types. Llama-2 7b is found to provide the best forecasting accuracy for crude oil futures price forecasting.