A Comprehensive Big Data Framework for Energy Markets: Oil and Gas Demand Forecasting

Abstract

Demand forecasting in the energy sector is essential for both countries and companies to plan their supply and demand. Agents in the highly volatile oil markets have to act fast and data-driven. In the literature, studies on oil or gasoline demand forecasting are carried out using traditional econometric and AI-based models, using static data for long periods. In this paper, short-term gasoline demand forecasting literature has been investigated. We focus on short-term demand prediction based on big data analytics and investigate potential data sources and architectures to collect data. To this end, several iterative meetings were conducted between the Data Science department, the Oil Trading department of an Oil & Gas company, and researchers within an industry-academia cooperation project. Traditional data sources used for the problem are presented, and the applicability of real-time data to the problem is discussed. A big data architecture is proposed that can be used to predict the demand for petroleum products, mainly for gasoline in the U.S., for the transparency, amplitude, and availability of open data.