Dynamic transformer ODEs for large-scale reservoir inflow forecasting

Abstract

Forecasting incoming water demand is a critical step in efficient reservoir management and revenue optimization in large-scale cascade hydropower stations. It depends on multiple factors, such as weather conditions, grid dispatch, and electricity demand, and, in turn, facilitates a range of downstream decision-making, from natural hazards control and water ecology protection to power generation plans. Current efforts mainly rely on methodologies from statistical machine learning or deep neural networks to model the hydrological patterns from historical time series for inflow forecasting. However, existing models are restricted by short-term temporal dependencies and are prone to error accumulation issues due to the underlying autoregressive architecture. Meanwhile, most recent self-attention time-series models fail to achieve real-time inflow forecasting because of tremendous parameters and computational bottlenecks on learning long time series with fine granularity. We propose a novel framework, called DTODE (Dynamic Transformer Ordinary Differential Equations), for capturing nonlinear and non-stationary evolving patterns inherent in hydrological time series. Specifically, we present the dynamic self-attention mechanism combining transformer and ordinary differential equations that simultaneously captures long-range dependencies of observations from a dynamic system perspective. DTODE exploits a continuum of self-attention layers (instead of discrete counterparts) to learn the dynamics of multivariate time series while paying attention to the co-evolving time-related factors. Besides, our model is flexible in inferring the complex states at any time step, allowing us to forecast inflows at multiple time horizons. Comprehensive evaluations on real-world datasets show that DTODE significantly reduces forecasting errors compared to state-of-the-art inflow prediction systems.