Transformer Versus LSTM: A Comparison of Deep Learning Models for Karst Spring Discharge Forecasting

Abstract

AbstractKarst springs are essential drinking water resources, however, modeling them poses challenges due to complex subsurface flow processes. Deep learning models can capture complex relationships due to their ability to learn non‐linear patterns. This study evaluates the performance of the Transformer in forecasting spring discharges for up to 4 days. We compare it to the Long Short‐Term Memory (LSTM) Neural Network and a common baseline model on a well‐studied Austrian karst spring (LKAS2) with an extensive hourly database. We evaluated the models for two further karst springs with diverse discharge characteristics for comparing the performances based on four metrics. In the discharge‐based scenario, the Transformer performed significantly better than the LSTM for the spring with the longest response times (9% mean difference across metrics), while it performed poorer for the spring with the shortest response time (4% difference). Moreover, the Transformer better predicted the shape of the discharge during snowmelt. Both models performed well across all lead times and springs with 0.64–0.92 for the Nash–Sutcliffe efficiency and 10.8%–28.7% for the symmetric mean absolute percentage error for the LKAS2 spring. The temporal information, rainfall and electrical conductivity were the controlling input variables for the non‐discharge based scenario. The uncertainty analysis revealed that the prediction intervals are smallest in winter and autumn and highest during snowmelt. Our results thus suggest that the Transformer is a promising model to support the drinking water abstraction management, and can have advantages due to its attention mechanism particularly for longer response times.