Deep transfer learning based on transformer for flood forecasting in data-sparse basins

Abstract

There exists a substantial disparity in the distribution of streamflow gauge and basin characteristic information, with a majority of flood observations being recorded from a limited number of well-monitored locations. Transferring hydrological knowledge from data-rich to data-sparse basins has been a persistent issue. While artificial intelligence methods, such as Long-Short-Term Memory Neural Networks (LSTMs) have been attempted for simulating across different basins, the problem of gradient vanishing still inevitably lead to difficulties in responding to new basin datasets. A novel solution is presented in the form of the Transfer Learning Framework based on Transformer (TL-Transformer), which can accurately predict flooding in data-sparse basins (targets) by using models from data-rich basins (sources) without requiring extensive basin attributes at the target location. The framework was demonstrated in the middle reaches of the Yellow River, and performance was evaluated using the Nash Sutcliffe efficiency coefficient, root mean square error, and bias. The results show that TL-Transformer outperforms other models, including TOPMODEL, MLP, TL-MLP, LSTM, TL-LSTM and Transformer in all target basin stations. Compared with those models that without transfer learning, TL-Transformer showed significantly improved performance, where NSE increased by 0.2 and reached above 0.75, RMSE and bias were also controlled within 60 and 20, respectively. Furthermore, pre-training on basins with hydrological similarities increases the benefits of Transfer Learning, and we explained this response phenomenon through the topographic index. These results suggest that deep learning can tap into the commonalities in hydrological data across basins to improve the accuracy of flood forecast applications in areas with limited observations.