Comparison of Optimization Algorithms in Deep Learning-Based Neural Networks for Hydrological Forecasting: Case Study of Nam River Daily Runoff

Abstract

Neural networks have been widely used in forecasting hydrometeorological variables such as rainfall-runoff simulation and statistical downscaling. Furthermore, deep learning approaches have been developed in recent decades. To improve the performance in deep learning, many optimization algorithms have been proposed in literature to estimate the weights of connections between nodes. An essential need exists to provide information about how the proposed optimization algorithms behave in different applications so that a reasonable algorithm can be selected for a particular application. In the current study, we used rainfall datasets of the Nam River basin to test several available algorithms used in runoff forecasting such as Stochastic Gradient Descent (SGD), Adaptive Gradient Algorithm (Adagrad), Root-Mean-Square prop (RMSprop), Adaptive Delta (Adadelta), Adaptive Moment Estimation (Adam), and Adam with Nesterov Momentum (Nadam). Among these, Adam and Nadam, which are the most recently developed, presented better performance in predicting runoff even though the difference was limited. This difference might be critical when using modeling procedures that must be repeated numerous times, such as deep learning. Further extensive studies might refine the parameter estimation algorithm and allow the use of deep learning for hydrometeorological applications with the tested recent optimization algorithms. Keywords: Neural Network, Optimization, Rainfall-Runoff, Simulation