Confidence intervals of the Kling-Gupta efficiency

Abstract

• The confidence intervals of the KG efficiency do not have a closed-form description. • The uncertainty of KG efficiency is attributed to measurement errors of training data. • Statistical inference with an informal goodness-of-fit metric is coined diagnostic regression. • Parameter confidence regions of KG efficiency in agreement with linear regression theory. • Marginal parameter distributions of KG efficiency are approximately Gaussian. The Kling-Gupta efficiency, hereafter referred to as KG efficiency rather than its common abbreviation KGE, proposed by Gupta et al. (2009) has become a widely used metric for evaluating the goodness-of-fit of n -vectors of observations, y ̃ = [ y ̃ 1 y ̃ 2 … y ̃ n ] ⊤ , and corresponding model simulations, y θ = y 1 θ y 2 θ … y n θ ⊤ . This metric rectifies some of the shortcomings of the coefficient of determination, R 2 , also known to hydrologists as the efficiency of Nash and Sutcliffe (1970) , by using a Euclidean-distance based weighting of the correlation, bias and temporal variability of the observed, y ̃ , and simulated, y θ , data. But as the KG efficiency is not borne out of assumptions with respect to the statistical distribution of the residuals, e θ = y ̃ - y θ , we cannot formally characterize its uncertainty. The NS efficiency suffers a similar problem, yet, statistical theory postulates that its confidence intervals should follow a beta distribution in certain special cases. Without a formal description of the confidence intervals of the KG efficiency, we cannot (amongst others) quantify parameter uncertainty, compute confidence and prediction limits on simulated model responses, inform decision makers about critical modeling uncertainties, evaluate model adequacy and assess the information content of calibration data. More fundamentally, without confidence intervals we cannot establish whether the KG efficiency is a consistent, efficient and unbiased estimator. In this paper we present an empirical description of the confidence intervals of the KG efficiency. We relate the unknown probability distribution of the KG efficiency to the measurement errors of the training data record, y ̃ , and use the bootstrap method to carry out statistical inference. We illustrate our method by application to a simple linear regression function for which the least squares parameter confidence regions are exactly known and two hydrologic models of contrasting complexity. The empirical parameter confidence regions and/or intervals of the KG efficiency are compared to those derived from generalized least squares, objective function contouring and Bayesian analysis using Markov chain Monte Carlo simulation. The marginal parameter distributions of the KG efficiency are generally well described by a normal distribution. Results further confirm that the distribution of the KG efficiency is a complex function of data length and the magnitude, distribution and structure of the measurement errors. This prohibits an analytic description of the empirical confidence regions and/or intervals of the KG efficiency and reiterates the need for the bootstrap method.