THE EFFECT OF DATA INDEPENDENCE IN MODEL CALIBRATION AND MODEL TESTING1

Abstract

ABSTRACT: With the increased use of models in hydrologic design, there is an immediate need for a comprehensive comparison of hydrologic models, especially those intended for use at ungaged locations (i.e., where measured data are either not available or inadequate for model calibration). But some past comparisons of hydrologic models have used the same data base for both calibration and testing of the different models or implied that the results of model calibration are indicative of the accuracy at ungaged locations. This practice was examined using both the regression equation approach to peak discharge estimation and a unit hydrograph model that was intended for use in urban areas. The results suggested that the lack of data independence in the calibration and testing of regression equations may lead to both biased results and misleading statements about prediction accuracy. Additionally, although split‐sample testing is recognized as desirable, the split‐samples should be selected using a systematic‐random sampling scheme, rather than random sampling, because random sampling with small samples may lead to a testing sample that is not representative of the population. A systematic‐random sampling technique should lead to more valid conclusions about model reliability. For models like a unit hydrograph model, which are more complex and for which calibration is a more involved process, data independence is not as critical because the data fitting error variation is not as dominant as the error variation due to the calibration process and the inability of the model structure to conform with data variability.