A comparison of automatic and manual calibration using the Pitman model

Abstract

Although considerable advances have been made in automatic model calibration, most applications have been confined to research and not operational modelling. In Southern Africa, manual calibration still predominates even in research applications and automatic model calibration is often viewed as dangerous. The concern often cited is that automatic calibration fails to attach physical reality to the parameters and the resulting modelling may therefore not make hydrologic sense. Manual calibration on the other hand is often experienced as tedious, time consuming and subjective. An analysis of the limitations and dangers of manual and automatic calibration using the widely applied Pitman model is the subject of this paper. Nine subcatchments from three southern African basins that had been previously calibrated manually were calibrated automatically using the shuffled complex evolution (SCE-UA) method. For all catchments, 10 randomly initialized calibration runs were carried out. Automatic calibration was found to obtain considerably better performance for 2 of the 9 subcatchments modelled suggesting, not unexpectedly, that manual calibration is more prone to achieving suboptimal parameter sets than automatic calibration. For one basin, automatic calibration obtained slightly better performance than manual calibration but gave notably poorer validation performance suggesting inadequacy of model structure. This particular basin features dambos that were not explicitly incorporated in the model structure but whose impact on some parameter values was inferred from the automatic calibration. Previous manual calibration of the same basin had failed to make any such inferences. By setting appropriate parameter search ranges in automatic calibration, the danger of obtaining unrealistic parameters is reduced. If adequate data are available, it may be practical to automatically calibrate parameters in groups using only the portions of the streamflow hydrograph that they apply. This would require the level of understanding of the model and catchment processes that is typical to manual calibration but reduce the tediousness and subjectivity of manual calibration by applying an optimizer instead of manual trial and error.