Automatic optimization algorithms are used routinely to calibrate conceptual rainfall‐runoff (CRR) models. The goal of calibration is to estimate a feasible and unique (global) set of parameter estimates that best fit the observed runoff data. Most if not all optimization algorithms have difficulty in locating the global optimum because of response surfaces that contain multiple local optima with regions of attraction of differing size, discontinuities, and long ridges and valleys. Extensive research has been undertaken to develop efficient and robust global optimization algorithms over the last 10 years. This study compares the performance of two probabilistic global optimization methods: the shuffled complex evolution algorithm SCE‐UA, and the three‐phase simulated annealing algorithm SA‐SX. Both algorithms are used to calibrate two parameter sets of a modified version of Boughton's [1984] SFB model using data from two Australian catchments that have low and high runoff yields. For the reduced, well‐identified parameter set the algorithms have a similar efficiency for the low‐yielding catchment, but SCE‐UA is almost twice as robust. Although the robustness of the algorithms is similar for the high‐yielding catchment, SCE‐UA is six times more efficient than SA‐SX. When fitting the full parameter set the performance of SA‐SX deteriorated markedly for both catchments. These results indicated that SCE‐UA's use of multiple complexes and shuffling provided a more effective search of the parameter space than SA‐SX's single simplex with stochastic step acceptance criterion, especially when the level of parameterization is increased. Examination of the response surface for the low‐yielding catchment revealed some reasons why SCE‐UA outperformed SA‐SX and why probabilistic optimization algorithms can experience difficulty in locating the global optimum.