Improving predictability of the future by grasping probability less tightly

Abstract

In the last 30 years, whilst there has been an explosion in our ability to make quantative predictions, less progress has been made in terms of building useful forecasts to aid decision support. In most real world systems, single point forecasts are fundamentally limited because they only simulate a single scenario and thus do not account for observational uncertainty. Ensemble forecasts aim to account for this uncertainty but are of limited use since it is unclear how they should be interpreted. Building probabilistic forecast densities is a theoretically sound approach with an end result that is easy to interpret for decision makers; it is not clear how to implement this approach given finite ensemble sizes and structurally imperfect models. This thesis explores methods that aid the interpretation of model simulations into predictions of the real world. This includes evaluation of forecasts, evaluation of the models used to make forecasts and the evaluation of the techniques used to interpret ensembles of simulations as forecasts. Bayes theorem is a fundamental relationship used to update a prior probability of the occurence of some event given new information. Under the assumption that each of the probabilities in Bayes theorem are perfect, it can be shown to make optimal use of the information available. Bayes theorem can also be applied to probability density functions and thus updating some previously constructed forecast density with a new one can be expected to improve forecast skill, as long as each forecast density gives a good representation of the uncertainty at that point in time. The relevance of the probability calculus, however, is in doubt when the forecasting system is imperfect, as is always the case in real world systems. Taking the view that we wish to maximise the logarithm of the probability density placed on the outcome, two new approaches to the combination of forecast densities formed at different lead times are introduced and shown to be informative even in the imperfect model scenario, that is a case where the Bayesian approach is shown to fail.