Indices for the Objective Assessment of Tsunami Forecast Models

Abstract

Many past studies have verified numerical simulations of tsunamis using only qualitative and subjective methods. This paper investigates the relative merits of several indices that can be used to objectively verify tsunami model performance. A number of commonly used indices, such as error in the maximum amplitude and root-mean-square error, are considered, as well as some further indices that have been developed for other specific applications. Desirable qualities of the indices are presented and these include computational efficiency, invariance when applied to tsunamis of any size or to time series of varying length (including relatively short series), and the ability to clearly identify a single best prediction from within a set of simulations. A scenario from the T2 tsunami scenario database is chosen as the control. From this, time series of sea-level elevations are extracted at designated test points located at a range of distances from the tsunami source region. Parameters of the T2 database are perturbed in order to examine the performance of the indices. Of the indices examined, several performed better than others, with Wilmott’s Index of Agreement and Watterson’s transformed Mielke index found to be the best. Combining data from multiple locations was shown to improve the performance of the indices. This study forms the basis for future evaluation of the indices using real observations of tsunamis.