Using new neighborhood-based intensity-scale verification metrics to evaluate WRF precipitation forecasts at 4 and 12 km grid spacings

Abstract

Wavelet-decomposition-based intensity-scale skill (ISS) score is a verification metric which decomposes the forecast fields into different scales and then calculates verification scores. However, due to the “double-penalty” issues, ISS at small scales are often very low even when the forecasts appear subjectively skillful. The displacement error is an important reason for the low ISS. To address this problem, verification methods based on a combination of neighborhood and scale-separation verification approaches are explored. Instead of calculating ISS, a neighborhood-based fractions skill score at different spatial scales, which we call IS_FSS is proposed. Additionally, to reduce the impact of intensity bias, percentile-based instead of fixed thresholds are used in IS_FSS, leading to ISP_FSS. Those two newly developed verification scores are then used to assess WRF forecasts at 4 km and 12 km grid spacings (WRF-4 and WRF-12, respectively) for a case and the entire Meiyu season of 2016. Compared to ISS scores, both IS_FSS and ISP_FSS show more positive verification scores of both WRF-4 and WRF-12 at small scales. Moreover, IS_FSS and ISP_FSS are able to differentiate WRF-12 and WRF-4 forecasts at smaller scales when ISS cannot. Both scores indicate that WRF-4 outperforms WRF-12 for spatial scales from 12 km through 96 km. The scores of WRF-12 improve more when using ISP_FSS than WRF-4, because the former has higher intensity bias.