Spatial verification of ensemble rainfall forecasts over India

Abstract

Increased resolution of Numerical Weather Prediction (NWP) models can lead to improved prediction of heavy rains; however, the forecasts often suffer from location mismatch and intensity errors which can lead to poor traditional verification scores. Spatial verification can provide more realistic statistics to ascertain the quality of forecasts. This study applies contiguous rain areas (CRA), an object based spatial verification method, to evaluate ensemble rainfall forecasts from the NCMRWF Ensemble Prediction System (NEPS-G) over Indian land region for recent three monsoon seasons (2018–2020) using four thresholds 10, 20, 40 and 80 mm/d. The main aim in this study is (a) to quantify the spatial errors and contribution from error components (displacement, volume and pattern errors) which is achieved by the analysis of results from rainfall forecasts during three monsoon seasons over India; (b) to determine if spread-skill relationship observed using traditional measures is also evident in all or some of the object parameters / forecast attributes (followingGallus, 2010); (c) to assess the skill of probabilistic forecast of the object parameters / forecast attributes. This study assesses the skill in the probabilistic forecasts of attributes. Verification of these probabilistic forecasts of attributes is presented in terms of standard verification metrics like Spread Vs Skill, BS, reliability diagram and ROC curve. The analysis of contribution from displacement, volume and pattern errors to the total error shows that the volume contributes the least followed by displacement and finally pattern (~70%). For heavier rainfall events, the pattern is better matched and contribution from displacement and volume is higher. Spread-skill relationship shows that the uncertainties in the NEPS system are better represented for the rainfall area and volume over the west coast and for maximum rainfall in the core monsoon region. Error in the mean of all the attributes is seen to be lower in the west coast as compared to the core monsoon region. Higher rainfall intensity, smaller areas and higher volume are better predicted in terms of having lower BS, reliability values (as a component of BS) and higher BSS and AROC. Similarly these attributes also have ROC curve more aligned towards the top left diagonal and the reliability curve more aligned along the diagonal line of perfect reliability. Spatial verification results indicate that rainfall area and volume demonstrate higher reliability and skill in the forecasts and intensity of rainfall is still a challenging attribute to predict.