Ensemble versus deterministic lightning forecast performance at a convective scale over Indian region

Abstract

The present study quantifies the improvement achieved in lightning forecast skill of the NCMRWF regional ensemble prediction system (NEPS-R) compared to its deterministic counterpart (CNTL). The lightning forecasts over study regions of East and Northeast India (ENEI) and Peninsular India (PI) during the pre-monsoon season and Central-East and Northeast India (CENEI) during the monsoon season have been verified using lightning observations from the Indian Institute of Tropical Meteorology (IITM) Lightning Detection Network (LDN). The persisting systematic negative bias in deterministic and EPS-based forecasts of the ensemble mean (EnsMean) and ensemble maximum (EnsMax) indicate the lack of spread among the members, supported by the low values of ensemble spread over all the study regions. EnsMean has the lowest RMSE, with a decrease in error ranging from 0.8 % to 2.18 % compared to CNTL. Categorical skill scores indicate that the EPS-based forecasts (EnsMean and EnsMax) are more skillful than the deterministic forecast at all thresholds and lead times. Further, Fractions Skill Score (FSS) establishes the superiority of the ensemble forecasts over the deterministic forecasts, where for threshold >1, EnsMean is skillful at comparatively smaller neighborhood sizes (ENEI and PI ~68 km; CENEI ~36 km for day-1) than CNTL (ENEI-116 km; PI-196 km; CENEI-68 km). EnsMax at higher thresholds (>5 and > 10) is skillful at lesser neighborhood sizes ranging from 116 to 276 km compared to CNTL (>401 km) for day-1. Hence, skillful re-scaled EPS forecasts based on FSS could provide better guidance for the forecasters. The Continuous Ranked Probability Score of EPS forecasts is lower by around 9 % than the Mean Absolute Error of CNTL forecasts, and the ROC of EPS shows better discrimination of events and non-events compared to CNTL. These highlight the merits of using an EPS over a deterministic system for forecasting a field of high spatial variability, like lightning, and thereby, the use of vast computational resources to run a convective scale EPS is justified.