Adapting the SAL method to evaluate reflectivity forecasts of summer precipitation in the central United States

Abstract

AbstractThe Structure Amplitude Location (SAL) method was originally developed to evaluate forecast accumulated‐precipitation fields through identification and comparison of objects in both the forecast and the observed fields. This study describes a small modification for use with instantaneous composite‐reflectivity forecasts, where objects' minimum size and reflectivity thresholds are prescribed. Both the original and modified SAL methods are used to evaluate daily 0000 UTC 12‐km North American Model (NAM) forecasts, against NCEP/EMC 4‐km Stage IV accumulated‐precipitation estimates, during the summer of 2015 for a central US domain. Results show substantial sensitivity to the reflectivity threshold. This is likely related to sampling more signal from convective cell cores, and progressively ignoring stratiform rain areas, as threshold increases. Setting the threshold too high (40 dBZ) yields only 7% of time periods on which error scores can be computed, as opposed to 94% using a low threshold (5 dBZ). The primary difference between the two methods is a larger structural error in SAL using reflectivity, likely related to the unresolved convective peaks in the 12‐km NAM forecasts; this error is smoothed out when accumulated precipitation is evaluated. SAL using reflectivity also reveals a diurnal cycle of skill, with minimum skill occurring around 1800–2200 UTC (early to late afternoon local time, before average convective activity reaches its maximum) and maximum skill occurring around 1000 UTC (just before sunrise). We conclude that both methods yield useful results, but results presented herein may not be generalisable to other verification domains or SAL formulations.