Short-Range Ensemble Forecast System Research Articles

Utilizing the High-Resolution Ensemble Forecast System to Produce Calibrated Probabilistic Thunderstorm Guidance

Abstract The High-Resolution Ensemble Forecast system (HREF) calibrated thunder guidance is a suite of probabilistic forecast products designed to predict the likelihood of at least one cloud-to-ground (CG) lightning flash within 20 km (12 miles) of a point during a given 1-, 4-, and 24-h time interval. This guidance takes advantage of a combination of storm attribute and environmental fields produced by the convection-allowing HREF to objectively improve upon lightning forecasts generated by the non-convection-allowing Short-Range Ensemble Forecast system (SREF). This study provides an overview of how the HREF calibrated thunder guidance was developed and calibrated to be statistically reliable against observed CG lightning flashes recorded by the National Lightning Detection Network (NLDN). Performance metrics for the 1-, 4-, and 24-h guidance are provided and compared to the respective SREF calibrated probabilistic lightning forecasts. The HREF calibrated thunder guidance has been implemented operationally within the National Weather Service and is now available to the public. Significance Statement The NOAA Storm Prediction Center has created a suite of new calibrated probabilistic thunderstorm guidance products from a convection-allowing model ensemble, the HREF. The new guidance is a notable improvement over the long-running SREF calibrated thunder guidance and is now operational across the National Weather Service.

Initial Conditions for Convection-Allowing Ensembles over the Conterminous United States

AbstractFive sets of 48-h, 10-member, convection-allowing ensemble (CAE) forecasts with 3-km horizontal grid spacing were systematically evaluated over the conterminous United States with a focus on precipitation across 31 cases. The various CAEs solely differed by their initial condition perturbations (ICPs) and central initial states. CAEs initially centered about deterministic Global Forecast System (GFS) analyses were unequivocally better than those initially centered about ensemble mean analyses produced by a limited-area single-physics, single-dynamics 15-km continuously cycling ensemble Kalman filter (EnKF), strongly suggesting relative superiority of the GFS analyses. Additionally, CAEs with flow-dependent ICPs derived from either the EnKF or multimodel 3-h forecasts from the Short-Range Ensemble Forecast (SREF) system had higher fractions skill scores than CAEs with randomly generated mesoscale ICPs. Conversely, due to insufficient spread, CAEs with EnKF ICPs had worse reliability, discrimination, and dispersion than those with random and SREF ICPs. However, members in the CAE with SREF ICPs undesirably clustered by dynamic core represented in the ICPs, and CAEs with random ICPs had poor spinup characteristics. Collectively, these results indicate that continuously cycled EnKF mean analyses were suboptimal for CAE initialization purposes and suggest that further work to improve limited-area continuously cycling EnKFs over large regional domains is warranted. Additionally, the deleterious aspects of using both multimodel and random ICPs suggest efforts toward improving spread in CAEs with single-physics, single-dynamics, flow-dependent ICPs should continue.

Postprocessing Next-Day Ensemble Probabilistic Precipitation Forecasts Using Random Forests

Abstract Most ensembles suffer from underdispersion and systematic biases. One way to correct for these shortcomings is via machine learning (ML), which is advantageous due to its ability to identify and correct nonlinear biases. This study uses a single random forest (RF) to calibrate next-day (i.e., 12–36-h lead time) probabilistic precipitation forecasts over the contiguous United States (CONUS) from the Short-Range Ensemble Forecast System (SREF) with 16-km grid spacing and the High-Resolution Ensemble Forecast version 2 (HREFv2) with 3-km grid spacing. Random forest forecast probabilities (RFFPs) from each ensemble are compared against raw ensemble probabilities over 496 days from April 2017 to November 2018 using 16-fold cross validation. RFFPs are also compared against spatially smoothed ensemble probabilities since the raw SREF and HREFv2 probabilities are overconfident and undersample the true forecast probability density function. Probabilistic precipitation forecasts are evaluated at four precipitation thresholds ranging from 0.1 to 3 in. In general, RFFPs are found to have better forecast reliability and resolution, fewer spatial biases, and significantly greater Brier skill scores and areas under the relative operating characteristic curve compared to corresponding raw and spatially smoothed ensemble probabilities. The RFFPs perform best at the lower thresholds, which have a greater observed climatological frequency. Additionally, the RF-based postprocessing technique benefits the SREF more than the HREFv2, likely because the raw SREF forecasts contain more systematic biases than those from the raw HREFv2. It is concluded that the RFFPs provide a convenient, skillful summary of calibrated ensemble output and are computationally feasible to implement in real time. Advantages and disadvantages of ML-based postprocessing techniques are discussed.

Validation of Mountain Precipitation Forecasts from the Convection-Permitting NCAR Ensemble and Operational Forecast Systems over the Western United States

Abstract Convection-permitting ensembles can capture the large spatial variability and quantify the inherent uncertainty of precipitation in areas of complex terrain; however, such systems remain largely untested over the western United States. In this study, we assess the capabilities of deterministic and probabilistic cool-season quantitative precipitation forecasts (QPFs) produced by the 10-member, convection-permitting (3-km horizontal grid spacing) NCAR Ensemble using observations collected by SNOTEL stations at mountain locations across the western United States and precipitation analyses from PRISM. We also examine the performance of operational forecast systems run by NCEP including the High Resolution Rapid Refresh (HRRR) model, the NAM forecast system with a 3-km continental United States (CONUS) nest, GFS, and the Short-Range Ensemble Forecast system (SREF). Overall, we find that higher-resolution models, such as the HRRR, NAM-3km CONUS nest, and an individual member of the NCAR Ensemble, are more deterministically skillful than coarser models, especially over the narrow interior ranges of the western United States, likely because they better resolve topography and thus better simulate orographic precipitation. The 10-member NCAR Ensemble is also more probabilistically skillful than 13-member subensembles composed of each SREF dynamical core, but less probabilistically skillful than the full 26-member SREF, as a result of insufficient spread. These results should help guide future short-range model development and inform forecasters about the capabilities and limitations of several widely used deterministic and probabilistic modeling systems over the western United States.

Evaluation and Postprocessing of Ensemble Fire Weather Predictions over the Northeast United States

AbstractThe Short-Range Ensemble Forecast (SREF) system is verified and bias corrected for fire weather days (FWDs) defined as having an elevated probability of wildfire occurrence using a statistical Fire Weather Index (FWI) over a subdomain of the northeastern United States (NEUS) between 2007 and 2014. The SREF is compared to the Rapid Update Cycle and Rapid Refresh analyses for temperature, relative humidity, specific humidity, and the FWI. An additive bias correction is employed using the most recent previous 14 days [sequential bias correction (SBC)] and the most recent previous 14 FWDs [conditional bias correction (CBC)]. Synoptic weather regimes on FWDs are established using cluster analysis (CA) on North American Regional Reanalysis sea level pressure, 850-hPa temperature, 500-hPa temperature, and 500-hPa geopotential height. SREF severely underpredicts FWI (by two indices at FWI = 3) on FWDs, which is partially corrected using SBC and largely corrected with CBC. FWI underprediction is associated with a cool (ensemble mean error of −1.8 K) and wet near-surface model bias (ensemble mean error of 0.46 g kg−1) that decreases to near zero above 800 hPa. Although CBC improves reliability and Brier skill scores on FWDs, ensemble FWI values exhibit underdispersion. CA reveals three synoptic weather regimes on FWDs, with the largest cool and wet biases associated with a departing surface low pressure system. These results suggest the potential benefit of an operational analog bias correction on FWDs. Furthermore, CA may help elucidate model error during certain synoptic weather regimes.

Eastern U.S. Verification of Ensemble Precipitation Forecasts

Abstract The quality of ensemble precipitation forecasts across the eastern United States is investigated, specifically, version 2 of the National Centers for Environmental Prediction (NCEP) Global Ensemble Forecast System Reforecast (GEFSRv2) and Short Range Ensemble Forecast (SREF) system, as well as NCEP’s Weather Prediction Center probabilistic quantitative precipitation forecast (WPC-PQPF) guidance. The forecasts are verified using multisensor precipitation estimates and various metrics conditioned upon seasonality, precipitation threshold, lead time, and spatial aggregation scale. The forecasts are verified, over the geographic domain of each of the four eastern River Forecasts Centers (RFCs) in the United States, by considering first 1) the three systems or guidance, using a common period of analysis (2012–13) for lead times from 1 to 3 days, and then 2) GEFSRv2 alone, using a longer period (2004–13) and lead times from 1 to 16 days. The verification results indicate that, across the eastern United States, precipitation forecast bias decreases and the skill and reliability improve as the spatial aggregation scale increases; however, all the forecasts exhibit some underforecasting bias. The skill of the forecasts is appreciably better in the cool season than in the warm one. The WPC-PQPFs tend to be superior, in terms of the correlation coefficient, relative mean error, reliability, and forecast skill scores, than both GEFSRv2 and SREF, but the performance varies with the RFC and lead time. Based on GEFSRv2, medium-range precipitation forecasts tend to have skill up to approximately day 7 relative to sampled climatology.

A Comparison of 36–60-h Precipitation Forecasts from Convection-Allowing and Convection-Parameterizing Ensembles

Abstract Previous studies examining convection-allowing models (CAMs), as well as NOAA/Hazardous Weather Testbed Spring Forecasting Experiments (SFEs) have typically emphasized “day 1” (12–36 h) forecast guidance. These studies find a distinct advantage in CAMs relative to models that parameterize convection, especially for fields strongly tied to convection like precipitation. During the 2014 SFE, “day 2” (36–60 h) forecast products from a CAM ensemble provided by the Center for Analysis and Prediction of Storms (CAPS) at the University of Oklahoma were examined. Quantitative precipitation forecasts (QPFs) from the CAPS ensemble, known as the Storm Scale Ensemble Forecast (SSEF) system, are compared to NCEP’s operational Short Range Ensemble Forecast (SREF) system, which provides lateral boundary conditions for the SSEF, to see if the CAM ensemble outperforms the SREF through forecast hours 36–60. Equitable threat scores (ETSs) were computed for precipitation thresholds ranging from 0.10 to 0.75 in. for each SSEF and SREF member, as well as ensemble means, for 3-h accumulation periods. The ETS difference between the SSEF and SREF peaked during hours 36–42. Probabilistic forecasts were evaluated using the area under the receiver operating characteristic curve (ROC area). The SSEF had higher values of ROC area, especially at thresholds ≥ 0.50 in. Additionally, time–longitude diagrams of diurnally averaged rainfall were constructed for each SSEF/SREF ensemble member. Spatial correlation coefficients between forecasts and observations in time–longitude space indicated that the SSEF depicted the diurnal cycle much better than the SREF, which underforecasted precipitation with a peak that had a 3-h phase lag. A minority of SREF members performed well.

Bayesian Model Averaging with Stratified Sampling for Probabilistic Quantitative Precipitation Forecasting in Northern China during Summer 2010

Abstract To study the impact of training sample heterogeneity on the performance of Bayesian model averaging (BMA), two BMA experiments were performed on probabilistic quantitative precipitation forecasts (PQPFs) in the northern China region in July and August of 2010 generated from an 11-member short-range ensemble forecasting system. One experiment, as in many conventional BMA studies, used an overall training sample that consisted of all available cases in the training period, while the second experiment used stratified sampling BMA by first dividing all available training cases into subsamples according to their ensemble spread, and then performing BMA on each subsample. The results showed that ensemble spread is a good criterion to divide ensemble precipitation cases into subsamples, and that the subsamples have different statistical properties. Pooling the subsamples together forms a heterogeneous overall sample. Conventional BMA is incapable of interpreting heterogeneous samples, and produces unreliable PQPF. It underestimates the forecast probability at high-threshold PQPF and local rainfall maxima in BMA percentile forecasts. BMA with stratified sampling according to ensemble spread overcomes the problem reasonably well, producing sharper predictive probability density functions and BMA percentile forecasts, and more reliable PQPF than the conventional BMA approach. The continuous ranked probability scores, Brier skill scores, and reliability diagrams of the two BMA experiments were examined for all available forecast days, along with a logistic regression experiment. Stratified sampling BMA outperformed the raw ensemble and conventional BMA in all verifications, and also showed better skill than logistic regression in low-threshold forecasts.

Verification of precipitation forecasts from two numerical weather prediction models in the Middle Atlantic Region of the USA: A precursory analysis to hydrologic forecasting

Summary Accurate precipitation forecasts are required for accurate flood forecasting. The structures of different precipitation forecasting systems are constantly evolving, with improvements in forecasting techniques, increases in spatial and temporal resolution, improvements in model physics and numerical techniques, and better understanding of, and accounting for, predictive uncertainty. Hence, routine verification is necessary to understand the quality of forecasts as inputs to hydrologic modeling. In this study, we verify precipitation forecasts from the National Centers for Environmental Prediction (NCEP) 11-member Global Ensemble Forecast System Reforecast version 2 (GEFSRv2), as well as the 21-member Short Range Ensemble Forecast (SREF) system. Specifically, basin averaged precipitation forecasts are verified for different basin sizes (spatial scales) in the operating domain of the Middle Atlantic River Forecast Center (MARFC), using multi-sensor precipitation estimates (MPEs) as the observed data. The quality of the ensemble forecasts is evaluated conditionally upon precipitation amounts, forecast lead times, accumulation periods, and seasonality using different verification metrics. Overall, both GEFSRv2 and SREF tend to overforecast light to moderate precipitation and underforecast heavy precipitation. In addition, precipitation forecasts from both systems become increasingly reliable with increasing basin size and decreasing precipitation threshold, and the 24-hourly forecasts show slightly better skill than the 6-hourly forecasts. Both systems show a strong seasonal trend, characterized by better skill during the cool season than the warm season. Ultimately, the verification results lead to guidance on the expected quality of the precipitation forecasts, together with an assessment of their relative quality and unique information content, which is useful and necessary for their application in hydrologic forecasting.

Climatology and Ensemble Predictions of Nonconvective High Wind Events in the New York City Metropolitan Region

Abstract Damaging wind events not associated with severe convective storms or tropical cyclones can cause significant problems with transportation, infrastructure, and public safety over the northeastern United States. These nonconvective wind events (NCWEs) are difficult to forecast in New York City and surrounding regions as revealed by the relatively poor probability of detection (POD) and false alarm ratio (FAR) in recent years. This paper investigates the climatology of NCWEs between 15 September and 15 May over 13 cool seasons from 2000/01 through 2012/13. The events are separated into three distinct synoptic patterns: pre-cold-frontal (PRF), post-cold-frontal (POF), and nor’easter/coastal storm (NEC) cases. Relationships between observed winds and some atmospheric parameters such as 900-hPa geopotential height gradient, 3-h mean sea level pressure (MSLP) tendency, low-level wind profile, and stability are also presented. Overall, PRF and NEC have the largest FAR, because several events with a low-level jet at 1–2 km above the surface have relatively strong low-level stability that limits vertical momentum mixing. The POD is lowest for the POF events, which have a strong diurnal influence given the relatively deep mixed layer. Verification is also conducted over the cool seasons from 2009/10 to 2013/14 using the Short Range Ensemble Forecast (SREF) system from the National Centers for Environmental Prediction (NCEP). Both deterministic and probabilistic verification metrics are used to evaluate the performance of the ensemble during NCWEs. Although the SREF has more forecast skill than any of the deterministic SREF control members, it is rather poorly calibrated and exhibits a significant overconfidence given its positive wind speed bias in the lower troposphere.

Using a mesoscale ensemble to predict forecast error and perform targeted observation

Using NCEP short range ensemble forecast (SREF) system, demonstrated two fundamental on-going evolutions in numerical weather prediction (NWP) are through ensemble methodology. One evolution is the shift from traditional single-value deterministic forecast to flow-dependent (not statistical) probabilistic forecast to address forecast uncertainty. Another is from a one-way observation-prediction system shifting to an interactive two-way observation-prediction system to increase predictability of a weather system. In the first part, how ensemble spread from NCEP SREF predicting ensemble-mean forecast error was evaluated over a period of about a month. The result shows that the current capability of predicting forecast error by the 21-member NCEP SREF has reached to a similar or even higher level than that of current state-of-the-art NWP models in predicting precipitation, e.g., the spatial correlation between ensemble spread and absolute forecast error has reached 0.5 or higher at 87 h (3.5 d) lead time on average for some meteorological variables. This demonstrates that the current operational ensemble system has already had preliminary capability of predicting the forecast error with usable skill, which is a remarkable achievement as of today. Given the good spread-skill relation, the probability derived from the ensemble was also statistically reliable, which is the most important feature a useful probabilistic forecast should have. The second part of this research tested an ensemble-based interactive targeting (E-BIT) method. Unlike other mathematically-calculated objective approaches, this method is subjective or human interactive based on information from an ensemble of forecasts. A numerical simulation study was performed to eight real atmospheric cases with a 10-member, bred vector-based mesoscale ensemble using the NCEP regional spectral model (RSM, a sub-component of NCEP SREF) to prove the concept of this E-BIT method. The method seems to work most effective for basic atmospheric state variables, moderately effective for convective instabilities and least effective for precipitations. Precipitation is a complex result of many factors and, therefore, a more challenging field to be improved by targeted observation.

Impact of Bias-Correction Type and Conditional Training on Bayesian Model Averaging over the Northeast United States

Abstract The performance of a multimodel ensemble over the northeast United States is evaluated before and after applying bias correction and Bayesian model averaging (BMA). The 13-member Stony Brook University (SBU) ensemble at 0000 UTC is combined with the 21-member National Centers for Environmental Prediction (NCEP) Short-Range Ensemble Forecast (SREF) system at 2100 UTC. The ensemble is verified using 2-m temperature and 10-m wind speed for the 2007–09 warm seasons, and for subsets of days with high ozone and high fire threat. The impacts of training period, bias-correction method, and BMA are explored for these potentially hazardous weather events using the most recent consecutive (sequential training) and most recent similar days (conditional training). BMA sensitivity to the selection of ensemble members is explored. A running mean difference between forecasts and observations using the last 14 days is better at removing temperature bias than is a cumulative distribution function (CDF) or linear regression approach. Wind speed bias is better removed by adjusting the modeled CDF to the observation. High fire threat and ozone days exhibit a larger cool bias and a greater negative wind speed bias than the warm-season average. Conditional bias correction is generally better at removing temperature and wind speed biases than sequential training. Greater probabilistic skill is found for temperature using both conditional bias correction and BMA compared to sequential bias correction with or without BMA. Conditional and sequential BMA results are similar for 10-m wind speed, although BMA typically improves probabilistic skill regardless of training.

Verification of Precipitation Forecasts from NCEP’s Short-Range Ensemble Forecast (SREF) System with Reference to Ensemble Streamflow Prediction Using Lumped Hydrologic Models

Abstract Precipitation forecasts from the Short-Range Ensemble Forecast (SREF) system of the National Centers for Environmental Prediction (NCEP) are verified for the period April 2006–August 2010. Verification is conducted for 10–20 hydrologic basins in each of the following: the middle Atlantic, the southern plains, the windward slopes of the Sierra Nevada, and the foothills of the Cascade Range in the Pacific Northwest. Mean areal precipitation is verified conditionally upon forecast lead time, amount of precipitation, season, forecast valid time, and accumulation period. The stationary block bootstrap is used to quantify the sampling uncertainties of the verification metrics. In general, the forecasts are more skillful for moderate precipitation amounts than either light or heavy precipitation. This originates from a threshold-dependent conditional bias in the ensemble mean forecast. Specifically, the forecasts overestimate low observed precipitation and underestimate high precipitation (a type-II conditional bias). Also, the forecast probabilities are generally overconfident (a type-I conditional bias), except for basins in the southern plains, where forecasts of moderate to high precipitation are reliable. Depending on location, different types of bias correction may be needed. Overall, the northwest basins show the greatest potential for statistical postprocessing, particularly during the cool season, when the type-I conditional bias and correlations are both high. The basins of the middle Atlantic and southern plains show less potential for statistical postprocessing, as the type-II conditional bias is larger and the correlations are weaker. In the Sierra Nevada, the greatest benefits of statistical postprocessing should be expected for light precipitation, specifically during the warm season, when the type-I conditional bias is large and the correlations are strong.

Forecast of Low Visibility and Fog from NCEP: Current Status and Efforts

Based on the visibility analysis data during November 2009 through April 2010 over North America from the Aviation Digital Database Service (ADDS), the performance of low visibility/fog predictions from the current operational 12 km-NAM, 13 km-RUC and 32 km-WRF-NMM models at the National Centers for Environmental Prediction (NCEP) was evaluated. The evaluation shows that the performance of the low visibility/fog forecasts from these models is still poor in comparison to those of precipitation forecasts from the same models. In order to improve the skill of the low visibility/fog prediction, three efforts have been made at NCEP, including application of a rule-based fog detection scheme, extension of the NCEP Short Range Ensemble Forecast System (SREF) to fog ensemble probabilistic forecasts, and a combination of these two applications. How to apply these techniques in fog prediction is described and evaluated with the same visibility analysis data over the same period of time. The evaluation results demonstrate that using the multi-rule-based fog detection scheme significantly improves the fog forecast skill for all three models relative to visibility-diagnosed fog prediction, and with a combination of both rule-based fog detection and the ensemble technique, the performance skill of fog forecasting can be further raised.

A Comparative Verification of Localized Aviation Model Output Statistics Program (LAMP) and Numerical Weather Prediction (NWP) Model Forecasts of Ceiling Height and Visibility

Abstract In an effort to support aviation forecasting, the National Weather Service’s Meteorological Development Laboratory (MDL) has recently redeveloped the Localized Aviation Model Output Statistics (MOS) Program (LAMP) system. LAMP is designed to run hourly in NWS operations and produce short-range aviation forecast guidance at 1-h projections out to 25 h. This paper compares and contrasts LAMP ceiling height and visibility forecasts with forecasts produced by the 20-km Rapid Update Cycle model (RUC20), the Weather Research and Forecasting Nonhydrostatic Mesoscale Model (WRF-NMM), and the Short-Range Ensemble Forecast system (SREF). RUC20 and WRF-NMM forecasts of continuous ceiling height and visibility were interpolated to stations and converted into categorical forecasts. These interpolated forecasts were also categorized into instrument flight rule (IFR) or lower conditions and verified against LAMP forecasts at stations in the contiguous United States. LAMP and SREF probabilistic forecasts of ceiling height and visibility from LAMP and the SREF system were also verified. This study demonstrates that for the 0000 and 1200 UTC cycles over the contiguous United States, LAMP station-based categorical forecasts of ceiling height, visibility, and IFR conditions or lower are more accurate than the RUC20 and WRF-NMM ceiling height and visibility forecasts interpolated to stations. Moreover, for the 0900 and 2100 UTC forecast cycles and verification periods studied here, LAMP ceiling height and visibility probabilities exhibit better reliability and skill than the SREF system.

Evaluation of an ensemble precipitation prediction system over the Western Mediterranean area

A short-range ensemble precipitation forecast system has been constructed to be applied to over the Mediterranean area down to a 25-km grid spacing using ten members from the fifth-generation Pennsylvania State University-National Center for Atmospheric Research Model (MM5). The ensemble system consists of ten members, each run with a different combination of two different initial conditions from global models and five different subgrid-scale physics configurations for a 2-weeks period of October 2006. The mesoscale verification is made by using observational precipitation data of Spanish Climatic Network. In terms of the total daily precipitation, spatially averaged over the region, time evolution of bias and root mean square error have shown larger departures in the first and final part of the selected period, maintaining quasi-zero values in the middle of the period. Although the created short-range ensemble shows high-spread-skill correlation values for daily precipitation, the asymmetric shape of the rank histogram shows some underdispersion, suggesting a bias behaviour. Confidence intervals have been built to show observational errors, showing more errors in the bins that suggest bias and underdispersive behaviour of the EPS and less errors in the first bins. Additionally, mean spatial forecast rainfall distributions have shown similar patterns to the observed mean precipitation ones, matching the largest accumulated rainfall values and settling down over the same areas. Relative Operative Characteristics diagrams, showing very outstanding areas, and the attribute diagram are indicative of usefulness of the forecasting system, depicting in general a good agreement between forecast probability and the mean observed frequency. Epsgrams of the spatially averaged total daily precipitation for each ensemble member and for the ensemble system illustrate, in general, good agreement among the different members, showing the spread of the ensemble system with departures between them through the time period.

Verification of Extratropical Cyclones within the NCEP Operational Models. Part II: The Short-Range Ensemble Forecast System

Abstract This paper verifies the strengths and positions of extratropical cyclones around North America and the adjacent oceans within the Short Range Ensemble Forecast (SREF) system at the National Centers for Environmental Prediction (NCEP) during the 2004–07 cool seasons (October–March). The SREF mean for cyclone position and central pressure has a smaller error than the various subgroups within SREF and the operational North American Mesoscale (NAM) model in many regions on average, but not the operational Global Forecast System (GFS) for many forecast times. Inclusion of six additional Weather Research and Forecasting (WRF) model members into SREF during the 2006–07 cool season did not improve the SREF mean predictions. The SREF has slightly more probabilistic skill over the eastern United States and western Atlantic than the western portions of the domain for cyclone central pressure. The SREF also has slightly greater probabilistic skill than the combined GFS and NAM for central pressure, which is significant at the 90% level for many regions and thresholds. The SREF probabilities are fairly reliable, although the SREF is overconfident at higher probabilities in all regions. The inclusion of WRF did not improve the SREF probabilistic skill. Over the eastern Pacific, eastern Canada, and western Atlantic, the SREF is overdispersed on average, especially early in the forecast, while across the central and eastern United States the SREF is underdispersed later in the forecast. There are relatively large biases in cyclone central pressure within each SREF subgroup. As a result, the best-member diagrams reveal that the SREF members are not equally accurate for the cyclone central pressure and displacement. Two cases are presented to illustrate examples of SREF developing large errors early in the forecast for cyclones over the eastern United States.

Hydrometeorological Short-Range Ensemble Forecasts in Complex Terrain. Part I: Meteorological Evaluation

Abstract This paper addresses the question of whether it is better to include lower-resolution members of a nested suite of numerical precipitation forecasts to increase ensemble size, or to utilize high-resolution members only to maximize forecast details in regions of complex terrain. A short-range ensemble forecast (SREF) system is formed from three models running in nested configurations at 108-, 36-, 12-, and 4-km horizontal grid spacings. The forecasts are sampled at 27 precipitation-gauge locations, representing 15 pluvial watersheds in southwestern British Columbia, Canada. This is a region of complex topography characterized by high mountains, glaciers, fjords, and land–ocean boundaries. Matching forecast–observation pairs are analyzed for two consecutive wet seasons: October 2003–March 2004 and October 2004–March 2005. The northwest coast of North America is typically subject to intense landfalling Pacific cyclones and frontal systems during these months. Using forecast analysis tools that are well designed for SREF systems, it is found that utilizing the full suite of ensemble members, including the lowest-resolution members, produced the highest quality probabilistic forecasts of precipitation. A companion paper assesses the economic value of SREF probabilistic forecasts for hydroelectric operations.

Reliable Probabilistic Quantitative Precipitation Forecasts from a Short-Range Ensemble Forecasting System during the 2005/06 Cool Season

Abstract A simple binning technique developed to produce reliable probabilistic quantitative precipitation forecasts (PQPFs) from a multimodel short-range ensemble forecasting system is evaluated during the cool season of 2005/06. The technique uses forecasts and observations of 3-h accumulated precipitation amounts from the past 12 days to adjust the present day’s 3-h quantitative precipitation forecasts from each ensemble member for each 3-h forecast period. Results indicate that the PQPFs obtained from this simple binning technique are significantly more reliable than the raw (original) ensemble forecast probabilities. Brier skill scores and areas under the relative operating characteristic curve also reveal that this technique yields skillful probabilistic forecasts of rainfall amounts during the cool season. This holds true for accumulation periods of up to 48 h. The results obtained from this wintertime experiment parallel those obtained during the summer of 2004. In an attempt to reduce the effects of a small sample size on two-dimensional probability maps, the simple binning technique is modified by implementing 5- and 9-point smoothing schemes on the adjusted precipitation forecasts. Results indicate that the smoothed ensemble probabilities remain an improvement over the raw (original) ensemble forecast probabilities, although the smoothed probabilities are not as reliable as the unsmoothed adjusted probabilities. The skill of the PQPFs also is increased as the ensemble is expanded from 16 to 22 members during the period of study. These results reveal that simple postprocessing techniques have the potential to provide greatly improved probabilistic guidance of rainfall events for all seasons of the year.

Bias-Corrected Short-Range Ensemble Forecasts of Near-Surface Variables during the 2005/06 Cool Season

Abstract A postprocessing method initially developed to improve near-surface forecasts from a summertime multimodel short-range ensemble forecasting system is evaluated during the cool season of 2005/06. The method, known as the bias-corrected ensemble (BCE) approach, uses the past complete 12 days of model forecasts and surface observations to remove the mean bias of near-surface variables from each ensemble member for each station location and forecast time. In addition, two other performance-based weighted-average BCE schemes, the exponential smoothing method BCE and the minimum variance estimate BCE, are implemented and evaluated. Values of root-mean-squared error from the 2-m temperature and dewpoint temperature forecasts indicate that the BCE approach outperforms the routinely available Global Forecast System (GFS) model output statistics (MOS) forecasts during the cool season by 9% and 8%, respectively. In contrast, the GFS MOS provides more accurate forecasts of 10-m wind speed than any of the BCE methods. The performance-weighted BCE schemes yield no significant improvement in forecast accuracy for 2-m temperature and 2-m dewpoint temperature when compared with the original BCE, although the weighted BCE schemes are found to improve the forecast accuracy of the 10-m wind speed. The probabilistic forecast guidance provided by the BCE system is found to be more reliable than the raw ensemble forecasts. These results parallel those obtained during the summers of 2002–04 and indicate that the BCE method is a promising and inexpensive statistical postprocessing scheme that could be used in all seasons.