Short-Range Ensemble Forecast System Research Articles

Reliable Probabilistic Quantitative Precipitation Forecasts from a Short-Range Ensemble Forecasting System

Abstract A simple binning technique is developed to produce reliable 3-h probabilistic quantitative precipitation forecasts (PQPFs) from the National Centers for Environmental Prediction (NCEP) multimodel short-range ensemble forecasting system obtained during the summer of 2004. The past 12 days’ worth of forecast 3-h accumulated precipitation amounts and observed 3-h accumulated precipitation amounts from the NCEP stage-II multisensor analyses are used to adjust today’s 3-h precipitation forecasts. These adjustments are done individually to each of ensemble members for the 95 days studied. Performance of the adjusted ensemble precipitation forecasts is compared with the raw (original) ensemble predictions. Results show that the simple binning technique provides significantly more skillful and reliable PQPFs of rainfall events than the raw forecast probabilities. This is true for the base 3-h accumulation period as well as for accumulation periods up to 48 h. Brier skill scores and the area under the relative operating characteristics curve also indicate that this technique yields skillful probabilistic forecasts. The performance of the adjusted forecasts also progressively improves with the increased accumulation period. In addition, the adjusted ensemble mean QPFs are very similar to the raw ensemble mean QPFs, suggesting that the method does not significantly alter the ensemble mean forecast. Therefore, this simple postprocessing scheme is very promising as a method to provide reliable PQPFs for rainfall events without degrading the ensemble mean forecast.

Evaluation of a Mesoscale Short-Range Ensemble Forecast System over the Northeast United States

Abstract A short-range ensemble forecast system was constructed over the northeast United States down to 12-km grid spacing using 18 members from the fifth-generation Pennsylvania State University–National Center for Atmospheric Research Mesoscale Model (MM5). The ensemble consisted of 12 physics members with varying planetary boundary layer schemes and convective parameterizations as well as seven different initial conditions (ICs) [five National Centers for Environmental Prediction (NCEP) Eta-bred members at 2100 UTC and the 0000 UTC NCEP Global Forecast System (GFS) and Eta runs]. The full 18-member ensemble (ALL) was verified at the surface for the warm (May–September 2003) and cool (October 2003–March 2004) seasons. A randomly chosen subset of seven physics (PHS) members at each forecast hour was used to quantitatively compare with the seven IC members. During the warm season, the PHS ensemble predictions for surface temperature and wind speed had more skill than the IC ensemble and a control (shared PHS and IC member) run initialized 12 h later (CTL12). During the cool and warm seasons, a 14-day running-mean bias calibration applied to the ALL ensemble (ALLBC) added 10%–30% more skill for temperature, wind speed, and sea level pressure, with the ALLBC far outperforming the CTL12. For the 24-h precipitation, the PHS ensemble had comparable probabilistic skill to the IC ensemble during the warm season, while the IC subensemble was more skillful during the cool season. All ensemble members had large diurnal surface biases, with ensemble variance approximating ensemble uncertainty only for wind direction. Selection of ICs was also important, because during the cool season the NCEP-bred members introduced large errors into the IC ensemble for sea level pressure, while none of the subensembles (PHS, IC, or ALL) outperformed the GFS–MM5 for sea level pressure.

Hydrometeorological Short-Range Ensemble Forecasts in Complex Terrain. Part I: Meteorological Evaluation

This paper addresses the question of whether it is better to include lower-resolution members of a nested suite of numerical precipitation forecasts to increase ensemble size, or to utilize high-resolution members only to maximize forecast details in regions of complex terrain. A short-range ensemble forecast (SREF) system is formed from three models running in nested configurations at 108-, 36-, 12-, and 4-km horizontal grid spacings. The forecasts are sampled at 27 precipitation-gauge locations, representing 15 pluvial watersheds in southwestern British Columbia, Canada. This is a region of complex topography characterized by high mountains, glaciers, fjords, and land–ocean boundaries. Matching forecast–observation pairs are analyzed for two consecutive wet seasons: October 2003–March 2004 and October 2004–March 2005. The northwest coast of North America is typically subject to intense landfalling Pacific cyclones and frontal systems during these months. Using forecast analysis tools that are well designed for SREF systems, it is found that utilizing the full suite of ensemble members, including the lowest-resolution members, produced the highest quality probabilistic forecasts of precipitation. A companion paper assesses the economic value of SREF probabilistic forecasts for hydroelectric operations.

Value of Human-Generated Perturbations in Short-Range Ensemble Forecasts of Severe Weather

Abstract During the spring of 2003, the Storm Prediction Center, in partnership with the National Severe Storms Laboratory, conducted an experiment to explore the value of having operational severe weather forecasters involved in the generation of a short-range ensemble forecasting system. The idea was to create a customized ensemble to provide guidance on the severe weather threat over the following 48 h. The forecaster was asked to highlight structures of interest in the control run and, using an adjoint model, a set of perturbations was obtained and used to generate a 32-member fifth-generation Pennsylvania State University–National Center for Atmospheric Research Mesoscale Model (MM5) ensemble. The performance of this experimental ensemble is objectively evaluated and compared with other available forecasts (both deterministic and ensemble) using real-time severe weather reports and precipitation in the central and eastern parts of the continental United States. The experimental ensemble outperforms the operational forecasts considered in the study for episodes with moderate-to-high probability of severe weather occurrence and those with moderate probability of heavy precipitation. On the other hand, the experimental ensemble forecasts of low-probability severe weather and low precipitation amounts have less skill than the operational models, arguably due to the lack of global dispersion in a system designed to target the spread over specific areas of concern for severe weather. Results from an additional test ensemble constructed by combining automatic and manually perturbed members show the best results for numerical forecasts of severe weather for all probability values. While the value of human contribution in the numerical forecast is demonstrated, further research is needed to determine how to better use the skill and experience of the forecaster in the construction of short-range ensembles.

Bias‐corrected short‐range ensemble forecasts of near surface variables

AbstractA multimodel short‐range ensemble forecasting system created as part of a National Oceanic and Atmospheric Administration program on improved high temperature forecasting during the summer of 2003 is evaluated. Results from this short‐range ensemble system indicate that using the past complete 12 days of forecasts to bias correct today's forecast yields ensemble mean forecasts of 2‐m temperature, 2‐m dewpoint temperature, and 10‐m wind speed that are competitive with or better than those available from any of the model output statistics presently generated operationally in the United States. However, the bias‐corrected ensemble system provides more than just the ensemble mean forecast. Probabilities produced by this system are skilful and reliable, and have been found to be valuable when evaluated in a cost‐loss model. The ensemble appears to provide better guidance for more unlikely events, such as very warm temperatures, that likely have the greatest economic significance. Industries that are sensitive to the weather, such as power companies, transportation and agriculture, may benefit from the probability information provided. Thus, it is possible to develop post‐processing for short‐range ensemble forecasting systems that is competitive with or better than traditional post‐processing techniques, thereby allowing the rapid production of useful and accurate guidance forecasts of many near surface variables. Copyright © 2005 Royal Meteorological Society

Short-Range Ensemble Forecasts of Precipitation Type

Abstract Short-range ensemble forecasting is extended to a critical winter weather problem: forecasting precipitation type. Forecast soundings from the operational NCEP Short-Range Ensemble Forecast system are combined with five precipitation-type algorithms to produce probabilistic forecasts from January through March 2002. Thus the ensemble combines model diversity, initial condition diversity, and postprocessing algorithm diversity. All verification numbers are conditioned on both the ensemble and observations recording some form of precipitation. This separates the forecast of type from the yes–no precipitation forecast. The ensemble is very skillful in forecasting rain and snow but it is only moderately skillful for freezing rain and unskillful for ice pellets. However, even for the unskillful forecasts the ensemble shows some ability to discriminate between the different precipitation types and thus provides some positive value to forecast users. Algorithm diversity is shown to be as important as initial condition diversity in terms of forecast quality, although neither has as big an impact as model diversity. The algorithms have their individual strengths and weaknesses, but no algorithm is clearly better or worse than the others overall.

Aspects of Effective Mesoscale, Short-Range Ensemble Forecasting

Abstract This study developed and evaluated a short-range ensemble forecasting (SREF) system with the goal of producing useful, mesoscale forecast probability (FP). Real-time, 0–48-h SREF predictions were produced and analyzed for 129 cases over the Pacific Northwest. Eight analyses from different operational forecast centers were used as initial conditions for running the fifth-generation Pennsylvania State University–National Center for Atmospheric Research (PSU–NCAR) Mesoscale Model (MM5). Model error is a large source of forecast uncertainty and must be accounted for to maximize SREF utility, particularly for mesoscale, sensible weather phenomena. Although inclusion of model diversity improved FP skill (both reliability and resolution) and increased dispersion toward statistical consistency, dispersion remained inadequate. Conversely, systematic model errors (i.e., biases) must be removed from an SREF since they contribute to forecast error but not to forecast uncertainty. A grid-based, 2-week, running-mean bias correction was shown to improve FP skill through 1) better reliability by adjusting the ensemble mean toward the mean of the verifying analysis, and 2) better resolution by removing unrepresentative ensemble variance. Comparison of the multimodel (each member uses a unique model) and varied-model (each member uses a unique version of MM5) approaches indicated that the multimodel SREF exhibited greater dispersion and superior performance. It was also found that an ensemble of unequally likely members can be skillful as long as each member occasionally performs well. Finally, smaller grid spacing led to greater ensemble spread as smaller scales of motion were modeled. This study indicates substantial utility in current SREF systems and suggests several avenues for further improvement.

Application of a Limited-Area Short-Range Ensemble Forecast System to a Case of Heavy Rainfall in the Mediterranean Region

Abstract Severe weather risk assessment is becoming an increasing component of the daily operational activity at advanced meteorological forecasting centers. To improve its forecast capabilities and develop a severe weather warning system, the Sardinian Regional Meteorological Service (SAR) and the Centre for Advanced Studies, Research and Development in Sardinia (CRS4) have been testing a Limited-area Ensemble Prediction System (SAR-LEPS). The SAR-LEPS system is described and preliminary results of its use to predict the intense and localized flooding event of 12–13 November 1999 are discussed. Results for this specific case study indicate that SAR-LEPS provided valuable, additional information to the ECMWF ensemble system. This suggests that the operational use of SAR-LEPS could prove to be an important component in the decision making process in issuing threat warnings for severe weather conditions.

Short-Range Ensemble Predictions of 2-m Temperature and Dewpoint Temperature over New England

Abstract A multimodel short-range ensemble forecasting system created as part of a National Oceanic and Atmospheric Administration pilot program on temperature and air quality forecasting over New England during the summer of 2002 is evaluated. A simple 7-day running mean bias correction is applied individually to each of the 23 ensemble members. Various measures of accuracy are used to compare these bias-corrected ensemble predictions of 2-m temperature and dewpoint temperature with those available from the nested grid model (NGM) model output statistics (MOS). Results indicate that the bias-corrected ensemble mean prediction is as accurate as the NGM MOS for temperature predictions, and is more accurate than the NGM MOS for dewpoint temperature predictions, for the 48 days studied during the warm season. When the additional probabilistic information from the ensemble is examined, results indicate that the ensemble clearly provides value above that of NGM MOS for both variables, especially as the events ...

Initial Results of a Mesoscale Short-Range Ensemble Forecasting System over the Pacific Northwest

Motivated by the promising results of global-scale ensemble forecasting, a number of groups have attempted mesoscale, short-range ensemble forecasting (SREF), focusing mainly over the eastern half of the United States. To evaluate the performance of mesoscale SREF over the Pacific Northwest and to test the value of using different initial analyses as a means of ensemble forecast generation, a five-member mesoscale SREF system was constructed in which the Pennsylvania State University–National Center for Atmospheric Research fifth-generation Mesoscale Model (MM5) was run with initializations and forecast boundary conditions from major operational centers. The ensemble system was evaluated over the Pacific Northwest from January to June 2000. The model verification presented in this study considers only near-surface weather variables, especially the observed 10-m wind direction. The ensemble mean forecast displays lower mean absolute wind direction errors than the component ensemble members when averaged over all cases. The frequency with which the ensemble mean forecast verifies best is no better than the frequency of any individual member forecast. The wind direction forecast errors for the 12-km ensemble mean forecasts are comparable to 4-km deterministic forecast errors. Ensemble mean forecasts are observed to retain much of the orographically forced mesoscale structure in the component forecasts while smoothing out phase differences for propagating features. The correlation between forecast spread and forecast error for wind direction is approximately 0.6 for most lead times. Spread–error correlations rise to roughly 0.8 when only cases with high or low spread are considered. Such high correlations suggest that the ensemble system possesses the ability to predict forecast skill for high- and low-spread cases. The tendency toward higher spread–error correlation when cases with medium spread are filtered out is also found for each component member of the ensemble.

Verification of Eta–RSM Short-Range Ensemble Forecasts

Motivated by the success of ensemble forecasting at the medium range, the performance of a prototype shortrange ensemble forecast system is examined. The ensemble dataset consists of 15 case days from September 1995 through January 1996. There are 15 members of the ensemble, 10 from an 80-km version of the eta model and five from the regional spectral model. Initial conditions include various in-house analyses available at the National Centers for Environmental Prediction as well as bred initial conditions interpolated from the mediumrange forecast ensemble. Forecasts from the 29-km mesoeta model were archived as well for comparison. The performance of the ensemble is first evaluated by the criterion of ‘‘uniformity of verification rank.’’ Assuming a perfect forecast model, equally plausible initial conditions, and the verification is a plausible member of the ensemble, these imply the verification when pooled with the 15 ensemble forecasts and sorted is equally likely to occur in each of the 16 ranks. Hence, over many independent samples, a histogram of the rank distribution should be nearly uniform. Using data from the ensemble forecasts, rank distributions were populated and found to be nonuniform. This was determined to be largely a result of model and initial condition deficiencies and not problems with the verification data. The uniformity of rank distributions varied with atmospheric baroclinicity for midtropospheric forecast variables but not for precipitation forecasts. Examination of the error characteristics of individual ensemble members showed that the assumption of identical errors for each member is not met with this particular ensemble configuration, primarily because of the use of both bred and nonbred initial conditions in this test. Further, there were both differences in the accuracy of eta and regional spectral model bred member forecasts. The performance of various summary forecasts from the ensemble such as its mean showed that the ensemble can generate forecasts that have similar or lower error than forecasts from the 29-km mesoeta, which was approximately equivalent in computational expense. Also, by combining the ensemble forecasts with rank information from other cases, reliable ensemble precipitation forecasts could be created, indicating the potential for useful probabilistic forecasts of quantitative precipitation from the ensemble.