Statistical Post-processing Methods Research Articles

Comparing Area Probability Forecasts of (Extreme) Local Precipitation Using Parametric and Machine Learning Statistical Postprocessing Methods

AbstractProbabilistic forecasts, which communicate forecast uncertainties, enable users to make better weather-based decisions. Using precipitation and numerous instability indices from the deterministic model HARMONIE–AROME (HA; a nonhydrostatic numerical weather prediction model) as potential predictors, we generate summer areal probabilistic maximum hourly precipitation forecasts across 11 regions of the Netherlands. We compare the skill of three statistical postprocessing methods: an extended logistic regression (ELR), a zero-adjusted gamma distribution (ZAGA), and a machine learning-based method, quantile regression forests (QRF). Forecast skill for low and moderate precipitation thresholds increases with the inclusion of extra predictors, in addition to HA precipitation. HA precipitation is the most important predictor at all lead times in ELR and QRF, while in ZAGA, the most important predictor for the location parameter shifts over lead times from HA precipitation to indices of atmospheric instability. All three methods improve upon a climatological forecast for low and moderate precipitation thresholds. ZAGA and QRF are generally the most skillful methods at moderate thresholds. QRF tends to be the most skillful method at higher thresholds, particularly during the afternoon period. Forecasts are reliable at low and moderate thresholds but tend to be overconfident at higher thresholds. QRF and ZAGA have more potential economic value than the deterministic forecast, with value remaining at high thresholds. A maximum local hourly precipitation threshold of 30 mm h−1 (a criterion in the Royal Netherlands Meteorological Institute’s code yellow warning for severe thunderstorms) is skillfully forecast by QRF in the afternoon period at short lead times.

Generating wind power scenarios for probabilistic ramp event prediction using multivariate statistical post-processing

Abstract. Wind power forecasting is gaining international significance as more regions promote policies to increase the use of renewable energy. Wind ramps, large variations in wind power production during a period of minutes to hours, challenge utilities and electrical balancing authorities. A sudden decrease in wind-energy production must be balanced by other power generators to meet energy demands, while a sharp increase in unexpected production results in excess power that may not be used in the power grid, leading to a loss of potential profits. In this study, we compare different methods to generate probabilistic ramp forecasts from the High Resolution Rapid Refresh (HRRR) numerical weather prediction model with up to 12 h of lead time at two tall-tower locations in the United States. We validate model performance using 21 months of 80 m wind speed observations from towers in Boulder, Colorado, and near the Columbia River gorge in eastern Oregon. We employ four statistical post-processing methods, three of which are not currently used in the literature for wind forecasting. These procedures correct biases in the model and generate short-term wind speed scenarios which are then converted to power scenarios. This probabilistic enhancement of HRRR point forecasts provides valuable uncertainty information of ramp events and improves the skill of predicting ramp events over the raw forecasts. We compute Brier skill scores for each method with regard to predicting up- and down-ramps to determine which method provides the best prediction. We find that the Standard Schaake shuffle method yields the highest skill at predicting ramp events for these datasets, especially for up-ramp events at the Oregon site. Increased skill for ramp prediction is limited at the Boulder, CO, site using any of the multivariate methods because of the poor initial forecasts in this area of complex terrain. These statistical methods can be implemented by wind farm operators to generate a range of possible wind speed and power scenarios to aid and optimize decisions before ramp events occur.

Uncertainty assessment of future projections on water resources according to climate downscaling and hydrological models

Abstract Quantifying the uncertainty of future projection is important to assess the reliable climate change impact. In this sense, this study is aimed at investigating the uncertainty sources of various water variables (seasonal dam inflow, 1-day maximum dam inflow, and 30-day minimum dam inflow) according to downscaling methods and hydrological modeling. Five regional climate models (RCMs), five statistical post-processing methods and two hydrological models were applied for the uncertainty analysis. The changes for seasonal dam inflow are 0.1, 58.8, 5.1, and 1.1 mm for the SWAT model and 2.1, 76.1, −8.5, and −2.9 mm for the VIC model in spring, summer, autumn, and winter, respectively. The effects of the hydrological model is smaller than that of RCM for future projections of the seasonal dam inflow. The changes of annual 1-day maximum dam inflow vary according to the selection of RCM whereas the changes of annual 30-day minimum dam inflow are sensitive to the selection of hydrological model. The RCM is the dominant source of uncertainty of all seasonal dam inflow (except for winter) and high flow, whereas the hydrological model is the dominant source of uncertainty in winter dam inflow and low flow. Considering these results, the appropriate multi-model ensemble chain according to target variable will be necessary for reliable climate change impact assessment.

Skill of Global Raw and Postprocessed Ensemble Predictions of Rainfall over Northern Tropical Africa

AbstractAccumulated precipitation forecasts are of high socioeconomic importance for agriculturally dominated societies in northern tropical Africa. In this study, the performance of nine operational global ensemble prediction systems (EPSs) is analyzed relative to climatology-based forecasts for 1–5-day accumulated precipitation based on the monsoon seasons during 2007–14 for three regions within northern tropical Africa. To assess the full potential of raw ensemble forecasts across spatial scales, state-of-the-art statistical postprocessing methods were applied in the form of Bayesian model averaging (BMA) and ensemble model output statistics (EMOS), and results were verified against station and spatially aggregated, satellite-based gridded observations. Raw ensemble forecasts are uncalibrated and unreliable, and often underperform relative to climatology, independently of region, accumulation time, monsoon season, and ensemble. The differences between raw ensemble and climatological forecasts are large and partly stem from poor prediction for low precipitation amounts. BMA and EMOS postprocessed forecasts are calibrated, reliable, and strongly improve on the raw ensembles but, somewhat disappointingly, typically do not outperform climatology. Most EPSs exhibit slight improvements over the period 2007–14, but overall they have little added value compared to climatology. The suspicion is that parameterization of convection is a potential cause for the sobering lack of ensemble forecast skill in a region dominated by mesoscale convective systems.

Comparison of Statistical Post-Processing Methods for Probabilistic Wind Speed Forecasting

In this study, the statistical post-processing methods that include bias-corrected and probabilistic forecasts of wind speed measured in PyeongChang, which is scheduled to host the 2018 Winter Olympics, are compared and analyzed to provide more accurate weather information. The six post-processing methods used in this study are as follows: mean bias-corrected forecast, mean and variance bias-corrected forecast, decaying averaging forecast, mean absolute bias-corrected forecast, and the alternative implementations of ensemble model output statistics (EMOS) and Bayesian model averaging (BMA) models, which are EMOS and BMA exchangeable models by assuming exchangeable ensemble members and simplified version of EMOS and BMA models. Observations for wind speed were obtained from the 26 stations in PyeongChang and 51 ensemble member forecasts derived from the European Centre for Medium-Range Weather Forecasts (ECMWF Directorate, 2012) that were obtained between 1 May 2013 and 18 March 2016. Prior to applying the post-processing methods, reliability analysis was conducted by using rank histograms to identify the statistical consistency of ensemble forecast and corresponding observations. Based on the results of our study, we found that the prediction skills of probabilistic forecasts of EMOS and BMA models were superior to the biascorrected forecasts in terms of deterministic prediction, whereas in probabilistic prediction, BMA models showed better prediction skill than EMOS. Even though the simplified version of BMA model exhibited best prediction skill among the mentioned six methods, the results showed that the differences of prediction skills between the versions of EMOS and BMA were negligible.

A review on statistical postprocessing methods for hydrometeorological ensemble forecasting

Computer simulation models have been widely used to generate hydrometeorological forecasts. As the raw forecasts contain uncertainties arising from various sources, including model inputs and outputs, model initial and boundary conditions, model structure, and model parameters, it is necessary to apply statistical postprocessing methods to quantify and reduce those uncertainties. Different postprocessing methods have been developed for meteorological forecasts (e.g., precipitation) and for hydrological forecasts (e.g., streamflow) due to their different statistical properties. In this paper, we conduct a comprehensive review of the commonly used statistical postprocessing methods for both meteorological and hydrological forecasts. Moreover, methods to generate ensemble members that maintain the observed spatiotemporal and intervariable dependency are reviewed. Finally, some perspectives on the further development of statistical postprocessing methods for hydrometeorological ensemble forecasting are provided.WIREs Water2017, 4:e1246. doi: 10.1002/wat2.1246This article is categorized under:Science of Water > MethodsScience of Water > Water Extremes

Ensemble Postprocessing of Daily Precipitation Sums over Complex Terrain Using Censored High-Resolution Standardized Anomalies

Probabilistic forecasts provided by numerical ensemble prediction systems have systematic errors and are typically underdispersive. This is especially true over complex topography with extensive terrain-induced small-scale effects, which cannot be resolved by the ensemble system. To alleviate these errors, statistical postprocessing methods are often applied to calibrate the forecasts. This article presents a new full-distributional spatial postprocessing method for daily precipitation sums based on the standardized anomaly model output statistics (SAMOS) approach. Observations and forecasts are transformed into standardized anomalies by subtracting the long-term climatological mean and dividing by the climatological standard deviation. This removes all site-specific characteristics from the data and makes it possible to fit one single regression model for all stations at once. As the model does not depend on the station locations, it directly allows the creation of probabilistic forecasts for any arbitrary location. SAMOS uses a left-censored power-transformed logistic response distribution to account for the large fraction of zero observations (dry days), the limitation to nonnegative values, and the positive skewness of the data. ECMWF reforecasts are used for model training and to correct the ECMWF ensemble forecasts with the big advantage that SAMOS does not require an extensive archive of past ensemble forecasts as only the most recent four reforecasts are needed, and it automatically adapts to changes in the ECMWF ensemble model. The application of the new method to the central Alps shows that the new method is able to depict the small-scale properties and returns accurate fully probabilistic spatial forecasts.

Future Projection and Uncertainty Analysis of Low Flow on Climate Change in Dam Basins

The low flow is the necessary and important index to establish national water planning, however there are lots of uncertainty in the low flow estimation. Therefore, the objectives of this study are to assess the climate change uncertainty and the effects of hydrological models on low flow estimation. The 5 RCMs (HadGEM3-RA, RegCM4, MM5, WRF, and RSM), 5 statistical post-processing methods and 2 hydrological models were applied for evaluation. The study area were selected as Chungju dam and Soyang river dam basin, and the 30 days minimum flow is used for the low flow evaluation. The results of the uncertainty analysis showed that the hydrological model was the largest source of uncertainty about 41.5% in the low flow projection. The uncertainty of hydrological model is higher than the other steps (RCM, statistical post-processing). Also, VIC model is more sensitive for climate change compared to SWAT model. Therefore, the hydrological model should be thoroughly reviewed for the climate change impact assessment on low flow.

Improved Gridded Wind Speed Forecasts by Statistical Postprocessing of Numerical Models with Block Regression

Abstract Numerical weather forecast errors are routinely corrected through statistical postprocessing by several national weather services. These statistical postprocessing methods build a regression function called model output statistics (MOS) between observations and forecasts that is based on an archive of past forecasts and associated observations. Because of limited spatial coverage of most near-surface parameter measurements, MOS have been historically produced only at meteorological station locations. Nevertheless, forecasters and forecast users increasingly ask for improved gridded forecasts. The present work aims at building improved hourly wind speed forecasts over the grid of a numerical weather prediction model. First, a new observational analysis, which performs better in terms of statistical scores than those operationally used at Météo-France, is described as gridded pseudo-observations. This analysis, which is obtained by using an interpolation strategy that was selected among other alternative strategies after an intercomparison study conducted internally at Météo-France, is very parsimonious since it requires only two additive components, and it requires little computational resources. Then, several scalar regression methods are built and compared, using the new analysis as the observation. The most efficient MOS is based on random forests trained on blocks of nearby grid points. This method greatly improves forecasts compared with raw output of numerical weather prediction models. Furthermore, building each random forest on blocks and limiting those forests to shallow trees does not impair performance compared with unpruned and pointwise random forests. This alleviates the storage burden of the objects and speeds up operations.

Comparison of prediction performance using statistical postprocessing methods

As the 2018 Winter Olympics are to be held in Pyeongchang, both general weather information on Pyeongchang and specific weather information on this region, which can affect game operation and athletic performance, are required. An ensemble prediction system has been applied to provide more accurate weather information, but it has bias and dispersion due to the limitations and uncertainty of its model. In this study, homogeneous and nonhomogeneous regression models as well as Bayesian model averaging (BMA) were used to reduce the bias and dispersion existing in ensemble prediction and to provide probabilistic forecast. Prior to applying the prediction methods, reliability of the ensemble forecasts was tested by using a rank histogram and a residualquantile-quantile plot to identify the ensemble forecasts and the corresponding verifications. The ensemble forecasts had a consistent positive bias, indicating over-forecasting, and were under-dispersed. To correct such biases, statistical post-processing methods were applied using fixed and sliding windows. The prediction skills of methods were compared by using the mean absolute error, root mean square error, continuous ranked probability score, and continuous ranked probability skill score. Under the fixed window, BMA exhibited better prediction skill than the other methods in most observation station. Under the sliding window, on the other hand, homogeneous and non-homogeneous regression models with positive regression coefficients exhibited better prediction skill than BMA. In particular, the homogeneous regression model with positive regression coefficients exhibited the best prediction skill.

Uncertainty Assessment of Future High and Low Flow Projections According to Climate Downscaling and Hydrological Models

The quantitative assessment of change in water availability and appropriate water resources management are needed for corresponding adaptation. However, there are large uncertainties in climate change impact assessment on water resources. In this sense, the aims of this study are to suggest the uncertainty assessment method for climate change impact assessment and to investigate the uncertainty characteristics for high and low flow between downscaling methods and hydrological models. The 5 RCMs (HadGEM3-RA, RegCM4, MM5, WRF, and RSM), 5 statistical post-processing methods (SPP), and 2 hydrological models (HYM) were applied on the Chungju dam basin, Korea. The results of uncertainty analysis showed that RCM has the largest sources of uncertainty in 1-day maximum dam inflow (about 40.7%), while HYM has the largest sources of uncertainty in 30-days minimum dam inflow (about 41.5%). In other words, high flow was mainly effected by RCM and low flow by HYM. The proposed methodology in this study can be used to quantify the uncertainties caused by RCM, SPP, and HYM.

Statistical Postprocessing of High-Resolution Regional Climate Model Output

Abstract Statistical postprocessing techniques have become essential tools for downscaling large-scale information to the point scale, and also for providing a better probabilistic characterization of hydrometeorological variables in simulation and forecasting applications at both short and long time scales. In this paper, the authors assess the utility of statistical postprocessing methods for generating probabilistic estimates of daily precipitation totals, using deterministic high-resolution outputs obtained with the Weather Research and Forecasting (WRF) Model. After a preliminary assessment of WRF simulations over a historical period, the performance of three postprocessing techniques is compared: multinomial logistic regression (MnLR), quantile regression (QR), and Bayesian model averaging (BMA)—all of which use WRF outputs as potential predictors. Results demonstrate that the WRF Model has skill in reproducing observed precipitation events, especially during fall/winter. Furthermore, it is shown that the spatial distribution of skill obtained from statistical postprocessing is closely linked with the quality of WRF precipitation outputs. A detailed comparison of statistical precipitation postprocessing approaches reveals that, although the poorest performance was obtained using MnLR, there is not an overall best technique. While QR should be preferred if skill (i.e., small probability forecast errors) and reliability (i.e., match between forecast probabilities and observed frequencies) are target properties, BMA is recommended in cases when discrimination (i.e., prediction of occurrence versus nonoccurrence) and statistical consistency (i.e., equiprobability of the observations within their ensemble distributions) are desired. Based on the results obtained here, the authors believe that future research should explore frameworks reconciling hierarchical Bayesian models with the use of the extreme value theory for high precipitation events.

Comparing statistical post-processing of regional and global climate scenarios for hydrological impacts assessment: A case study of two Canadian catchments

Summary Projected climate change effects on streamflow are investigated for the 2041–2070 horizon following the SRES A2 emissions scenario over two snowmelt-dominated catchments in Canada. A 16-member ensemble of SWAT hydrological model (HM) simulations, based on a comprehensive ensemble of the Canadian Regional Climate Model (CRCM) simulations driven by two global climate models (GCMs), with five realizations of the Canadian CGCM3 and three realizations of the German ECHAM5 is established per catchment. This study aims to evaluate, once model bias has been removed by statistical post-processing (SP), how the RCM-simulated climate changes differ from those of the parent GCMs, and how they affect the assessment of climate change-induced hydrological impacts at the catchment scale. The variability of streamflow caused by the use of different SP methods (mean-based versus distribution-based) within each statistical post-processing pathway of climate model outputs (bias correction versus perturbation) is also evaluated, as well as the uncertainty of natural climate variability. The simulations cover 1971–2000 in the reference period and 2041–2070 in the future period. For a set of criteria, results based on raw and statistically post-processed model outputs for the reference climate are compared with observations. This process demonstrates that SP is important not only for GCMs outputs, but also for CRCM outputs. SP leads to a high level of agreement between the CRCM and the driving GCMs in reproducing patterns of observed climate. The ensemble spread of the climate change signal on streamflow is large and varies with catchments and hydrological periods (winter/summer flows). The results of various hydrological indicators show that most of the uncertainty arises from the natural climate variability followed by the statistical post-processing. The uncertainty linked to the choice of statistical pathway is much larger than that associated with the choice of the method in quantifying the hydrological impacts. We find that the incorporation of dynamical downscaling of global models through the CRCM as an intermediate step in the GCM–RCM–SP–HM model chain does not lead to considerable differences in the assessment of the climate change impacts on streamflow for the study catchments.

Calibrating 2-m Temperature of Limited-Area Ensemble Forecasts Using High-Resolution Analysis

Abstract Although the quality of numerical ensemble prediction systems (EPS) has greatly improved during the last few years, these systems still show systematic deficiencies. Specifically, they are underdispersive and lack both reliability and sharpness. A variety of statistical postprocessing methods allows for improving direct model output. Since 2007, Aire Limitée Adaptation Dynamique Développement International Limited Area Ensemble Forecasting (ALADIN-LAEF) has been in operation at the Central Institute for Meteorology and Geodynamics (ZAMG), and its 2-m temperature model output subject to calibration. This work follows the approach of nonhomogeneous Gaussian regression (NGR) that addresses a statistical correction of the first and second moment (mean bias and dispersion) for Gaussian-distributed continuous variables. It is based on the multiple linear regression technique and provides a predictive probability density function (PDF) in terms of a normal distribution. Fitting the regression coefficients, a minimum continuous ranked probability score (CRPS) estimation has been chosen instead of the more traditional maximum likelihood technique. The use of high-resolution analysis data on a 1 km × 1 km grid as training data improves the forecast skill in terms of CRPS by about 35%, especially on the local scale. The percentage of outliers decreases significantly without loss of sharpness. Sensitivity studies confirm that about half of the total improvement can be attributed to the effect of a bias correction. The training length plays a minor role, at least for the chosen verification period. A rescaling of the predictive PDF is important in order to obtain sharp forecasts, especially in the short range. Applying the same method to the global ensemble from the European Centre for Medium-Range Weather Forecasts (ECMWF) gives improvements of similar magnitude. However, the calibrated 2-m temperature of ALADIN-LAEF still remains slightly better than the 2-m temperature from calibrated ECMWF-EPS, which leads to the conclusion that statistical downscaling of EPS cannot replace dynamical downscaling. Finally, an advanced version of NGR, the so-called NGR-TD, which uses time-weighted averaging within minimum CRPS estimation, is able to yield a further improvement of about 5% in terms of the CRPS.

Comments on “Hydrometeorological Accuracy Enhancement via Postprocessing of Numerical Weather Forecasts in Complex Terrain”

In their paper, McCollor and Stull (2008, hereinafter referred to as MS08) show test results obtained using several simple statistical postprocessing methods applied to temperature and precipitation forecasts from the Canadian Global NWP model. Based on 1 yr of observations from 19 stations, the methods were evaluated for forecast ranges up to day 8. The six test methods were also evaluated against forecasts from the Canadian Meteorological Centre’s (CMC) Updateable Model Output Statistics (UMOS) system (Wilson and Vallee 2002, 2003), and against the original direct model output forecasts (DMO), for the same period as a standard of comparison. This latter comparison was done for the six stations where UMOS forecasts were available. An interesting feature of the paper is the use of the time-integrated forecast–observed water mass balance ratio as the evaluation variable for precipitation forecasts. Based on the 1-yr test dataset, MS08 concludes that the six simple bias correction methods outperform the direct model output forecasts and the model output statistics (MOS) forecasts for temperature and that the three postprocessing methods that could be applied to 24-h precipitation outperformed the direct model output precipitation forecasts.

Reducing the unwanted draining vein BOLD contribution in fMRI with statistical post-processing methods

Recent BOLD fMRI data analysis methods show promise in reducing contributions from draining veins. The phase regressor method developed by [Menon, R.S., 2002. Post-acquisition suppression of large-vessel BOLD signals in high-resolution fMRI. Magn. Reson. Med., 47, 1–9] creates phase and magnitude images, regresses magnitude as a function of phase, and subtracts phase-estimated magnitudes from the observed magnitudes. The corrected magnitude images are used to compute cortical activations. The complex constant phase method, developed by [Rowe, D.B., Logan, B.R., 2004. A complex way to compute fMRI activation. NeuroImage, 23, 1078–1092], uses complex-valued reconstructed images and a nonlinear regressor model to compute magnitude cortical activations assuming temporally constant phase. In both methods, the usage of the phase information is claimed to bias against voxels with task-related phase changes caused by some draining veins. The behavior of the statistical methods in data with several task-related magnitude and phase changes is compared. The power of the statistical methods for determining voxels with specific task-related magnitude and phase change combinations are determined in ideal simulated data. The phase regressor and complex constant phase activation determination techniques are examined to characterize the responses of the models to select task-related phase and magnitude change combinations in representative simulated time series. Possible draining veins in human preliminary data are discussed and analyzed with the models and the current challenges which prevent these methods from being reliably implemented are discussed.

Improvement of Local Categorical Precipitation Forecasts from an NWP Model by Various Statistical Postprocessing Methods

Several statistical postprocessing methods are applied to results from a numerical weather prediction (NWP) model to test the potential for increasing the accuracy of its local precipitation forecasts. Categorical (Yes/No) forecasts for 12hr precipitation sums equalling or exceeding 0.1, 2.0 and 5.0 mm are selected for improvement. The two 12hr periods 0600-1800 UTC and 1800-0600 UTC are treated separately based on NWP model initial times 0000 UTC and 1200 UTC, respectively. Input data are taken from three successive summer seasons, April-September, 1994-96. The forecasts are prepared and verified for five synoptic stations, four located in the western Czech Republic, and one in Germany near the Czech-German border.