Statistical Postprocessing Research Articles

Discrete Postprocessing of Total Cloud Cover Ensemble Forecasts

AbstractThis paper presents an approach to postprocess ensemble forecasts for the discrete and bounded weather variable of total cloud cover. Two methods for discrete statistical postprocessing of ensemble predictions are tested: the first approach is based on multinomial logistic regression and the second involves a proportional odds logistic regression model. Applying them to total cloud cover raw ensemble forecasts from the European Centre for Medium-Range Weather Forecasts improves forecast skill significantly. Based on stationwise postprocessing of raw ensemble total cloud cover forecasts for a global set of 3330 stations over the period from 2007 to early 2014, the more parsimonious proportional odds logistic regression model proved to slightly outperform the multinomial logistic regression model.

New method of determination of spot welding-adhesive joint fatigue life using full field strain evolution

Fatigue tests were conducted since more than two hundred years ago. Despite this long period, as fatigue phenomena are very complex, assessment of fatigue response of standard materials or composites still requires a long time. Quite precise way to estimate fatigue parameters is to test at least 30 standardized specimens for the analysed material and further statistical post processing is required. In case of structural elements analysis like hybrid joints (Figure 1), the situation is much more complex as more factors influence the fatigue load capacity due to much more complicated structure of the joint in comparison to standard materials specimen, i.e. occurrence of: welded hot spots or rivets, adhesive layers, local notches creating the stress concentrations, etc.In order to shorten testing time some rapid methods are known: Locati's method [1] - step by step load increments up to failure, Prot's method [2] - constant increase of the load amplitude up to failure; Lehr's method [2] - seeking for the point during regular fatigue loading when an increase of temperature or strains become non-linear. The present article proposes new method of the fatigue response assessment - combination of the Locati's and Lehr's method.

Probing NWP model deficiencies by statistical postprocessing

This article describes a Model Output Statistics (MOS) framework for wind speed forecast accuracy improvement. The approach is used to identify non‐intuitive explanatory value from a diagnostic variable in an operational numerical weather prediction (NWP) model generating global weather forecasts four times daily, with numerous users worldwide. The analysis is based on two years of hourly wind speed time series measured at three locations: offshore, in coastal flat terrain, and inland in complex topography. Together with wind speed and wind direction predictors derived from the prognostic wind field, the lifted index NWP model diagnostic is found to perform well as statistical wind speed forecast model predictor.

Comparison of Different Configurations of Quantile Regression in Estimating Predictive Hydrological Uncertainty

Three different configurations of quantile regression as a statistical post processing technique to predict hydrological uncertainty in flood forecasts are investigated. The first configuration applies quantile regression of forecast errors conditioned on forecast discharge in original domain (QR-ORI) while the second configuration includes normal quantile transformation of forecast discharge and forecast errors prior to quantile regression (QR-NQT) to take into account the heteroscedasticity of the forecast errors. A weighted quantile regression is introduced as the third configuration (QR-WT) where weights are assigned during calibration of the quantile regression model to put more emphasis on higher flows, which is the regime of interest for flood forecasting systems. Probabilistic forecasts derived using these three different configurations are compared using different verification measures for the flood forecasting system of Sava River in Slovenia. Results show an improvement of QR-NQT over QR-ORI, largely depending on the size of the training and validation data set and the range of discharges in both data sets. It is also shown that QR-WT outperforms QR-ORI when the highest 25% discharges are considered.

Statistical Postprocessing of Ensemble Precipitation Forecasts by Fitting Censored, Shifted Gamma Distributions*

Abstract A parametric statistical postprocessing method is presented that transforms raw (and frequently biased) ensemble forecasts from the Global Ensemble Forecast System (GEFS) into reliable predictive probability distributions for precipitation accumulations. Exploratory analysis based on 12 years of reforecast data and ⅛° climatology-calibrated precipitation analyses shows that censored, shifted gamma distributions can well approximate the conditional distribution of observed precipitation accumulations given the ensemble forecasts. A nonhomogeneous regression model is set up to link the parameters of this distribution to ensemble statistics that summarize the mean and spread of predicted precipitation amounts within a certain neighborhood of the location of interest, and in addition the predicted mean of precipitable water. The proposed method is demonstrated with precipitation reforecasts over the conterminous United States using common metrics such as Brier skill scores and reliability diagrams. It yields probabilistic forecasts that are reliable, highly skillful, and sharper than the previously demonstrated analog procedure. In situations with limited predictability, increasing the size of the neighborhood within which ensemble forecasts are considered as predictors can further improve forecast skill. It is found, however, that even a parametric postprocessing approach crucially relies on the availability of a sufficiently large training dataset.

Statistical Postprocessing of High-Resolution Regional Climate Model Output

Abstract Statistical postprocessing techniques have become essential tools for downscaling large-scale information to the point scale, and also for providing a better probabilistic characterization of hydrometeorological variables in simulation and forecasting applications at both short and long time scales. In this paper, the authors assess the utility of statistical postprocessing methods for generating probabilistic estimates of daily precipitation totals, using deterministic high-resolution outputs obtained with the Weather Research and Forecasting (WRF) Model. After a preliminary assessment of WRF simulations over a historical period, the performance of three postprocessing techniques is compared: multinomial logistic regression (MnLR), quantile regression (QR), and Bayesian model averaging (BMA)—all of which use WRF outputs as potential predictors. Results demonstrate that the WRF Model has skill in reproducing observed precipitation events, especially during fall/winter. Furthermore, it is shown that the spatial distribution of skill obtained from statistical postprocessing is closely linked with the quality of WRF precipitation outputs. A detailed comparison of statistical precipitation postprocessing approaches reveals that, although the poorest performance was obtained using MnLR, there is not an overall best technique. While QR should be preferred if skill (i.e., small probability forecast errors) and reliability (i.e., match between forecast probabilities and observed frequencies) are target properties, BMA is recommended in cases when discrimination (i.e., prediction of occurrence versus nonoccurrence) and statistical consistency (i.e., equiprobability of the observations within their ensemble distributions) are desired. Based on the results obtained here, the authors believe that future research should explore frameworks reconciling hierarchical Bayesian models with the use of the extreme value theory for high precipitation events.

Comparing statistical post-processing of regional and global climate scenarios for hydrological impacts assessment: A case study of two Canadian catchments

Summary Projected climate change effects on streamflow are investigated for the 2041–2070 horizon following the SRES A2 emissions scenario over two snowmelt-dominated catchments in Canada. A 16-member ensemble of SWAT hydrological model (HM) simulations, based on a comprehensive ensemble of the Canadian Regional Climate Model (CRCM) simulations driven by two global climate models (GCMs), with five realizations of the Canadian CGCM3 and three realizations of the German ECHAM5 is established per catchment. This study aims to evaluate, once model bias has been removed by statistical post-processing (SP), how the RCM-simulated climate changes differ from those of the parent GCMs, and how they affect the assessment of climate change-induced hydrological impacts at the catchment scale. The variability of streamflow caused by the use of different SP methods (mean-based versus distribution-based) within each statistical post-processing pathway of climate model outputs (bias correction versus perturbation) is also evaluated, as well as the uncertainty of natural climate variability. The simulations cover 1971–2000 in the reference period and 2041–2070 in the future period. For a set of criteria, results based on raw and statistically post-processed model outputs for the reference climate are compared with observations. This process demonstrates that SP is important not only for GCMs outputs, but also for CRCM outputs. SP leads to a high level of agreement between the CRCM and the driving GCMs in reproducing patterns of observed climate. The ensemble spread of the climate change signal on streamflow is large and varies with catchments and hydrological periods (winter/summer flows). The results of various hydrological indicators show that most of the uncertainty arises from the natural climate variability followed by the statistical post-processing. The uncertainty linked to the choice of statistical pathway is much larger than that associated with the choice of the method in quantifying the hydrological impacts. We find that the incorporation of dynamical downscaling of global models through the CRCM as an intermediate step in the GCM–RCM–SP–HM model chain does not lead to considerable differences in the assessment of the climate change impacts on streamflow for the study catchments.

Uncertainty Quantification in Complex Simulation Models Using Ensemble Copula Coupling

Critical decisions frequently rely on high-dimensional output from complex computer simulation models that show intricate cross-variable, spatial and temporal dependence structures, with weather and climate predictions being key examples. There is a strongly increasing recognition of the need for uncertainty quantification in such settings, for which we propose and review a general multi-stage procedure called ensemble copula coupling (ECC), proceeding as follows: 1. Generate a raw ensemble, consisting of multiple runs of the computer model that differ in the inputs or model parameters in suitable ways. 2. Apply statistical postprocessing techniques, such as Bayesian model averaging or nonhomogeneous regression, to correct for systematic errors in the raw ensemble, to obtain calibrated and sharp predictive distributions for each univariate output variable individually. 3. Draw a sample from each postprocessed predictive distribution. 4. Rearrange the sampled values in the rank order structure of the raw ensemble to obtain the ECC postprocessed ensemble. The use of ensembles and statistical postprocessing have become routine in weather forecasting over the past decade. We show that seemingly unrelated, recent advances can be interpreted, fused and consolidated within the framework of ECC, the common thread being the adoption of the empirical copula of the raw ensemble. Depending on the use of Quantiles, Random draws or Transformations at the sampling stage, we distinguish the ECC-Q, ECC-R and ECC-T variants, respectively. We also describe relations to the Schaake shuffle and extant copula-based techniques. In a case study, the ECC approach is applied to predictions of temperature, pressure, precipitation and wind over Germany, based on the 50-member European Centre for Medium-Range Weather Forecasts (ECMWF) ensemble.

Simultaneous calibration of ensemble river flow predictions over an entire range of lead times

[1] Probabilistic estimates of future water levels and river discharge are usually simulated with hydrologic models using ensemble weather forecasts as main inputs. As hydrologic models are imperfect and the meteorological ensembles tend to be biased and underdispersed, the ensemble forecasts for river runoff typically are biased and underdispersed, too. Thus, in order to achieve both reliable and sharp predictions statistical postprocessing is required. In this work Bayesian model averaging (BMA) is applied to statistically postprocess ensemble runoff raw forecasts for a catchment in Switzerland, at lead times ranging from 1 to 240 h. The raw forecasts have been obtained using deterministic and ensemble forcing meteorological models with different forecast lead time ranges. First, BMA is applied based on mixtures of univariate normal distributions, subject to the assumption of independence between distinct lead times. Then, the independence assumption is relaxed in order to estimate multivariate runoff forecasts over the entire range of lead times simultaneously, based on a BMA version that uses multivariate normal distributions. Since river runoff is a highly skewed variable, Box-Cox transformations are applied in order to achieve approximate normality. Both univariate and multivariate BMA approaches are able to generate well calibrated probabilistic forecasts that are considerably sharper than climatological forecasts. Additionally, multivariate BMA provides a promising approach for incorporating temporal dependencies into the postprocessed forecasts. Its major advantage against univariate BMA is an increase in reliability when the forecast system is changing due to model availability.

Spread Calibration of Ensemble MOS Forecasts

Abstract Ensemble forecasting systems often contain systematic biases and spread deficiencies that can be corrected by statistical postprocessing. This study presents an improvement to an ensemble statistical postprocessing technique, called ensemble kernel density model output statistics (EKDMOS). EKDMOS uses model output statistics (MOS) equations and spread–skill relationships to generate calibrated probabilistic forecasts. The MOS equations are multiple linear regression equations developed by relating observations to ensemble mean-based predictors. The spread–skill relationships are one-term linear regression equations that predict the expected accuracy of the ensemble mean given the ensemble spread. To generate an EKDMOS forecast, the MOS equations are applied to each ensemble member. Kernel density fitting is used to create a probability density function (PDF) from the ensemble MOS forecasts. The PDF spread is adjusted to match the spread predicted by the spread–skill relationship, producing a calibrated forecast. The improved EKDMOS technique was used to produce probabilistic 2-m temperature forecasts from the North American Ensemble Forecast System (NAEFS) over the period 1 October 2007–31 March 2010. The results were compared with an earlier spread adjustment technique, as well as forecasts generated by rank sorting the bias-corrected ensemble members. Compared to the other techniques, the new EKDMOS forecasts were more reliable, had a better calibrated spread–error relationship, and showed increased day-to-day spread variability.

1106 EFFECTS OF PRESSURE FLUCTUATIONS INDUCED BY SEPARATED FLOW ON TRANSMISSION NOISE IN WAVENUMBER-FREQUENCY DOMAIN

When investigating the transmission noise like a vehicle interior noise, it is very important to consider the pressure fluctuations in the near acoustic field generated by air flow around a vehicle. However, since these pressure fluctuations are constructed by both acoustic and air flow fluctuations, it is difficult to separate and evaluate the levels of these two components individually. Therefore, in this paper, the wavenumber-frequency spectrum method was applied to analyze the flow field and the sound field around a square cylinder which was placed near a flat plate in order to examine the behavior of the pressure fluctuations in the near acoustic field. The statistical post processing method based on a time-space correlation was used to analyze acoustic and air flow fluctuations in a wavenumber domain. Thus, the differences in the convective velocity and the levels of sound and turbulent flow pressure fluctuations were investigated by changing the distance between the square cylinder and the flat plate.

Verification of Precipitation Forecasts from NCEP’s Short-Range Ensemble Forecast (SREF) System with Reference to Ensemble Streamflow Prediction Using Lumped Hydrologic Models

Abstract Precipitation forecasts from the Short-Range Ensemble Forecast (SREF) system of the National Centers for Environmental Prediction (NCEP) are verified for the period April 2006–August 2010. Verification is conducted for 10–20 hydrologic basins in each of the following: the middle Atlantic, the southern plains, the windward slopes of the Sierra Nevada, and the foothills of the Cascade Range in the Pacific Northwest. Mean areal precipitation is verified conditionally upon forecast lead time, amount of precipitation, season, forecast valid time, and accumulation period. The stationary block bootstrap is used to quantify the sampling uncertainties of the verification metrics. In general, the forecasts are more skillful for moderate precipitation amounts than either light or heavy precipitation. This originates from a threshold-dependent conditional bias in the ensemble mean forecast. Specifically, the forecasts overestimate low observed precipitation and underestimate high precipitation (a type-II conditional bias). Also, the forecast probabilities are generally overconfident (a type-I conditional bias), except for basins in the southern plains, where forecasts of moderate to high precipitation are reliable. Depending on location, different types of bias correction may be needed. Overall, the northwest basins show the greatest potential for statistical postprocessing, particularly during the cool season, when the type-I conditional bias and correlations are both high. The basins of the middle Atlantic and southern plains show less potential for statistical postprocessing, as the type-II conditional bias is larger and the correlations are weaker. In the Sierra Nevada, the greatest benefits of statistical postprocessing should be expected for light precipitation, specifically during the warm season, when the type-I conditional bias is large and the correlations are strong.

Wind Speeds at Heights Crucial for Wind Energy: Measurements and Verification of Forecasts

AbstractWind speed measurements from one year from meteorological towers and wind turbines at heights between 20 and 250 m for various European sites are analyzed and are compared with operational short-term forecasts of the global ECMWF model. The measurement sites encompass a variety of terrain: offshore, coastal, flat, hilly, and mountainous regions, with low and high vegetation and also urban influences. The strongly differing site characteristics modulate the relative contribution of synoptic-scale and smaller-scale forcing to local wind conditions and thus the performance of the NWP model. The goal of this study was to determine the best-verifying model wind among various standard wind outputs and interpolation methods as well as to reveal its skill relative to the different site characteristics. Highest skill is reached by wind from a neighboring model level, as well as by linearly interpolated wind from neighboring model levels, whereas the frequently applied 10-m wind logarithmically extrapolated to higher elevations yields the largest errors. The logarithmically extrapolated 100-m model wind reaches the best compromise between availability and low cost for data even when the vertical resolution of the model changes. It is a good choice as input for further statistical postprocessing. The amplitude of measured, height-dependent diurnal variations is underestimated by the model. At low levels, the model wind speed is smaller than observed during the day and is higher during the night. At higher elevations, the opposite is the case.

Improving Predictions using Ensemble Bayesian Model Averaging

We present ensemble Bayesian model averaging (EBMA) and illustrate its ability to aid scholars in the social sciences to make more accurate forecasts of future events. In essence, EBMA improves prediction by pooling information from multiple forecast models to generate ensemble predictions similar to a weighted average of component forecasts. The weight assigned to each forecast is calibrated via its performance in some validation period. The aim is not to choose some “best” model, but rather to incorporate the insights and knowledge implicit in various forecasting efforts via statistical postprocessing. After presenting the method, we show that EBMA increases the accuracy of out-of-sample forecasts relative to component models in three applied examples: predicting the occurrence of insurgencies around the Pacific Rim, forecasting vote shares in U.S. presidential elections, and predicting the votes of U.S. Supreme Court Justices.

Effects of Model Resolution and Statistical Postprocessing on Shelter Temperature and Wind Forecasts

AbstractShelter temperature and wind forecasts from numerical weather prediction models are subject to large systematic errors. Kalman filtering and model output statistics (MOS) are commonly used postprocessing methods, but how effective are they in comparison with steadily increasing resolution of the forecast model? Observations from over 1100 stations in central Europe are used to compare the different postprocessing methods and the influence of model resolution in complex and simple terrain, respectively. A 1-yr period with hourly, or at least 3-hourly, data is used to achieve statistically meaningful results. Furthermore, the importance of real-time observations as MOS predictors and the effects of daily training of the MOS equations are studied.

A New Approach for Rolling Contact Fatigue Numerical Study. Application to a Brittle Epoxy Resin

The background of the present study is the rolling contact fatigue (RCF) in a brittle polymer disk. The disk has been tested on a two disk machine, under controlled normal and tangential loads, with no global slip. After several million cycles and under different operating conditions, it has been observed that (1) the tangential load highly influences the RCF phenomenon, (2) a network of regularly spaced cracks appears, and (3) in the driving position, the RCF phenomenon develops faster. To explain these observations, a numerical model based on the finite element method (FEM) has been built: the cracks have been quite simply modeled, stick-slip has been chosen as the friction model, and the disk-on-disk contact has been replaced by a disk-on-plane contact. To study the influence of some of the operating conditions, the design of experiments (DOE) techniques has been used. The statistical postprocessing associated to DOE has confirmed the experimental observations with a good reliability. In addition, with some mechanical considerations, scenarios of what experimentally happens are proposed. The association FEM/DOE is an original and efficient way to explain phenomena in the field of RCF: the accuracy of the FEM coupled with DOE statistical treatments make it possible to have a good predictability despite some uncontrolled parameters.

Probabilistic Forecasts Using Analogs in the Idealized Lorenz96 Setting

Abstract Three methods to make probabilistic weather forecasts by using analogs are presented and tested. The basic idea of these methods is that finding similar NWP model forecasts to the current one in an archive of past forecasts and taking the corresponding analyses as prediction should remove all systematic errors of the model. Furthermore, this statistical postprocessing can convert NWP forecasts to forecasts for point locations and easily turn deterministic forecasts into probabilistic ones. These methods are tested in the idealized Lorenz96 system and compared to a benchmark bracket formed by ensemble relative frequencies from direct model output and logistic regression. The analog methods excel at longer lead times.

IDDQ Test Practice in Nanotechnologies

As semiconductor wafer process technology went into nanotech level, IDDQ test faced tremendous challenges than it ever did due to high intrinsic leakage. A case study of IDDQ test in Availink ASICs in 130nm and 65nm process technologies demonstrates that IDDQ test is still feasible. The scheme includes IDDQ test vectors precise selection/generation, IDDQ measurement with high resolution/repeatability, proper statistical post processing algorithm in outlier detection, results verification, etc. As a result, IDDQ test is still considerable in manufacturing test strategy in nanotechnologies era.

Dynamical, Statistical–Dynamical, and Multimodel Ensemble Forecasts of Australian Spring Season Rainfall

Abstract The prediction skill of the Australian Bureau of Meteorology dynamical seasonal forecast model Predictive Ocean Atmosphere Model for Australia (POAMA) is assessed for probabilistic forecasts of spring season rainfall in Australia and the feasibility of increasing forecast skill through statistical postprocessing is examined. Two statistical postprocessing techniques are explored: calibrating POAMA prediction of rainfall anomaly against observations and using dynamically predicted mean sea level pressure to infer regional rainfall anomaly over Australia (referred to as “bridging”). A “homogeneous” multimodel ensemble prediction method (HMME) is also introduced that consists of the combination of POAMA’s direct prediction of rainfall anomaly together with the two statistically postprocessed predictions. Using hindcasts for the period 1981–2006, the direct forecasts from POAMA exhibit skill relative to a climatological forecast over broad areas of eastern and southern Australia, where El Niño and the Indian Ocean dipole (whose behavior POAMA can skillfully predict at short lead times) are known to exert a strong influence in austral spring. The calibrated and bridged forecasts, while potentially offering improvement over the direct forecasts because of POAMA’s ability to predict the main drivers of springtime rainfall (e.g., El Niño and the Southern Oscillation), show only limited areas of improvement, mainly because strict cross-validation limits the ability to capitalize on relatively modest predictive signals with short record lengths. However, when POAMA and the two statistical–dynamical rainfall forecasts are combined in the HMME, higher deterministic and probabilistic skill is achieved over any of the single models, which suggests the HMME is another useful method to calibrate dynamical model forecasts.

Climatic Study of Precipitation and Temperature Distribution Over the Alps and the Italian Peninsula, Using the High-Resolution Regional Climate Model COSMO-CLM

This report is concentrated on the High-Resolution Regional Climate Model COSMO-CLM and the climatic and statistical analysis of the model’s results concerning the precipitation and temperature on the Alpine area and the Italian Peninsula. The study represents a first attempt aimed to investigate the impact of climate change in these areas. For this purpose, firstly, an observational database for Italy and the Alpine area was created, which was used for the validation and statistical analysis of the COSMO model results. Then, a climatic analysis was performed for the precipitation and temperature data that derive from two regional climate simulations of the climate of the XX century (control run) using the model Cosmo-CLM and employing for the generation of boundary conditions, respectively, ECHAM4 T106 (~ 120km resolution, 12h time resolution) driving data and ERA40 (~ 120km resolution, 6h time res.). The space resolution of both simulations was 14Km (1/8 with respect to the global model) and the time step 150 sec. The study concerned the Alpine area extending from 2° -20° East and 40° - 52° North, for the time period 1971-2000. Furthermore, precipitation and temperature data output that derived from the regional climate simulation of the Cosmo-CLM model for the period 2000-2100, were studied. For the future simulation, the IPCC A1B emission scenario was employed. The space resolution of the simulation was 14Km (1/8 with respect to the global model) and the time step 150sec. The ECHAM4 global driving data were used. The model output for both time periods were compared in order to detect any potential future changes in the precipitation and temperature spatial distribution over the Alpine area. Finally, due to the still occurring misrepresentation of local climate in regional climate model COSMO-CLM, statistical post processing was applied to overcome this problem leading to qualitatively enhanced climate information. More specifically, in the frame of the activity of the European project TRUST, the statistical bias correction method Quantile Mapping was performed for the local area of Veneto (11.5°-12°E , 45.8°-46.3), in order to compare and analyze the relation of atmospheric parameters from climate model simulations to meteorological observations. The results of the study are presented analytically in following section.