Forecast Standard Deviation Research Articles

Forecaster Overconfidence and Market Survey Performance

The authors document using the ZEW panel of German stock market forecasters that weak forecasters tend to be overconfident in the sense that they provide extreme forecasts and their confidence intervals are less likely to contain eventual realizations. They further show that moderate filters based on forecast accuracy of past performance over short rolling windows, which delicately balance ignoring relevant information and noise reduction, are somewhat successful in improving predictability. While poor performance can be due to various factors, a filter based on forecaster overconfidence, a prior tendency to have high forecast standard deviations, also improves the performance of market survey forecasts.

A Comparison of the Downstream Predictability Associated with ET and Baroclinic Cyclones

The impact of the extratropical transition (ET) of tropical cyclones and baroclinic cyclogenesis in the western North Pacific (WNP), Atlantic, and southern Indian Ocean (SIO) basins on the predictability of the downstream midlatitude flow is assessed using 30 years of cases from the Global Ensemble Forecast System (GEFS) Reforecast, version 2. In all three basins, ET is associated with statistically larger 500-hPa geopotential height forecast standard deviation (SD) compared to the forecast climatology. The higher SD values originate from where the TC enters the midlatitudes and spread downstream at the group velocity of the associated wave packet. Of the three basins, WNP ET is associated with the largest amplitude and longest-lasting SD anomalies. Forecasts initialized 2–4 days prior to the onset of ET have larger SD anomalies compared to forecasts initialized during or after the onset of ET. By contrast, the region of positive SD anomaly associated with winter baroclinic cyclones is confined to the upstream trough, with fall cyclones exhibiting some downstream propagation characteristics similar to ET. The ET cases with the larger downstream SD anomaly are characterized by a more amplified ridge downstream of the TC as it enters the midlatitudes. By contrast, ET cases with an upstream trough, large TC position variability at the onset of ET, latent heat release, or upper-tropospheric PV advection by the irrotational wind are not characterized by significantly larger downstream SD compared to cases without an upstream trough or smaller values of these quantities.

Impact of Bias Magnification on Supply Chain Costs: The Mitigating Role of Forecast Sharing

ABSTRACTThe impact of forecast error magnification on supply chain cost has been well documented. Unlike past studies that measure forecast error in terms of forecast standard deviation, our study extends research to consider the impact of forecast bias, and the complex interaction between these variables. Simulating a two‐stage supply chain using realistic cost data we test the impact of bias magnification comparing two scenarios: one with forecast sharing between retailer and supplier, and one without. We then corroborate findings via survey data. Results show magnification of forecast bias to have a considerably greater impact on supply chain cost than magnification of forecast standard deviation. Particularly damaging is high bias in the presence of high forecast standard deviation. Forecast sharing is found to mitigate the impact of forecast error, however, primarily at higher levels of forecast standard deviation. At low levels of forecast standard deviation the benefits are not significant suggesting that engaging in such mitigation strategies may be less effective when there is little opportunity for improvement in accuracy. Furthermore, forecast sharing is found to be much less effective against high levels of bias. This is an important finding as managers often deliberately bias their forecasts and underscores the importance of exercising caution even with forecast sharing, particularly for forecasts that have inherently large errors. The findings provide a deeper understanding of the impact of forecast errors, suggest limitations of forecast sharing, and offer implications for research and practice alike.

Forecast Uncertainty and the Bank of England Interest Rate Decisions

To assess the Bank of England’s Monetary Policy Committee decisions on the official bank rate under forecast uncertainty, I estimate simple forecast-based interest rate rules augmented by the exact forecast standard deviations recovered directly from the Inflation Report fan charts. I find that forecast inflation uncertainty strongly intensifies the reaction of the interest rate decisions to a forecast deviation of inflation from target. Conversely, forecast output growth uncertainty attenuates the reaction of the interest rate decisions to a forecast deviation of output growth from its long-run mean. Asymmetries in forecast uncertainty are highly relevant for inflation. Forecast upward risks to inflation contribute strongly to the intensifying effect of forecast inflation uncertainty, while forecast downward risks have hardly any significant impact. Moreover, I find that forecast risks to inflation have a direct effect on the interest rate decisions, in particular when inflation is forecast close to target. Uncertainty forecasts obtained from the Survey of External Forecasters, though, contain no explanatory power.

On the different approaches of measuring uncertainty shocks

As uncertainty has become an increasingly prominent source of business cycle fluctuations, various uncertainty proxies have been proposed in the literature. This paper shows that uncertainty measures based on realized variables fluctuate more than the measures that are based on forecasts. More precisely, the variation in the realized cross-sectional standard deviation of profit growth and stock returns is larger than the variation in the forecast standard deviation. Moreover, the forecast standard deviation of profit growth and stock returns are negatively or uncorrelated, the uncertainty measures increase stock returns due to a risk premium, but they decrease profit growth.

Decomposition of a New Proper Score for Verification of Ensemble Forecasts

Abstract A new proper score, the error-spread score (ES), has recently been proposed for evaluation of ensemble forecasts of continuous variables. The ES is formulated with respect to the moments of the ensemble forecast. It is particularly sensitive to evaluating how well an ensemble forecast represents uncertainty: is the probabilistic forecast well calibrated? In this paper, it is shown that the ES can be decomposed into its reliability, resolution, and uncertainty components in a similar way to the Brier score. The first term evaluates the reliability of the forecast standard deviation and skewness, rewarding systems where the forecast moments reliably indicate the properties of the verification. The second term evaluates the resolution of the forecast standard deviation and skewness, and rewards systems where the forecast moments vary from the climatological moments according to the predictability of the atmospheric flow. The uncertainty term depends only on the observed error distribution and is independent of the forecast standard deviation or skewness. The decomposition was demonstrated using forecasts made with the European Centre for Medium-Range Weather Forecasts ensemble prediction system, and was able to identify the source of the skill in the forecasts at different latitudes.

A comparative verification of forecasts from two operational solar wind models

[1] The solar wind (SW) and interplanetary magnetic field (IMF) have a significant influence on the near-Earth space environment. In this study we evaluate and compare forecasts from two models that predict SW and IMF conditions: the Hakamada-Akasofu-Fry (HAF) version 2, operational at the Air Force Weather Agency, and Wang-Sheeley-Arge (WSA) version 1.6, executed routinely at the Space Weather Prediction Center. SW speed (Vsw) and IMF polarity (Bpol) forecasts at L1 were compared with Wind and Advanced Composition Explorer satellite observations. Verification statistics were computed by study year and forecast day. Results revealed that both models' mean Vsw are slower than observed. The HAF slow bias increases with forecast duration. WSA had lower Vsw forecast-observation difference (F-O) absolute means and standard deviations than HAF. HAF and WSA Vsw forecast standard deviations were less than observed. Vsw F-O mean square skill rarely exceeds that of recurrence forecasts. Bpol is correctly predicted 65%–85% of the time in both models. Recurrence beats the models in Bpol skill in nearly every year forecast day category. Verification by “event” (flare events ≤5 days before forecast start) and “nonevent” (no flares) forecasts showed that most HAF Vsw bias growth, F-O standard deviation decrease, and forecast standard deviation decrease were due to the event forecasts. Analysis of single time step Vsw increases of ≥20% in the nonevent forecasts indicated that both models predicted too many occurrences and missed many observed incidences. Neither model had skill above a random guess in predicting Vsw increase arrival time at L1.

Forecast Uncertainty and the Bank of England Interest Rate Decisions

To assess the Bank of England Monetary Policy Committee decisions about the Official Bank Rate under forecast uncertainty, I estimate simple forecast-based interest rate rules augmented by the forecast standard deviations recovered directly from the Inflation Report fan charts. I find that interest rate decisions react to deviations of the medium-term forecasts for inflation from target in order to pursue the inflation target. Forecast inflation uncertainty has a strongly intensifying effect on this reaction. Information from output growth is utilized in the form of near-term forecasts. The associated forecast uncertainty of output growth has an attenuating effect on the interest rate reaction. When accounting for asymmetries in forecast uncertainty I find that forecast upward risks to inflation contribute to the intensifying effect of forecast inflation uncertainty. The corresponding downward risks have no significant impact. As regards output growth, asymmetries in the forecast uncertainty have no significant impact on the interest rate reaction at all. Moreover, I find that forecast risks to inflation have a direct effect on the interest rate decisions, in particular when inflation is forecast close to target.

Assimilation of stratospheric and mesospheric temperatures from MLS and SABER into a global NWP model

Abstract. The forecast model and three-dimensional variational data assimilation components of the Navy Operational Global Atmospheric Prediction System (NOGAPS) have each been extended into the upper stratosphere and mesosphere to form an Advanced Level Physics High Altitude (ALPHA) version of NOGAPS extending to ~100 km. This NOGAPS-ALPHA NWP prototype is used to assimilate stratospheric and mesospheric temperature data from the Microwave Limb Sounder (MLS) and the Sounding of the Atmosphere using Broadband Emission Radiometry (SABER) instruments. A 60-day analysis period in January and February 2006, was chosen that includes a well documented stratospheric sudden warming. SABER and MLS temperatures indicate that the SSW caused the polar winter stratopause at ~40 km to disappear, then reform at ~80 km altitude and slowly descend during February. The NOGAPS-ALPHA analysis reproduces this observed stratospheric and mesospheric temperature structure, as well as realistic evolution of zonal winds, residual velocities, and Eliassen-Palm fluxes that aid interpretation of the vertically deep circulation and eddy flux anomalies that developed in response to this wave-breaking event. The observation minus forecast (O-F) standard deviations for MLS and SABER are ~2 K in the mid-stratosphere and increase monotonically to about 6 K in the upper mesosphere. Increasing O-F standard deviations in the mesosphere are expected due to increasing instrument error and increasing geophysical variance at small spatial scales in the forecast model. In the mid/high latitude winter regions, 10-day forecast skill is improved throughout the upper stratosphere and mesosphere when the model is initialized using the high-altitude analysis based on assimilation of both SABER and MLS data.

Quantifying costs of forecast errors: A case study of the warehouse environment

Our study evaluates the impact of forecast errors on organizational cost by simulating a labor-intensive warehouse environment using realistic cost data from a case study. Unlike past studies that measure forecast error in terms of forecast standard deviation, our study also considers the impact of forecast bias, and the complex interaction between these variables. Two cases of organizational cost curves are considered, with differing and asymmetric structures. Results find forecast bias to have a considerably greater impact on organizational cost than forecast standard deviation. Particularly damaging is a high bias in the presence of high forecast standard deviation. Although biasing the forecast in the least costly direction is shown to yield lower costs, sensitivity analysis shows that increasing bias beyond the optimum point rapidly increases costs. ‘Overshooting’ the optimal amount of bias appears to be more damaging than not biasing the forecast at all. Given that managers often deliberately bias their forecasts, this finding underscores the importance of having a good understanding of organizational cost structures before arbitrarily introducing bias. This finding also suggests that managers should exercise caution when introducing bias, particularly for forecasts that inherently have large errors. These findings have important implications for organizational decision making beyond the simulated warehouse, as high forecast errors are endemic to many labor-intensive organizations.

USING WAREHOUSE WORKFORCE FLEXIBILITY TO OFFSET FORECAST ERRORS

The impact of forecast errors on organizational cost, and the usefulness of worker flexibility measures in offsetting their negative effects were evaluated in a labor intensive warehouse environment. Unlike past studies measuring forecast error in terms of forecast standard deviation, this study also considers the impact of forecast bias, which occurs when the mean of the error distribution is non‐zero. Results indicate that forecast bias is more damaging to warehouse cost, compared to forecast standard deviation. Workforce flexibility measures, such as worker cross‐training and proportion of full and part‐time labor, provide a solid buffer against forecast standard deviation. However, they are much less effective against forecast bias. This has important managerial implications as a large portion of forecast bias is managerially introduced.

Real‐time forecasting of photosmog episodes: the Naples case study

AbstractIn this paper we analysed the ozone time series data collected by the local monitoring network in the Naples urban area (southern Italy) during the spring/summer period of 1996. Our aim was to identify a reliable and effective model that could be used for the real‐time forecasting of photosmog episodes. We studied the applicability of seasonal autoregressive integrated moving average models with some exogenous variables (ARIMAX) to our case study. The choice of exogenous variables—temperature, [NO2]/[NO] ratio and wind speed—was based on physical reasoning. The forecasting performance of all models was evaluated with data not used in model development, by means of an array of statistical indices: the comparison between observed and forecast means and standard deviations; intercept and slope of a least squares regression of forecast variable on observed variable; mean absolute and root mean square errors; and 95% confidence limits of forecast variable. The assessment of all models was also based on their tendency to forecast critical episodes. It was found that the model using information from the temperature data set to predict peak ozone levels gives satisfactory results, about 70% of critical episodes being correctly predicted by the 24 h ahead forecast function. Copyright © 2001 John Wiley & Sons, Ltd.

Empirical measures of inflation uncertainty: a cautionary note

A direct, ex ante, measure of inflation uncertainty in the US is compared with a number of proxies used in empirical studies. The direct estimate is the root mean subjective variance of the probability distributions for inflation reported by respondents to the ASA/NBER survey. The proxies include forecast standard deviations from ARIMA, ARCH and structural models of inflation. These proxies are not significantly correlated with the direct measure, nor with one another. Use of the proxies leads to incorrect inferences about the correlation between inflation and inflation uncertainty, and between inflation uncertainty and the real interest rate.