Accuracy Of Point Forecasts Research Articles

Forecasting Performance of an Open Economy DSGE Model

This paper analyzes the forecasting performance of an open economy dynamic stochastic general equilibrium (DSGE) model, estimated with Bayesian methods, for the Euro area during 1994Q1–2002Q4. We compare the DSGE model and a few variants of this model to various reduced-form forecasting models such as vector autoregressions (VARs) and vector error correction models (VECM), estimated both by maximum likelihood and two different Bayesian approaches, and traditional benchmark models, e.g., the random walk. The accuracy of point forecasts, interval forecasts and the predictive distribution as a whole are assessed in an out-of-sample rolling event evaluation using several univariate and multivariate measures. The results show that the open economy DSGE model compares well with more empirical models and thus that the tension between rigor and fit in older generations of DSGE models is no longer present. We also critically examine the role of Bayesian model probabilities and other frequently used low-dimensional summaries, e.g., the log determinant statistic, as measures of overall forecasting performance.

Data Collection Time Intervals for Stochastic Short-Term Traffic Flow Forecasting

The specification of time intervals for data collection is a fundamental determinant of the nature and utility of the resulting traffic condition data streams. In the context of short-term traffic flow forecasting, the establishment of the data collection time interval should play a key role in determining the corresponding appropriate forecasting approach. The data collection time interval provides the forecasting horizon for one-step-ahead forecasting. Nevertheless, the need for more rigorous understanding of the effects of data collection time interval specification within the context of short-term traffic flow forecasting is not well recognized. By contrast, it has been common practice in previous research to select the data collection time interval and forecasting approach without explicit consideration of time interval effects or systematic evaluation of available forecasting methods. A stochastic seasonal autoregressive integrated moving average plus generalized autoregressive conditional heteroscedasticity (SARIMA+GARCH) structure proposed in previous work holds promise in providing accurate point forecasts and reasonable forecasting confidence intervals. In this paper, a spectrum of data collection time intervals is tested with an online forecasting algorithm developed based on the SARIMA+GARCH structure to determine the applicable data collection time intervals for this structure. In this test, both the forecast accuracy and the validity of the forecasting confidence intervals are investigated. This work serves as an important step toward establishing a short-term traffic condition forecasting framework that identifies appropriate forecasting approaches for candidate data collection time intervals based on the functional requirements of specific applications.

The out-of-sample forecasting performance of nonlinear models of real exchange rate behavior

We analyze the out-of-sample forecasting performance of nonlinear models of U.S. dollar real exchange rate behavior from the extant empirical literature. Our analysis entails a comparison of point, interval, and density forecasts generated by nonlinear and linear autoregressive models. Using monthly data from the post-Bretton Woods period, there is little evidence to recommend either band-threshold or exponential smooth transition autoregressive models over simple linear autoregressive models in terms of out-of-sample forecasting performance at short horizons. Nonlinear models appear to offer more accurate point forecasts at long horizons for some countries. Overall, our results suggest that any nonlinearities in monthly real exchange rate data from the post-Bretton Woods period are quite “subtle” for band-threshold and exponential smooth transition autoregressive model specifications. Further evidence of this is provided by in-sample comparisons of the conditional densities implied by nonlinear and linear autoregressive models.

Empirical exchange rate models and currency risk: some evidence from density forecasts

A large literature in exchange rate economics has investigated the forecasting performance of empirical exchange rate models using conventional point forecast accuracy criteria. However, in the context of managing exchange rate risk, interest centers on more than just point forecasts. This paper provides a formal evaluation of recent exchange rate models based on the term structure of forward exchange rates, which previous research has shown to be satisfactory in point forecasting, in terms of density forecasting performance. The economic value of the exchange rate density forecasts is investigated in the context of an application to a simple risk management exercise.

The forecasting performance of various models for seasonality and nonlinearity for quarterly industrial production

Seasonality often accounts for the major part of short-run movements in quarterly or monthly macro economic time series. In addition, business cycle nonlinearity is a prominent feature of many such series. A forecaster can nowadays consider a wide variety of time series models that describe seasonal variation and nonlinear regime-switching behavior. In this paper we examine the forecasting performance of various models for seasonality and nonlinearity for quarterly industrial production series of 18 OECD countries. We find that the accuracy of point forecasts varies widely across series, across forecast horizons and across seasons. However, in general, linear models with fairly simple descriptions of seasonality outperform nonlinear at short forecast horizons, whereas nonlinear models with more elaborate seasonal components dominate at longer horizons. Finally, none of the models is found to render efficient forecasts and hence, forecast combination is worthwhile.

Evaluation of the 29-km Eta Model. Part I: Objective Verification at Three Selected Stations

This paper describes an objective verification of the National Centers for Environmental Prediction 29-km Eta Model from May 1996 through January 1998. The evaluation was designed to assess the model’s surface and upper-air point forecast accuracy at three selected locations during separate warm (May–August) and cool (October–January) season periods. In order to enhance sample sizes available for statistical calculations, the objective verification includes two consecutive warm and cool season periods. The statistical evaluation identified model biases that result from inadequate parameterization of physical processes. However, since the model biases are relatively small compared to the random error component, most of the total model error results from day-to-day variability in the forecasts and/or observations. To some extent, these nonsystematic errors reflect the variability in point observations that sample spatial and temporal scales of atmospheric phenomena that cannot be resolved by the model. On average, Meso Eta point forecasts provide useful guidance for predicting the evolution of the larger-scale environment. A more substantial challenge facing model users in real time is the discrimination of nonsystematic errors that tend to inflate the total forecast error. It is important that users maintain awareness of ongoing model updates because they modify the basic error characteristics, particularly near the surface. While some of the changes in error were expected, others were not consistent with the intent of the model updates and further emphasize the need for ongoing sensitivity studies and localized statistical verification efforts. Objective verification of point forecasts is a stringent measure of model performance, but when used alone, is not enough to quantify the overall value that model guidance may add to the forecast process. Therefore, results from a subjective verification of the Meso Eta Model over the Florida peninsula are discussed in the companion paper by Manobianco and Nutter.

Heuristics, biases and improvement strategies in judgmental time series forecasting

There is evidence that forecasts produced in business and other organizations often involve substantial elements of human judgment. In forming their judgments forecasters may have access to either time series or time series and contextual information. This paper reviews the literature to ascertain, for each information level, what we currently know about the heuristics people use when producing judgmental point forecasts and the biases which emanate from the use of these heuristics. The paper then uses evidence from the literature to review the effectiveness of a number of strategies which are designed to improve the accuracy of judgmental point forecasts.

THE PREDICTIVE PERFORMANCE OF THREE AUTOREGRESSIVE MOVING‐AVERAGE MODELS:A MONTE CARLO INVESTIGATION

Abstract. The relative accuracy of point and interval forecasts from three related autoregressive moving‐average (ARMA) models—multivariate, univariate, and transfer function—is evaluated in this study. It is found that the multivariate models produce the most accurate one‐ and three‐step‐ahead point forecasts of nonindependent series. However, the most accurate point forecasts of independent series are generated by the univariate models. Compared with the multivariate models, the transfer function predictions are relatively unreliable, but with the appropriate restrictions they are superior to the univariate forecasts in certain cases. Interval forecasts from the correctly specified models are reliable indicators of forecast dispersion.