Relative Forecast Performance Research Articles

New Findings Regarding the Out-of-Sample Predictive Impact of the Price of Crude Oil on the United States Industrial Production

Contrary to the extensive literature pioneered by James Hamilton in the early 1980s that focuses on analyzing the relationship between changes in the price of crude oil and the U.S. real gross domestic product growth (GDP) rate, Herrera et al. (2011) is essentially the first study that explores the in-sample predictive impact of the price of crude oil on the U.S. industrial production index. To date, almost nothing is known about the nature and degree of the out-of-sample predictive impact of the price of crude oil on the U.S. industrial production index. This study fills the gap. Using various nonlinear transformations of the price crude oil widely employed in the crude oil price/GDP predictability literature as well as crude oil price volatility measures, we document (rather surprisingly) that the form of nonlinearity that delivers the most consistent pattern of out-of-sample population-level predictability gains relative to the benchmark when forecasting ex-post revised as well as real-time U.S. industrial production has to do with crude oil price decreases below the minimum price in recent memory. In contrast to the GDP predictability literature, crude oil price increases beyond the maximum in recent memory do not afford any predictive power. On the contrary, they deteriorate relative forecast performance. These results go directly against a distinct sense of déjà vu that one would expect given the degree of affinity between industrial production and GDP. The predictive power afforded by crude oil price net decreases also translate into economic gains.

Evaluating UK Point and Density Forecasts from an Estimated DSGE Model: The Role of Off-Model Information Over the Financial Crisis

This paper investigates the real-time forecast performance of the Bank of England’s main DSGE model, COMPASS, before, during and after the financial crisis with reference to statistical and judgemental benchmarks. A general finding is that COMPASS’s relative forecast performance improves as the forecast horizon is extended (as does that of the Statistical Suite of forecasting models). The performance of forecasts from all three sources deteriorates substantially following the financial crisis.The deterioration is particularly marked for the DSGE model’s GDP forecasts. One possible explanation for that, and a key difference between DSGE models and judgemental forecasts, is that judgemental forecasts are implicitly conditioned on a broader information set, including faster-moving indicators that may be particularly informative when the state of the economy is evolving rapidly, as in periods of financial distress. Consistent with that interpretation, GDP forecasts from a version of the DSGE model augmented to include a survey measure of short-term GDP growth expectations are competitive with the judgemental forecasts at all horizons in the post-crisis period. More generally, a key theme of the paper is that both the type of off-model information and the method used to apply it are key determinants of DSGE model forecast accuracy.

Split-then-Combine Method for out-of-sample Combinations of Forecasts

Relative forecast performance of forecast units may periodically evolve over time. Therefore, it is desirable to take into account their forecast periodicity when forming forecast combinations. When dealing with small samples and small number of models, using panels is an efficient way of pulling out the additional information provided by that periodicity in the data. We capture this periodic information with different weights at different periods that we then keep in the out-of-sample combination. As in the simple average, we do not estimate weights, but instead compute them from panels of forecasts taken as given data. Empirical and bootstrap exercises illustrate the superiority of this method over fixed weight schemes.

Multiple-output support vector regression with a firefly algorithm for interval-valued stock price index forecasting

Highly accurate interval forecasting of a stock price index is fundamental to successfully making a profit when making investment decisions, by providing a range of values rather than a point estimate. In this study, we investigate the possibility of forecasting an interval-valued stock price index series over short and long horizons using multi-output support vector regression (MSVR). Furthermore, this study proposes a firefly algorithm (FA)-based approach, built on the established MSVR, for determining the parameters of MSVR (abbreviated as FA-MSVR). Three globally traded broad market indices are used to compare the performance of the proposed FA-MSVR method with selected counterparts. The quantitative and comprehensive assessments are performed on the basis of statistical criteria, economic criteria, and computational cost. In terms of statistical criteria, we compare the out-of-sample forecasting using goodness-of-forecast measures and testing approaches. In terms of economic criteria, we assess the relative forecast performance with a simple trading strategy. The results obtained in this study indicate that the proposed FA-MSVR method is a promising alternative for forecasting interval-valued financial time series.

Predictive Ability of Business Cycle Indicators under Test

Summary In this paper we assess the information content of seven widely cited early indicators for the euro area with respect to forecasting area-wide industrial production. To this end, we use various tests that are designed to compare competing forecast models. In addition to the standard Diebold-Mariano test, we employ tests that account for specific problems typically encountered in forecast exercises. Specifically, we pay attention to nested model structures, we alleviate the problem of data snooping arising from multiple pairwise testing, and we analyze the structural stability in the relative forecast performance of one indicator compared to a benchmark model. Moreover, we consider loss functions that overweight forecast errors in booms and recessions to check-whether a specific indicator that appears to be a good choice on average is also preferable in times of economic stress. We find that none of this indicators uniformly dominates all its competitors. The optimal choice rather depends on the specific forecast situation and the loss function of the user. For 1-month forecasts the business climate indicator of the European Commission and the OECD composite leading indicator generally work well, for 6-month forecasts the OECD composite leading indicator performs very good by all criteria, and for 12-month forecasts the FAZ-Euro indicator published by the Frankfurter Allgemeine Zeitung is the only one that can beat the benchmark AR(1) model.

Predictive ability of business cycle indicators under test: A case study for the Euro area industrial production

In this paper we assess the information content of seven widely cited early indicators for the euro area with respect to forecasting area-wide industrial production. To this end, we use various tests that are designed to compare competing forecast models. In addition to the standard Diebold-Mariano test, we employ tests that account for specific problems typically encountered in forecast exercises. Specifically, we pay attention to nested model structures, we alleviate the problem of data snooping arising from multiple pairwise testing, and we analyze the structural stability in the relative forecast performance of one indicator compared to a benchmark model. Moreover, we consider loss functions that overweight forecast errors in booms and recessions to check whether a specific indicator that appears to be a good choice on average is also preferable in times of economic stress. We find that on average three indicators have superior forecast ability, namely the EuroCoin indicator, the OECD composite leading indicator, and the FAZ-Euro indicator published by the Frankfurter Allgemeine Zeitung. If one is interested in one-month forecasts only, the business climate indicator of the European Commission yields the smallest errors. However, the results are not completely invariant against the choice of the loss function. Moreover, rolling local tests reveal that the indicators are particularly useful in times of unusual changes in industrial production while the simple autoregressive benchmark is difficult to beat during time of average production growth.

Predictive Ability of Business Cycle Indicators Under Test: A Case Study for the Euro Area Industrial Production

In this paper we assess the information content of seven widely cited early indicators for the euro area with respect to forecasting area-wide industrial production. To this end, we use various tests that are designed to compare competing forecast models. In addition to the standard Diebold-Mariano test, we employ tests that account for specific problems typically encountered in forecast exercises. Specifically, we pay attention to nested model structures, we alleviate the problem of data snooping arising from multiple pairwise testing, and we analyze the structural stability in the relative forecast performance of one indicator compared to a benchmark model. Moreover, we consider loss functions that overweight forecast errors in booms and recessions to check whether a specific indicator that appears to be a good choice on average is also preferable in times of economic stress. We find that there is not one best indicator that uniformly dominates all its competitors. The optimal choice rather depends on the specific forecast situation and the loss function of the user. For 1-month forecasts the business climate indicator of the European Commission and the OECD composite leading indicator generally work well, for 6-month forecasts the OECD composite leading indicator performs very good by all criteria, and for 12-month forecasts the FAZ-Euro indicator published by the Frankfurter Allgemeine Zeitung is the only one that can beat the benchmark AR(1) model.

Self-Fulfilling Inflation Inertia under Model Uncertainty

It is well known that the canonical New Keynesian model cannot produce inflation inertia observed in real data. The purpose of this research is to investigate expectation formation process as a potential source of inflation inertia. We examine a purely forward-looking New Keynesian Phillips curve with heterogeneous expectations, where the coordination of agents' beliefs matters. Private agents with heterogeneous beliefs are assumed to choose from a list of potentially misspecified econometric models and base their selections on relative forecast performance. When the distribution of heterogeneity is allowed to vary, there exist multiple equilibria: rational expectations equilibrium without any inertia and misspecification equilibrium with inertia. This inertia does not come from any structural components, such as a backward-looking behavior of price-setters, but from the expectation formation process of the private agents. In this sense, the inflation inertia in our model can be viewed as a self-fulfilling prophecy of the private agents. We then combine model selection with the adaptive learning, through which the agents update the parameters of their models. Our simulations show that convergences toward both equilibria are possible and the quality of initial knowledge has long-standing effect on the economy.

Model uncertainty and endogenous volatility

This paper identifies two channels through which the economy can generate endogenous inflation and output volatility, an empirical regularity, by introducing model uncertainty into a Lucas-type monetary model. The equilibrium path of inflation depends on agents' expectations and a vector of exogenous random variables. Following Branch and Evans agents are assumed to underparameterize their forecasting models [Branch, W., Evans, G.W., 2006a. Intrinsic heterogeneity in expectation formation. Journal of Economic Theory 127, 264–295]. A Misspecification Equilibrium arises when beliefs are optimal, given the misspecification, and predictor proportions are based on relative forecast performance. We show that there may exist multiple Misspecification Equilibria, a subset of which is stable under least squares learning and dynamic predictor selection. The dual channels of least squares parameter updating and dynamic predictor selection combine to generate regime switching and endogenous volatility.

Tests of Conditional Predictive Ability

We propose a framework for out-of-sample predictive ability testing and forecast selection designed for use in the realistic situation in which the forecasting model is possibly misspecified, due to unmodeled dynamics, unmodeled heterogeneity, incorrect functional form, or any combination of these. Relative to the existing literature (Diebold and Mariano (1995) and West (1996)), we introduce two main innovations: (i) We derive our tests in an environment where the finite sample properties of the estimators on which the forecasts may depend are preserved asymptotically. (ii) We accommodate conditional evaluation objectives (can we predict which forecast will be more accurate at a future date?), which nest unconditional objectives (which forecast was more accurate on average?), that have been the sole focus of previous literature. As a result of (i), our tests have several advantages: they capture the effect of estimation uncertainty on relative forecast performance, they can handle forecasts based on both nested and nonnested models, they allow the forecasts to be produced by general estimation methods, and they are easy to compute. Although both unconditional and conditional approaches are informative, conditioning can help fine-tune the forecast selection to current economic conditions. To this end, we propose a two-step decision rule that uses current information to select the best forecast for the future date of interest. We illustrate the usefulness of our approach by comparing forecasts from leading parameter-reduction methods for macroeconomic forecasting using a large number of predictors.

The assimilation of AIRS radiance data at ECMWF

AbstractThe development of an assimilation system for radiance data from the Atmospheric InfraRed Sounder (AIRS) is described, in particular the identification of cloud contamination, bias correction and the characterization of errors in the measured radiances and radiative‐transfer model. The results of assimilation experiments are presented. These show that a conservative use of AIRS radiance data (in a system already extensively observed with other satellite data) results in a small, but consistent, improvement in the quality of analyses and forecasts. Larger impacts of AIRS are found in hypothetical experiments that test the use of radiances from only a single sounding instrument. In these, the use of AIRS is found to outperform the use of data either from a single Advanced Microwave Sounding Unit‐A (AMSU‐A) or from a single High‐resolution InfraRed Sounder (HIRS). In this hypothetical context the relative forecast performance of each sensor is found to correlate with the size and vertical scale of increments caused by the assimilation of the radiances. Copyright © 2006 Royal Meteorological Society.

Intrinsic heterogeneity in expectation formation

We introduce the concept of Misspecification Equilibrium to dynamic macroeconomics. Agents choose between a list of misspecified econometric models and base their selection on relative forecast performance. A Misspecification Equilibrium is a stochastic process in which agents forecast optimally given their choices, with forecast model parameters and predictor proportions endogenously determined. Under appropriate conditions, the Misspecification Equilibrium will exhibit Intrinsic Heterogeneity, in which all predictors are used at all times, even in the neoclassical limit in which only the most successful predictors are used. This equilibrium is attainable under least-squares learning and dynamic predictor selection based on average profits.

STAR and ANN models: forecasting performance on the Spanish “Ibex-35” stock index

This paper studies whether it is possible to exploit the nonlinear behaviour of daily returns on the Spanish Ibex-35 stock index returns to improve forecasts over short and long horizons. In this sense, we examine the out-of-sample forecast performance of smooth transition autoregression (STAR) models and artificial neural networks (ANNs). We use one-step (obtained by using recursive and nonrecursive regressions) and multi-step-ahead forecasting methods. The forecasts are evaluated with statistical and economic criteria. In terms of statistical criteria, we compared the out-of-sample forecasts using goodness of forecast measures and various testing approaches. The results indicate that ANNs consistently surpass the random walk model and, although the evidence for this is weaker, provide better forecasts than the linear AR model and the STAR models for some forecast horizons and forecasting methods. In terms of the economic criteria, we assess the relative forecast performance in a simple trading strategy including the impact of transaction costs on trading strategy profits. The results indicate a better fit for ANN models, in terms of the mean net return and Sharpe risk-adjusted ratio, by using one-step-ahead forecasts. These results show there is a good chance of obtaining a more accurate fit and forecast of the daily stock index returns by using one-step-ahead predictors and nonlinear models, but that these are inherently complex and present a difficult economic interpretation.

Flexible regression models and relative forecast performance

In this paper, four alternative flexible nonlinear regression model approaches are reviewed and their performance evaluated based on various measures of out-of-sample forecast accuracy. The class of flexible regression model considered includes Neural Networks, Projection Pursuit models and the Random Field regression model approach recently suggested by Hamilton [Econometrica 69 (2001) 537–573]. An empirical illustration is provided, showing that linear models for the US unemployment rate and the growth rate in US industrial production cannot outperform the “best” flexible nonlinear regression models in terms of out-of-sample forecast accuracy. The results indicate a possible presence of a nonlinear component in the conditional mean function of both time series.

A Bayesian forecasting approach to constructing regional input output based employment multipliers

Abstract A Bayesian mixed estimation framework is used to examine the forecast accuracy of alternative closures of an input‐output model for the Oklahoma economy. The closures correspond to textbook Type I and Type II multipliers, as well as variations of extended input‐output and Type IV multipliers. Relative forecast performance of the alternative IO model closures determines which set of multipliers should be used for impact analysis. The exercise reveals differences in forecast accuracy across alternative IO model closures, suggesting that before closures of a particular IO model are adopted, they should be tested for accuracy in predicting the time series data for the regional economy under scrutiny.

An Evaluation of Alternative Strategies for Incorporating Interindustry Relationships into a Regional Employment Forecasting Model

Alternative strategies for incorporating interindustry relationships into industry employment forecasting equations are evaluated. Strategies that are based on using input-output information to select which industry employment variables to include as independent variables in each equation are com-pared to those of including input-output-based aggregate demand variables. The strategies also differ according to the degree of endogeneity between industry employment levels, whether model selection procedures are used, and whether restrictions are placed upon the coefficients during estimation. Relative forecast performance of the alternative models and an examination of the employment multipliers serve as criteria in the evaluation of the models.

A comparison between linear and nonlinear forecasts for nonlinear AR models

In this paper the relative forecast performance of nonlinear models to linear models is assessed by the conditional probability that the absolute forecast error of the nonlinear forecast is smaller than that of the linear forecast. The comparison probability is explicitly expressed and is shown to be an increasing function of the distance between nonlinear and linear forecasts under certain conditions. This expression of the comparison probability may not only be useful in determining the predictor, which is either a more accurate or a simpler forecast, to be used but also provides a good explanation for an odd phenomenon discussed by Pemberton. The relative forecast performance of a nonlinear model to a linear model is demonstrated to be sensitive to its forecast origins. A new forecast is thus proposed to improve the relative forecast performance of nonlinear models based on forecast origins. #1997 John Wiley & Sons, Ltd.

Nonlinear modelling and prediction with feedforward and recurrent networks

In feedforward networks, signals flow in only one direction without feedback. Applications in forecasting, signal processing and control require explicit treatment of dynamics. Feedforward networks can accommodate dynamics by including past input and target values in an augmented set of inputs. A much richer dynamic representation results from also allowing for internal network feedbacks. These types of network models are called recurrent network models and are used by Jordan (1986) for controlling and learning smooth robot movements, and by Elman (1990) for learning and representing temporal structure in linguistics. In Jordan's network, past values of network output feed back into hidden units; in Elman's network, past values of hidden units feed back into themselves. The main focus of this study is to investigate the relative forecast performance of the Elman type recurrent network models in comparison to feedforward networks with deterministic and noisy data. The salient property of the Elman type recurrent network architecture is that the hidden unit activation functions (internal states) are fed back at every time step to provide an additional input. This recurrence gives the network dynamical properties which make it possible for the network to have internal memory. Exactly how this memory is represented in the internal states is not determined in advance. Instead, the network must discover the underlying temporal structure of the task and learn to encode that structure internally. The simulation results of this paper indicate that recurrent networks filter noise more successfully than feedforward networks in small as well as large samples.