Forecast Criteria Research Articles

Forecast Evaluation in the Presence of Unobserved Volatility#

A number of volatility forecasting studies have led to the perception that the ARCH- and Stochastic Volatility-type models provide poor out-of-sample forecasts of volatility. This is primarily based on the use of traditional forecast evaluation criteria concerning the accuracy and the unbiasedness of forecasts. In this paper we provide an analytical assessment of volatility forecasting performance. We use the volatility and log volatility framework to prove how the inherent noise in the approximation of the true- and unobservable-volatility by the squared return, results in a misleading forecast evaluation, inflating the observed mean squared forecast error and invalidating the Diebold–Mariano statistic. We analytically characterize this noise and explicitly quantify its effects assuming normal errors. We extend our results using more general error structures such as the Compound Normal and the Gram–Charlier classes of distributions. We argue that evaluation problems are likely to be exacerbated by non-normality of the shocks and that non-linear and utility-based criteria can be more suitable for the evaluation of volatility forecasts.

The relationship of extreme precipitation events to weather conditions in Nordland, Norway

We examined the distribution of extreme daily precipitation events (100 mm or more per day) in relation to weather conditions in the county of Nordland, Norway. Precipitation data included 88 weather stations and cover the period January 1964–March 2002. Over this period, extreme precipitation took two forms: frontal precipitation (associated with a warm or cold front) and warm sector precipitation (associated with the air masses between two fronts). Our analyses show that 57% of the extreme daily precipitation situations were classified as warm sector precipitation. This stands in contrast to claims that significant warm sector precipitation will very rarely, if ever, propagate as far north as Nordland. Because extreme precipitation may be linked to the risk of avalanches, earthflows and mudflows, it may be important to re-evaluate the forecasting criteria for heavy rain in north Norway.

Inferences On Predictability Of Foreign Exchange Rates Via Generalized Spectrum And Nonlinear Models

It is often documented, based on autocorrelation, variance ratio and power spectrum, that exchange rates approximately follow a martingale process. Because autocorrelation, variance ratio and spectrum check serial uncorrelatedness rather than martingale difference, they may deliver misleading conclusions in favor of the martingale hypothesis when the test statistics are insignificant. In this paper, we explore whether there exists a gap between serial uncorrelatedness and martingale difference for exchange rate changes, and if so, whether nonlinear time series models admissible in the gap can outperform the martingale model in out-of-sample forecasts. Applying the generalized spectral tests of Hong (1999) to five major exchange rates, we find that the changes of exchange rates are often serially uncorrelated, but there exists strong nonlinearity in conditional mean, in addition to the well-known volatility clustering. To forecast the conditional mean, we consider the linear autoregressive, autoregressive polynomial, artificial neural network and functional-coefficient models, as well as their combination. The functional coefficient model allows the autoregressive coefficients to depend on investment positions via an moving average technical trading rule. We evaluate out-of-sample forecasts of these models relative to the martingale model, using four criteria-the mean squared forecast error, the mean absolute forecast error, the mean forecast trading return, and the mean correct forecast direction. White's (2000) reality check method is used to avoid data-snooping bias. It is found that some nonlinear models, particularly their combination, do have superior predictive ability over the martingale model for some currencies in terms of certain forecast evaluation criteria.

A market-augmented model for SIMEX Brent crude oil futures contracts

Brent crude oil futures contracts are traded on both the Singapore International Monetary Exchange (SIMEX) and the International Petroleum Exchange (IPE). Through a mutual offset system between SIMEX and IPE, Brent crude oil futures contracts can be traded up to nineteen hours each day. The inter-relationship between the two futures contracts, the spot price of Brent crude oil and the riskfree interest rate, suggest the existence of cointegration among SIMEX Brent crude oil futures prices, lagged IPE Brent crude oil futures prices, Brent spot prices and the London Inter-bank Offer Rate (LIBOR). Error-correction representations of two standard futures pricing models, namely the unbiased expectations and cost-of-carry hypotheses, are formulated for SIMEX Brent crude oil futures contracts. These formulations are augmented by including the lagged IPE futures price in the mispricing error. The resulting Augmented Unbiased Expectations Hypothesis (AUEH) and the Augmented Cost-of-Carry (ACOC) models are estimated and tested against each other, and also against the standard unbiased expectations and cost-of-carry models, using nested and non-nested testing procedures. Forecasting comparisons are also made among the various models and the autoregressive integrated moving average models fitted to SIMEX Brent crude oil futures prices. Results from the nonnested tests and the forecasting criteria show clearly that the augmented models outperform their standard (non-augmented) counterparts.

Specifying Nonlinear Econometric Models by Flexible Regression Models and Relative Forecast Performance

The paper considers the task of selecting a flexible nonlinear model which can be used as a baseline model. The baseline model may be used as a testing ground for more structural models which are congruent with economic theory. From the limited empirical evidence obtained here it is tentatively suggested to find a baseline nonlinear flexible form for a univariate time series by following the procedure: 1. Recursively, based on h extra periods at a time specify and estimate a linear form by use of model selection criteria like Cross Validation and/or BIC. 2. After a preliminary test for linearity, recursively, specify and estimate flexible regression models like the FNL suggested by Hamilton (1999) and the Projection Pursuit model suggested by Aldrin, Boelviken and Schweder (1993) for cases of moderate nonlinearities. Use the Cross Validation and the BIC criteria. 3. Based on the remaining part of the data set select the best nonlinear flexible form by use of forecast criteria measuring the absolute forecast performance and the directional forecast performance in h-steps ahead predictions, and compare the best flexible form to the linear specification by use of the Diebold Mariano tests, see Deibold and Mariano (1995) and the forecast encompassing tests suggested by Harvey, Lebourne, and Newhold (1998). The results indicate that the FNL method and the Projection Pursuit Model are the preferable models to apply and that the CV and BIC are the best selection criteria, while the forecast encompassing tests properly modified as suggested by Harvey et. al. (1998) possess better power properties than the Diebold- Mariano test.

Evaluation of onion downy mildew disease risk in New Zealand using meteorological forecasting criteria

Meteorological prediction criteria for onion downy mildew (Peronospora destructor), adapted from the disease forecasting system “Downcast”, were used to examine patterns of disease risk in two climatically contrasted New Zealand onion growing regions. Potential sporulation and infection events were predicted to occur in both regions in all months between September and April, although South Auckland had more events (27-43% of days per month) than Canterbury (10-18% of days per month). Predicted disease risk was most often limited in Canterbury when relative humidity failed to reach the level required for sporulation. In South Auckland, risk was most often limited by insufficient leaf wetness duration for infection, but in one low rainfall year, relative humidity was the most often limiting factor, as in Canterbury. It appears that in Canterbury, and in some seasons in South Auckland, forecasting criteria could be used to modify fungicide timing

A nonparametric investigation of the 90‐day t‐bill rate

AbstractWe employ a nonlinear, nonparametric method to model the stochastic behavior of changes in the 90‐day U.S. T‐bill rate. The estimation technique is locally weighted regression (LWR), a nearest‐neighbor method, and the forecasting criteria are the root mean square error (RMSE) and mean absolute deviation (MAD) measures. We compare the forecasting performance of the nonparametric fit to the performance of two benchmark linear models: an autoregressive model and a random‐walk‐with‐drift model. The nonparametric fit results in significant improvements in forecasting accuracy as compared to benchmark linear models both in‐sample and out‐of‐sample, thus establishing the presence of substantial nonlinear mean predictability in the 90‐day T‐bill rate.

An assessment of the economic value of non‐linear foreign exchange rate forecasts

AbstractRecent research suggests that non‐linear methods cannot improve the point forecasts of high‐frequency exchange rates. These studies have been using standard forecasting criteria such as smallest mean squared error (MSE) and smallest mean absolute error (MAE). It is, however, premature to conclude from this evidence that non‐linear forecasts of high‐frequency financial returns are economically or statistically insignificant. We prove a proposition which implies that the standard forecasting criteria are not necessarily particularly suited for assessment of the economic value of predictions of non‐linear processes where the predicted value and the prediction error may not be independently distributed. Adopting a simple non‐linear forecasting procedure to 15 daily exchange rate series we find that although, when compared to simple random walk forecasts, all the non‐linear forecasts give a higher MSE and MAE, when applied in a simple trading strategy these forecasts result in a higher mean return. It is also shown that the ranking of portfolio payoffs based on forecasts from a random walk, and linear and non‐linear models, is closely related to a non‐parametric test of market timing.

An econometric model of the monetary sector in Indonesia

The purpose of this article is to develop an econometric model of the Indonesian monetary sector over the period 1969–80. Before we constructed such a model, we tested the hypothesis that there is a causal relationship between money supply and the aggregate price index using quarterly data. The hypothesis of a causal relationship from prices to money was supported while the hypothesis of contemporaneous causality could not be easily dismissed. In the model, both money supply and the aggregate price index were used as endogenous variables and they were decomposed into several disaggregates. The possibility of a structural break after the oil price shock (post‐1973) was examined using forecasting criteria. The predictive performance for 1969–73 is better than for 1974–80.

Estimating Time Series Models Using the Relevant Forecast Evaluation Criterion

SUMMARY The usual procedure in time series analysis is for the parameters to be estimated using ordinary least squares. However, the. forecasts are often evaluated using a totally different criterion, mean absolute percentage error, for example. Here, via the examination of a large number of data series, we examine the effects of using the forecast evaluation criterion as the estimation criterion. In a recent paper, Makridakis et al. (1982) presented the results of a comprehensive empirical study comparing the forecasting ability of various time series techniques. The methods used in that experiment were of two distinct types. Most were techniques which may be considered automatic, in the sense that the data are put into the computer, a program executed and the forecasts produced. The only really personalzed technique, requiring considerable input from the analyst at various stages during the model selection and verification process, was the Box and Jenkins (1970) method. In fact, the analysis of the 111 series for the Makridakis experiment took the present authors over 3 months of part-time effort, with some of the more difficult series requiring up to five or six iterations of the well-known cycle of identification, estimation and diagnostic checking. Overall, it would not be too much of an understatement to describe our reaction to the published results as disappointment, especially given the positive statistics of Newbold and Granger (1974) and others in previous studies. For example, the mean absolute percentage error (MAPE) averaged across the 111 series for the Box-Jenkins method was 18.0, while the Parzen (1982) method and simple exponential smoothing on the deseasonalized data resulted in 15.4 and 16.8 respectively; both substantial reductions. After all, one would expect gains in accuracy from the more personalized and time-consuming Box-Jenkins procedure. Newbold and Granger (1974), however, considered only three techniques, Box-Jenkins, Holt- Winters and stepwise autoregression, and the relationship between the forecast performance of the first two of these is essentially the same there as in Makridakis et al. (1982). Further, Andersen (1982) has re-examined some of the series to determine reasons for the disparity between the reported and anticipated results. He found that if one considers only series of reasonable length and uses the (perhaps artificial) forecast criterion of geometric mean of squared one step ahead forecast errors, then the comparison of forecast statistics from the Box-Jenkins models in Makridakis et al. (1982) and long autoregressive models is also essentially the same as that from the Box-Jenkins and stepwise autoregressive models in Newbold and Granger (1974). In this paper we consider a different approach. The usual practice in time series analysis, used in both the experiment and Andersen (1982), is for ARIMA models to be estimated by least squares. Theoretically, minimization of the residual sum of squares is asymptotically optimal

Price Risk Reduction in Marketing Corn from Using Hedging Strategies

The use of hedging with commodity futures markets to reduce the price risk in corn production is examined. Both intra-year and inter-year risk are evaluated with different hedging strategies. Strategies involve no hedge, hedge and hold, controlled hedge placement and hold, and in and out hedging. Both technical and forecasting criteria are used to place hedges in the more active strategies. Substantial risk reduction is possible, often without a reduction in price received. Considerable basis risk diminishes the risk reducing properties of a hedge and hold strategy.