Forecast Error Criteria Research Articles

Modeling and Forecasting of Rainfall Time Series. A Case Study for Pakistan Tayyab Raza Fraz

The change of weather conditions is considered as the major problem, particularly for developing country like Pakistan. Machine learning and artificial neural network models have become attractive forecast techniques for rainfall as compared to traditional statistical methods in the last few years. The behavioral pattern in rainfall (mm) annually by 1901 to 2020 is studied. Moreover, forecasts of three models based on past observations are evaluated. Fundamentally, different techniques are used for model development. Three modeling techniques include a traditional linear time series ARMA model, an emerging nonlinear threshold technique SETAR model, and influential machine learning technique NAR model. Evaluation of forecast performance is based on three forecast error criteria namely MSE, RMSE, and MAPE. Results indicate that the rainfall (mm) will slightly increase in the coming ten years i.e. 2021 to 2030. Furthermore, the findings also reveal that the NAR model is a suitable and appropriate model to forecast the rainfall which outperforms the ARMA as well as the SETAR model.

Variable targeting and reduction in large vector autoregressions with applications to workforce indicators

We develop statistical tools for time series analysis of large multivariate datasets, when a few core series are of principal interest and there are many potential auxiliary predictive variables. The methodology, based on Vector Autoregressions (VAR), handles the case where unrestricted fitting is precluded by a large number of series and a huge parameter space. In particular, we adopt a forecast error criterion and use Granger-causality tests in a sequential manner to build a VAR model that targets the main variables. This approach affects variable reduction (or equivalently, sparsity restrictions) in a computationally fast way that remains feasible for large dimensions. The search for the best model results in a VAR, fitted with a selection of supporting series, that has the best possible forecast performance with respect to the core variables. We apply the statistical methodology to model real Gross Domestic Product and the national Unemployment Rate, two time series widely monitored by economists and policy-makers, based on a large set of Quarterly Workforce Indicators comprising various major sectors of the economy and different measures of labor market conditions.

When are Direct Multi‐step and Iterative Forecasts Identical?

Although both direct multi‐step‐ahead forecasting and iterated one‐step‐ahead forecasting are two popular methods for predicting future values of a time series, it is not clear that the direct method is superior in practice, even though from a theoretical perspective it has lower mean squared error (MSE). A given model can be fitted according to either a multi‐step or a one‐step forecast error criterion, and we show here that discrepancies in performance between direct and iterative forecasting arise chiefly from the method of fitting, and is dictated by the nuances of the model's misspecification. We derive new formulas for quantifying iterative forecast MSE, and present a new approach for assessing asymptotic forecast MSE. Finally, the direct and iterative methods are compared on a retail series, which illustrates the strengths and weaknesses of each approach. Copyright © 2015 John Wiley & Sons, Ltd.

Multi-step-ahead estimation of time series models

Abstract We study the fitting of time series models via the minimization of a multi-step-ahead forecast error criterion that is based on the asymptotic average of squared forecast errors. Our objective function uses frequency domain concepts, but is formulated in the time domain, and allows the estimation of all linear processes (e.g., ARIMA and component ARIMA). By using an asymptotic form of the forecast mean squared error, we obtain a well-defined nonlinear function of the parameters that is proven to be minimized at the true parameter vector when the model is correctly specified. We derive the statistical properties of the parameter estimates, and study the asymptotic impact of model misspecification on multi-step-ahead forecasting. The method is illustrated through a forecasting exercise, applied to several time series.

The Dynamics of Economic Functions: Modeling and Forecasting the Yield Curve

The class of functional signal plus noise (FSN) models is introduced that provides a new, general method for modeling and forecasting time series of economic functions. The underlying, continuous economic function (or “signal”) is a natural cubic spline whose dynamic evolution is driven by a cointegrated vector autoregression for the ordinates (or “y-values”) at the knots of the spline. The natural cubic spline provides flexible cross-sectional fit and results in a linear state-space model. This FSN model achieves dimension reduction, provides a coherent description of the observed yield curve and its dynamics as the cross-sectional dimension N becomes large, and can be feasibly estimated and used for forecasting when N is large. The integration and cointegration properties of the model are derived. The FSN models are then applied to forecasting 36-dimensional yield curves for U.S. Treasury bonds at the 1-month-ahead horizon. The method consistently outperforms the dynamic Nelson–Siegel and random walk forecasts on the basis of both mean squared forecast error criteria and economically relevant loss functions derived from the realized profits of pairs trading algorithms. The analysis also highlights in a concrete setting the dangers of attempting to infer the relative economic value of model forecasts on the basis of their associated mean squared forecast errors.

Initial Expectations in New Keynesian Models with Learning

This paper examines how the estimation results for a standard New Keynesian model with constant gain least squares learning is sensitive to the stance taken on agents beliefs at the beginning of the sample. The New Keynesian model is estimated under rational expectations and under learning with three different frameworks for how expectations are set at the beginning of the sample. The results show that initial beliefs can have an impact on the predictions of an estimated model; in fact previous literature has exposed this sensitivity to explain the changing volatilities of output and inflation in the post-war United States. The results indicate statistical evidence for adaptive learning, however the rational expectations framework performs at least as well as the learning frameworks, if not better, in in-sample and out-of-sample forecast error criteria. Moreover, learning is not found to better explain time varying macroeconomic volatility any better than rational expectations. Finally, impulse response functions from the estimated models show that the dynamics following a structural shock can depend crucially on how expectations are initialized and what information agents are assumed to have.

The Dynamics of Economic Functions: Modelling and Forecasting the Yield Curve

The class of Functional Signal plus Noise (FSN) models is introduced that provides a new, general method for modelling and forecasting time series of economic functions. The underlying, continuous economic function (or `signal') is a natural cubic spline whose dynamic evolution is driven by a cointegrated vector autoregression for the ordinates (or 'y-values') at the knots of the spline. The natural cubic spline provides flexible cross-sectional fit and results in a linear, state space model. This FSN model achieves dimension reduction, provides a coherent description of the observed yield curve and its dynamics as the cross-sectional dimension N becomes large, and can feasibly be estimated and used for forecasting when N is large. The integration and cointegration properties of the model are derived. The FSN models are then applied to forecasting 36-dimensional yield curves for US Treasury bonds at the one month ahead horizon. The method consistently outperforms the Diebold and Li (2006) and random walk forecasts on the basis of both mean square forecast error criteria and economically relevant loss functions derived from the realised profits of pairs trading algorithms. The analysis also highlights in a concrete setting the dangers of attempts to infer the relative economic value of model forecasts on the basis of their associated mean square forecast errors.

Forecasting Currency Excess Returns: Can the Forward Bias Be Exploited?

AbstractThe forward bias anomaly implies that currency excess returns are predictable by the forward premium. Yet, recent studies suggest that statistical inference problems may spuriously account for this predictability. This article demonstrates that while currency excess returns are not predictable out of sample using a standard mean square forecast error criterion, the forward premium nonetheless has directional predictability. This directional forecasting accuracy translates into statistically significant profits from trading on the forward bias anomaly.

Long-Run Forecasting in Multicointegrated Systems

In this paper long-run forecasting of multicointegrating variables is investigated. Multicointegration typically occurs in dynamic systems involving both stock and flow variables whereby a in the form of shared stochastic trends is present across different levels of multiple time series. Hence, the effect of imposing this common feature restriction on out-of-sample valuation and forecasting accuracy of such variables is of interest. In particular, we compare the long-run forecasting performance of the multicointegrated variables between a model that correctly imposes the common feature restrictions and a (univariate) model that omits these multicointegrating restrictions completely. We employ different loss functions based on a range of mean square forecast error criteria, and the results indicate that different loss functions result in different ranking of models with respect to their infinite horizon forecasting performance. We consider loss functions using a standard trace mean square forecast error criterion (penalizing the forecast errors of flow variables only), and a loss function evaluating forecast errors of changes in both stock and flow variables. The latter loss function is based on the triangular representation of cointegrated systems and was initially suggested by Christoffersen and Diebold (1998). It penalizes deviations from long-run relations among the flow variables through cointegrating restrictions. We present a new loss function which further penalizes deviations in the long run relationship between the levels of stock and flow variables. It is derived from the triangular representation of multicointegrated systems. Using this criterion, system forecasts from a model incorporating multicointegration restrictions dominate forecasts from univariate models. The paper highlights the importance of carefully selecting loss functions in forecast evaluation of models involving stock and flow variables.

Heterogeneous expectations and tests of efficiency in the yen/dollar forward exchange rate market

This paper examines the efficiency of the forward yen/dollar market using micro survey data. Conventional tests of unbiasedness do not correspond directly to the zero-profit condition. Instead, we use the survey data to calculate potential profits of individual forecasters based on a natural trading rule. We find that although the survey data are not the best predictor of future spot rates in terms of typical mean square forecast error criteria, the survey data can be used to obtain on average positive profits. However, these profits are small and highly variable. Similar results are found when we examine profits generated by a trading rule using regression forecasts. The profits are found to be correlated with risk type variables but not other available information.

Professional economic forecasts: Are they worth their costs?

AbstractMost economic forecast evaluations dating back 20 years show that professional forecasters add little to the forecasts generated by the simplest of models. Using various types of forecast error criteria, these evaluations usually conclude that the professional forecasts are little better than the no‐change or ARIM A type forecast. It is our contention that this conclusion is mistaken because the conventional error criteria may not capture why forecasts are ma& or how they are used. Using forecast directional accuracy, the criterion which has been found to be highly correlated with profits in an interest rate setting, we find that professional GNP forecasts dominate the cheaper alternatives. Moreover, there appears to be no systematic relationship between this preferred criterion and the error measures used in previous studies.