Accuracy Of Point Forecasts Research Articles

Statistical post‐processing of ensemble forecasts of temperature in Santiago de Chile

AbstractModelling forecast uncertainty is a difficult task in any forecasting problem. In weather forecasting a possible solution is the use of forecast ensembles, which are obtained from multiple runs of numerical weather prediction models with various initial conditions and model parametrizations to provide information about the expected uncertainty. Currently all major meteorological centres issue forecasts using their operational ensemble prediction systems. However, it is a general problem that the spread of the ensemble is too small compared to observations at specific sites resulting in under‐dispersive forecasts, leading to a lack of calibration. In order to correct this problem, various statistical calibration techniques have been developed in the last two decades. In the present work different post‐processing techniques were tested for calibrating nine member ensemble forecasts of temperature for Santiago de Chile, obtained by the Weather Research and Forecasting model using different planetary boundary layer and land surface model parametrizations. In particular, the ensemble model output statistics and Bayesian model averaging techniques were implemented and, since the observations are characterized by large altitude differences, the estimation of model parameters was adapted to the actual conditions at hand. Compared to the raw ensemble, all tested post‐processing approaches significantly improve the calibration of probabilistic forecasts and the accuracy of point forecasts. The ensemble model output statistics method using parameter estimation based on expert clustering of stations (according to their altitudes) shows the best forecast skill.

Gaussian mixture model coupled recurrent neural networks for wind speed interval forecast

The potential of long short-term memory network on ultra-short term wind speed forecast attracted attentions of researchers in recent years. Extending a probabilistic long short-term memory network model to provide an uncertainty estimation than to make a point forecast is more valuable in practice. However, due to complex recurrent structure and feedback algorithm, large scale ensemble forecast based on resampling faces great challenges in reality. Instead, a reliable forecast method needs to be devised. Gaussian process regression is a probabilistic regression model based on Gaussian Process prior. It is reasonable to integrate Gaussian process regression with long short-term memory network for probabilistic wind speed forecast to leverage the superior fitting ability of the deep learning methods and to maintain the probability characteristics of Gaussian process regression. Hence, avoid the repeated training and heavy parameter optimization. The method is evaluated for wind speed forecast using the monitoring dataset provided by the National Wind Energy Technology Center. The results indicated that the proposed method improves the point forecast accuracy by up to 17.2%, and improves the interval forecast accuracy by up to 18.5% compared to state-of-the-art models. This study is of great significance for improving the accuracy and reliability of wind speed prediction and the sustainable development of new energy sources.

Forecasting aggregate equity return volatility using crude oil price volatility: The role of nonlinearities and asymmetries

To assess the potential nonlinear predictive impact of crude oil price volatility on aggregate equity return volatility, we consider autoregressions of monthly aggregate equity return realized volatility augmented with nonlinear transformations of crude oil price realized volatility and evaluate if they improve point forecasts. Out-of-sample results based on data from 1885m1 through 1895m12 and from 1983m1 through 2017m12 illustrate that our conclusions depend heavily on the notion of forecast improvement. At the population level, the null hypothesis of no out-of-sample predictability from crude oil price realized volatility to aggregate equity return realized volatility is rejected for the linear as well as certain nonlinear specifications. On the other hand, the null hypothesis of finite-sample equal predictive ability is rejected less frequently. Among the range of models, the autoregression augmented with the one-year net crude oil price realized volatility increase is the top performer, producing statistically significant more accurate point forecasts than the benchmark.

DYNAMIC PRINCIPAL COMPONENT REGRESSION: APPLICATION TO AGE-SPECIFIC MORTALITY FORECASTING

AbstractIn areas of application, including actuarial science and demography, it is increasingly common to consider a time series of curves; an example of this is age-specific mortality rates observed over a period of years. Given that age can be treated as a discrete or continuous variable, a dimension reduction technique, such as principal component analysis (PCA), is often implemented. However, in the presence of moderate-to-strong temporal dependence, static PCA commonly used for analyzing independent and identically distributed data may not be adequate. As an alternative, we consider a dynamic principal component approach to model temporal dependence in a time series of curves. Inspired by Brillinger’s (1974, Time Series: Data Analysis and Theory. New York: Holt, Rinehart and Winston) theory of dynamic principal components, we introduce a dynamic PCA, which is based on eigen decomposition of estimated long-run covariance. Through a series of empirical applications, we demonstrate the potential improvement of 1-year-ahead point and interval forecast accuracies that the dynamic principal component regression entails when compared with the static counterpart.

Enhancing survey‐based investment forecasts

AbstractWe investigate the accuracy of capital investment predictors from a national business survey of South African manufacturing. Based on data available to correspondents at the time of survey completion, we propose variables that might inform the confidence that can be attached to their predictions. Having calibrated the survey predictors' directional accuracy, we model the probability of a correct directional prediction using logistic regression with the proposed variables. For point forecasting, we compare the accuracy of rescaled survey forecasts with time series benchmarks and some survey/time series hybrid models. In addition, using the same set of variables, we model the magnitude of survey prediction errors. Directional forecast tests showed that three out of four survey predictors have value but are biased and inefficient. For shorter horizons we found that survey forecasts, enhanced by time series data, significantly improved point forecasting accuracy. For longer horizons the survey predictors were at least as accurate as alternatives. The usefulness of the more accurate of the predictors examined is enhanced by auxiliary information, namely the probability of directional accuracy and the estimated error magnitude.

Bayesian forecast combination in VAR-DSGE models

We evaluate the performance of individual and combination forecasts produced by Bayesian vector autoregressions (Bayesian VARs) with economic and/or non-economic information. In particular, we conduct an out-of-sample forecasting experiment using Bayesian VARs with statistical and/or DSGE prior(s) over two subsamples, representing the so-called “Great Inflation” and “Great Moderation” periods, respectively. Our main findings are summarized as follows. First, in most cases, the inclusion of prior information, either statistical or economic prior(s), improves the accuracy of point forecasts produced by the Bayesian VARs, relative to the ARMA(1,1) benchmark. Second, forecast combination helps to produce unbiased forecasts in most cases. Moreover, it significantly improves the forecast accuracy of the macro-econometric models, especially during the “Great Moderation” period. Third, the selection of the weighting scheme in forecast combination, simple averaging or Bayesian model averaging (BMA), does not affect the conclusions made above. Finally, partially in line with the recent literature, we find a positive relationship between the persistence of inflation and its relative forecast accuracy at the four-quarter horizon.

Conditional Residual Modeling for Probabilistic Load Forecasting

Probabilistic load forecasting (PLF) has gained widespread attention in recent years because it presents more uncertainty information about the future loads. To further improve the PLF performance, this letter proposes a novel PLF method to leverage existing point load forecasts by modeling the conditional forecast residual. Specifically, the method firstly conducts point forecasting using the historical load data and related factors to obtain the point forecast. Then, this point forecast is used as an additional input feature to describe the conditional distribution of the residual on the point forecast. Finally, the point forecast and conditional distribution of the residual are integrated together to produce the final probabilistic forecast. By comparing different point forecasting and quantile regression models, comprehensive case studies obtained from a publicly available load dataset with multiple zones demonstrate the advantages of our proposed method. This letter also informatively reveals the relationship between point and probabilistic forecast accuracies.

When providing optimistic and pessimistic scenarios can be detrimental to judgmental demand forecasts and production decisions

This paper examines the accuracy of judgmental forecasts of product demand and the quality of subsequent production level decisions under two different conditions: (i) the availability of only time series information on past demand; (ii) the availability of time series information together with scenarios that outline possible prospects for the product in the forthcoming period. An experiment indicated that production level decisions made by participants had a greater deviation from optimality when they also received optimistic and pessimistic scenarios. This resulted from less accurate point forecasts made by these participants. Further analysis suggested that participants focussed on the scenario that was congruent with the position of the latest observation relative to the series mean and discounted the opposing scenario. This led to greater weight being attached to this observation, thereby exacerbating the tendency of judgmental forecasters to see systematic changes in random movements in time series.

A Novel Hybrid Interval Prediction Approach Based on Modified Lower Upper Bound Estimation in Combination with Multi-Objective Salp Swarm Algorithm for Short-Term Load Forecasting

Effective and reliable load forecasting is an important basis for power system planning and operation decisions. Its forecasting accuracy directly affects the safety and economy of the operation of the power system. However, attaining the desired point forecasting accuracy has been regarded as a challenge because of the intrinsic complexity and instability of the power load. Considering the difficulties of accurate point forecasting, interval prediction is able to tolerate increased uncertainty and provide more information for practical operation decisions. In this study, a novel hybrid system for short-term load forecasting (STLF) is proposed by integrating a data preprocessing module, a multi-objective optimization module, and an interval prediction module. In this system, the training process is performed by maximizing the coverage probability and by minimizing the forecasting interval width at the same time. To verify the performance of the proposed hybrid system, half-hourly load data are set as illustrative cases and two experiments are carried out in four states with four quarters in Australia. The simulation results verified the superiority of the proposed technique and the effects of the submodules were analyzed by comparing the outcomes with those of benchmark models. Furthermore, it is proved that the proposed hybrid system is valuable in improving power grid management.

Forecasting US GNP growth: The role of uncertainty

AbstractA large number of models have been developed in the literature to analyze and forecast changes in output dynamics. The objective of this paper was to compare the predictive ability of univariate and bivariate models, in terms of forecasting US gross national product (GNP) growth at different forecasting horizons, with the bivariate models containing information on a measure of economic uncertainty. Based on point and density forecast accuracy measures, as well as on equal predictive ability (EPA) and superior predictive ability (SPA) tests, we evaluate the relative forecasting performance of different model specifications over the quarterly period of 1919:Q2 until 2014:Q4. We find that the economic policy uncertainty (EPU) index should improve the accuracy of US GNP growth forecasts in bivariate models. We also find that the EPU exhibits similar forecasting ability to the term spread and outperforms other uncertainty measures such as the volatility index and geopolitical risk in predicting US recessions. While the Markov switching time‐varying parameter vector autoregressive model yields the lowest values for the root mean squared error in most cases, we observe relatively low values for the log predictive density score, when using the Bayesian vector regression model with stochastic volatility. More importantly, our results highlight the importance of uncertainty in forecasting US GNP growth rates.

Effective real-time forecasting of inundation maps for early warning systems during typhoons

Accurate forecasts of hourly inundation depths are essential for inundation warning and mitigation during typhoons. In this paper, an effective forecasting model is proposed to yield 1- to 6-h lead-time inundation maps for early warning systems during typhoons. The proposed model based on Support Vector Machine (SVM) is composed of two modules, point forecasting and spatial expansion. In the first module, the rainfall intensity, inundation depth, cumulative rainfall and forecasted inundation depths are considered as model input for point forecasting. In the second module, the geographic information of inundation grids and the inundation forecasts of reference points are used to yield inundation maps for spatial expansion. The results show that the proposed model is able to provide accurate point forecasts at each inundation point. Moreover, the spatial expansion module is capable of producing accurate spatial inundation forecasts. Obviously, the proposed model provides reasonable spatial inundation forecasts, and is able to deal with the nonlinear relationships between inputs and desired output. In conclusion, the proposed model is suitable and useful for inundation forecasting.

Forecasting the Real Price of Oil Under Alternative Specifications of Constant and Time-Varying Volatility

This paper constructs a monthly real-time oil price dataset using backcasting and compares the forecast performance of alternative models of constant and time-varying volatility based on the accuracy of point and density forecasts of real oil prices of both real-time and ex-post revised data. The paper considers Bayesian autoregressive and autoregressive moving average models with respectively, constant volatility and two forms of time-varying volatility: GARCH and stochastic volatility. In addition to the standard time-varying models, more flexible models with volatility in mean and moving average innovations are used to forecast the real price of oil. The results show that time-varying volatility models dominate their counterparts with constant volatility in terms of point forecasting at longer horizons and density forecasting at all horizons. The inclusion of a moving average component provides a substantial improvement in the point and density forecasting performance for both types of time-varying models while stochastic volatility in mean is superfluous for forecasting oil prices.

Rule-based autoregressive moving average models for forecasting load on special days: A case study for France

This paper presents a case study on short-term load forecasting for France, with emphasis on special days, such as public holidays. We investigate the generalisability to French data of a recently proposed approach, which generates forecasts for normal and special days in a coherent and unified framework, by incorporating subjective judgment in univariate statistical models using a rule-based methodology. The intraday, intraweek, and intrayear seasonality in load are accommodated using a rule-based triple seasonal adaptation of a seasonal autoregressive moving average (SARMA) model. We find that, for application to French load, the method requires an important adaption. We also adapt a recently proposed SARMA model that accommodates special day effects on an hourly basis using indicator variables. Using a rule formulated specifically for the French load, we compare the SARMA models with a range of different benchmark methods based on an evaluation of their point and density forecast accuracy. As sophisticated benchmarks, we employ the rule-based triple seasonal adaptations of Holt-Winters-Taylor (HWT) exponential smoothing and artificial neural networks (ANNs). We use nine years of half-hourly French load data, and consider lead times ranging from one half-hour up to a day ahead. The rule-based SARMA approach generated the most accurate forecasts.

Grouped multivariate and functional time series forecasting:An application to annuity pricing

Age-specific mortality rates are often disaggregated by different attributes, such as sex, state, ethnic group and socioeconomic status. In making social policies and pricing annuity at national and subnational levels, it is important not only to forecast mortality accurately, but also to ensure that forecasts at the subnational level add up to the forecasts at the national level. This motivates recent developments in grouped functional time series methods (Shang and Hyndman, in press) to reconcile age-specific mortality forecasts. We extend these grouped functional time series forecasting methods to multivariate time series, and apply them to produce point forecasts of mortality rates at older ages, from which fixed-term annuities for different ages and maturities can be priced. Using the regional age-specific mortality rates in Japan obtained from the Japanese Mortality Database, we investigate the one-step-ahead to 15-step-ahead point-forecast accuracy between the independent and grouped forecasting methods. The grouped forecasting methods are shown not only to be useful for reconciling forecasts of age-specific mortality rates at national and subnational levels, but they are also shown to allow improved forecast accuracy. The improved forecast accuracy of mortality rates is of great interest to the insurance and pension industries for estimating annuity prices, in particular at the level of population subgroups, defined by key factors such as sex, region, and socioeconomic grouping.

Assessing point forecast accuracy by stochastic error distance

ABSTRACTWe propose point forecast accuracy measures based directly on distance of the forecast-error c.d.f. from the unit step function at 0 (“stochastic error distance,” or SED). We provide a precise characterization of the relationship between SED and standard predictive loss functions, and we show that all such loss functions can be written as weighted SEDs. The leading case is absolute error loss. Among other things, this suggests shifting attention away from conditional-mean forecasts and toward conditional-median forecasts.

Grouped Functional Time Series Forecasting: An Application to Age-Specific Mortality Rates

ABSTRACTAge-specific mortality rates are often disaggregated by different attributes, such as sex, state, and ethnicity. Forecasting age-specific mortality rates at the national and sub-national levels plays an important role in developing social policy. However, independent forecasts at the sub-national levels may not add up to the forecasts at the national level. To address this issue, we consider reconciling forecasts of age-specific mortality rates, extending the methods of Hyndman et al. in 2011 to functional time series, where age is considered as a continuum. The grouped functional time series methods are used to produce point forecasts of mortality rates that are aggregated appropriately across different disaggregation factors. For evaluating forecast uncertainty, we propose a bootstrap method for reconciling interval forecasts. Using the regional age-specific mortality rates in Japan, obtained from the Japanese Mortality Database, we investigate the one- to ten-step-ahead point and interval forecast accuracies between the independent and grouped functional time series forecasting methods. The proposed methods are shown to be useful for reconciling forecasts of age-specific mortality rates at the national and sub-national levels. They also enjoy improved forecast accuracy averaged over different disaggregation factors. Supplementary materials for the article are available online.

An evaluation of inflation expectations in Turkey

Expectations of inflation play a critical role in the process of price setting in the market. Central banks closely follow developments in inflation expectations to implement a successful monetary policy. The Central Bank of the Republic of Turkey (CBRT) conducts a survey of experts and decision makers in the financial and real sectors to reveal market expectations and predictions of current and future inflation. The survey is conducted every month. This paper examines the accuracy of these survey predictions using forecast evaluation techniques. We focus on both point and sign accuracy of the predictions. Although point predictions from CBRT surveys are compared with those of autoregressive models, sign predictions are evaluated on their value to a user. We also test the predictions for bias. Unlike the empirical evidence from other economies, our results show that autoregressive models outperform most of inflation expectations in forecasting inflation. This indicates that inflation expectations have poor point forecast accuracies. However, we show that sign predictions for all inflation expectations have value to a user.

Forecast accuracy of a BVAR under alternative specifications of the zero lower bound

AbstractThis paper discusses how the forecast accuracy of a Bayesian vector autoregression (BVAR) is affected by introducing the zero lower bound on the federal funds rate. As a benchmark I adopt a common BVAR specification, including 18 variables, estimated shrinkage, and no nonlinearity. Then I entertain alternative specifications of the zero lower bound. I account for the possibility that the effect of monetary policy on the economy is different in this regime, replace the federal funds rate by its shadow rate, consider a logarithmic transformation, feed in monetary policy shocks, or utilize conditional forecasts allowing for all shocks implemented through a rejection sampler. The latter two are also coupled with interest rate expectations from future contracts. It is shown that the predictive densities of all these specifications are greatly different, suggesting that this modeling choice is not innocuous. The comparison is based on the accuracy of point and density forecasts of major US macroeconomic series during the period 2009:1 to 2014:4. The introduction of the zero lower bound is not beneficial per se, but it depends on how it is done and which series is forecasted. With caution, I recommend the shadow rate specification and the rejection sampler combined with interest rate expectations to deal with the nonlinearity in the policy rate. Since the policy rate will remain low for some time, these findings could prove useful for practical forecasters.

DSGE-based forecasting: What should our perspective be?

The article compares the accuracy of point forecasts made with a structural dynamic stochastic general equilibrium model (DSGE) to those made with vector autoregressions estimated by OLS (VAR) and by Bayesian methods (BVAR). The main question addressed in the article is whether DSGE-based forecasts are as accurate as non-structural model ones. The comparison is made on the ground of mean squared forecast errors. The results show that the forecasting ability of the DSGE model is in general inferior to that of the BVAR. However, the difference is not critical. Moreover, for some variables and forecasting horizons, the DSGE model produces the forecast with the lowest error among all three models in question.

Reconciling Forecasts of Infant Mortality Rates at National and Sub-National Levels: Grouped Time-Series Methods

Mortality rates are often disaggregated by different attributes, such as sex, state, education, religion, or ethnicity. Forecasting mortality rates at the national and sub-national levels plays an important role in making social policies associated with the national and sub-national levels. However, base forecasts at the sub-national levels may not add up to the forecasts at the national level. To address this issue, we consider the problem of reconciling mortality rate forecasts from the viewpoint of grouped time-series forecasting methods (Hyndman et al. in, Comput Stat Data Anal 55(9):2579–2589, 2011). A bottom-up method and an optimal combination method are applied to produce point forecasts of infant mortality rates that are aggregated appropriately across the different levels of a hierarchy. We extend these two methods by considering the reconciliation of interval forecasts through a bootstrap procedure. Using the regional infant mortality rates in Australia, we investigate the one-step-ahead to 20-step-ahead point and interval forecast accuracies among the independent and these two grouped time-series forecasting methods. The proposed methods are shown to be useful for reconciling point and interval forecasts of demographic rates at the national and sub-national levels, and would be beneficial for government policy decisions regarding the allocations of current and future resources at both the national and sub-national levels.