Correlations Of Forecast Errors Research Articles

Characteristics of day‐ahead wind power forecast errors in Nordic countries and benefits of aggregation

AbstractThe growing proportion of wind power in the Nordic power system increases day‐ahead forecasting errors, which have a link to the rising need for balancing power. However, having a large interconnected synchronous power system has its benefits, because it enables to aggregate imbalances from large geographical areas. In this paper, day‐ahead forecast errors from four Nordic countries and the impacts of wind power plant dispersion on forecast errors in areas of different sizes are studied. The forecast accuracy in different regions depends on the amount of the total wind power capacity in the region, how dispersed the capacity is and the forecast model applied. Further, there is a saturation effect involved, after which the reduction in the relative forecast error is not very large anymore. The correlations of day‐ahead forecast errors between areas decline rapidly when the distance increases. All error statistics show a strong decreasing trend up to the area sizes of 50,000 km2. The average mean absolute error (MAE) in different regions is 5.7% of installed capacity. However, MAE of a smaller area can be over 8% of the capacity, but when all the Nordic regions are aggregated together, the capacity‐normalized MAE decreases to 2.5%. The average of the largest errors for different regions is 39.8% and when looking at the largest forecast errors for smaller areas, the largest errors can exceed 80% of the installed capacity, whereas at the Nordic level, the maximum forecast error is only 13.5% of the installed capacity. Copyright © 2016 John Wiley & Sons, Ltd.

Sampling Error Damping Method for a Cloud-Resolving Model Using a Dual-Scale Neighboring Ensemble Approach

Abstract In ensemble-based assimilation schemes for cloud-resolving models (CRMs), the precipitation-related variables have serious sampling errors. The purpose of the present study is to examine the sampling error properties and the forecast error characteristics of the operational CRM of the Japan Meteorological Agency (JMANHM) and to develop a sampling error damping method based on the CRM forecast error characteristics. The CRM forecast error was analyzed for meteorological disturbance cases using 100-member ensemble forecasts of the JMANHM. The ensemble forecast perturbation correlation had a significant noise associated with the precipitation-related variables, because of sampling errors. The precipitation-related variables were likely to suffer this sampling error in most precipitating areas. An examination of the forecast error characteristics revealed that the CRM forecast error satisfied the assumption of the spectral localization, while the spatial localization with constant scales, or variable localization, were not applicable to the CRM. A neighboring ensemble (NE) method was developed, which was based on the spectral localization that estimated the forecast error correlation at the target grid point, using ensemble members for neighboring grid points. To introduce this method into an ensemble-based variational assimilation scheme, the present study horizontally divided the NE forecast error into large-scale portions and deviations. As single observation assimilation experiments showed, this “dual-scale NE” method was more successful in damping the sampling error and generating plausible, deep vertical profile of precipitation analysis increments, compared to a simple spatial localization method or a variable localization method.

Co-movements between crude oil and food prices: A post-commodity boom perspective

Using the correlations of VAR forecast errors at different horizons, this paper analyzes the dynamics of co-movements between crude oil and food prices. For each food price considered, a bivariate VAR model is estimated on two subsample periods: a pre-commodity-boom (1990M1–2006M12) and a post-boom period (2007M1–2015M5). Our results reveal strong positive co-movements between crude oil and food prices in the aftermath of the commodity boom, while no statistically significant co-movements are observed over the pre-boom period. Hence, our findings provide further empirical evidence on the actual linkages between the crude oil and agricultural markets.

Oxidative Cross-Coupling Reactions

Using the correlations of VAR forecast errors at different horizons, this paper analyzes the dynamics of co-movements between crude oil and food prices. For each food price considered, a bivariate VAR model is estimated on two subsample periods: a pre-commodity-boom (1990M1–2006M12) and a post-boom period (2007M1–2015M5). Our results reveal strong positive co-movements between crude oil and food prices in the aftermath of the commodity boom, while no statistically significant co-movements are observed over the pre-boom period. Hence, our findings provide further empirical evidence on the actual linkages between the crude oil and agricultural markets.

Leveraging stochastic differential equations for probabilistic forecasting of wind power using a dynamic power curve

Short-term (hours to days) probabilistic forecasts of wind power generation provide useful information about the associated uncertainty of these forecasts. Standard probabilistic forecasts are usually issued on a per-horizon-basis, meaning that they lack information about the development of the uncertainty over time or the inter-temporal correlation of forecast errors for different horizons. This information is very important for forecast end-users optimizing time-dependent variables or dealing with multi-period decision-making problems, such as the management and operation of power systems with a high penetration of renewable generation. This paper provides input to these problems by proposing a model based on stochastic differential equations that allows generating predictive densities as well as scenarios for wind power. We build upon a probabilistic model for wind speed and introduce a dynamic power curve. The model thus decomposes the dynamics of wind power prediction errors into wind speed forecast errors and errors related to the conversion from wind speed to wind power. We test the proposed model on an out-of-sample period of 1year for a wind farm with a rated capacity of 21MW. The model outperforms simple as well as advanced benchmarks on horizons ranging from 1 to 24h. Copyright © 2016 John Wiley & Sons, Ltd.

European Commission's Forecasts Accuracy Revisited: Statistical Properties and Possible Causes of Forecast Errors

This paper updates a previous assessment of the European Commission's track record for forecasting key economic variables (Gonzalez Cabanillas and Terzi 2012) by extending the observation period to 2014. It also examines the accuracy of the Commission's forecasts over a shorter and more recent period (2000-2014) so that a comparison can be made between the performance of forecasts made before and after the Great Recession of 2008-2009. Going beyond the 2012 approach, this paper also examines the extent to which forecast errors can be explained by external or technical assumptions that prove incorrect ex post. It also updates the comparison of the Commission’s performance vis a vis the OECD, the IMF, a consensus forecast of market economists, and the ECB.Inclusion of the 2012-2014 period lowers the forecasting error for some key variables or leads to no change in others. Focusing on the years since the turn of the century, current-year and year-ahead forecasting errors for the three main variables examined (GDP growth, inflation and general government balances) have been larger in the crisis and post-crisis period (2008-2014) than in the pre-crisis period (2000-2007) for a large majority of Member States. This appears mainly to be the result of an anomalously large error in 2009, a year which confounded many forecasters. The country-by-country analysis confirms the finding of earlier studies which show that the Commission's forecasts are largely unbiased. The newly-introduced panel data approach also confirms the absence of bias in current-year GDP forecasts across EU Member States but shows that year-ahead forecasts for GDP growth tend to be slightly over optimistic across the whole sample. The analysis also shows that auto correlation of forecast errors is not a major issue in the Commission's forecasts. Other advanced tests shed more light on the performance of the Commission’s forecasts, demonstrating that they are directionally accurate and generally beat a naive forecast but that they are not always efficient in terms of their use of all available data.The decomposition of forecast errors shows that unexpected changes in external assumptions seem to have only a limited impact on current-year GDP growth forecasts. However, more than half of the variance in year-ahead forecast errors appears to come from external assumptions that prove to be incorrect ex post. Finally, the Commission’s economic forecasts come out as being more accurate than those of the market and comparable to those of the other international institutions considered.

Co-Movements between Crude Oil and Food Prices: A Post-Commodity Boom Perspective

Using the correlations of VAR forecast errors at different horizons, this paper analyses the dynamics of co-movements between crude oil and food prices. For each food price considered, the VAR model is estimated on two subsample periods: a pre-commodity boom (1990M1-2006M12) and a post-boom period (2007M1-2015M12). Results show strong positive co-movements between the crude oil and food prices in the aftermath of the commodity boom, while no statistically significant co-movements are observed over the pre-boom period. Our findings then provide an additional empirical evidence on the actual linkages between the crude oil and agricultural markets.

Performance Assessment of Hydrological Models Considering Acceptable Forecast Error Threshold

It is essential to consider the acceptable threshold in the assessment of a hydrological model because of the scarcity of research in the hydrology community and errors do not necessarily cause risk. Two forecast errors, including rainfall forecast error and peak flood forecast error, have been studied based on the reliability theory. The first order second moment (FOSM) and bound methods are used to identify the reliability. Through the case study of the Dahuofang (DHF) Reservoir, it is shown that the correlation between these two errors has great influence on the reliability index of hydrological model. In particular, the reliability index of the DHF hydrological model decreases with the increasing correlation. Based on the reliability theory, the proposed performance evaluation framework incorporating the acceptable forecast error threshold and correlation among the multiple errors can be used to evaluate the performance of a hydrological model and to quantify the uncertainties of a hydrological model output.

Spatial Postprocessing of Ensemble Forecasts for Temperature Using Nonhomogeneous Gaussian Regression

Abstract Statistical postprocessing techniques are commonly used to improve the skill of ensembles from numerical weather forecasts. This paper considers spatial extensions of the well-established nonhomogeneous Gaussian regression (NGR) postprocessing technique for surface temperature and a recent modification thereof in which the local climatology is included in the regression model to permit locally adaptive postprocessing. In a comparative study employing 21-h forecasts from the Consortium for Small Scale Modelling ensemble predictive system over Germany (COSMO-DE), two approaches for modeling spatial forecast error correlations are considered: a parametric Gaussian random field model and the ensemble copula coupling (ECC) approach, which utilizes the spatial rank correlation structure of the raw ensemble. Additionally, the NGR methods are compared to both univariate and spatial versions of the ensemble Bayesian model averaging (BMA) postprocessing technique.

Estimation of analysis and forecast error variances

Accurate estimates of error variances in numerical analyses and forecasts (i.e. difference between analysis or forecast fields and nature on the resolved scales) are critical for the evaluation of forecasting systems, the tuning of data assimilation (DA) systems and the proper initialisation of ensemble forecasts. Errors in observations and the difficulty in their estimation, the fact that estimates of analysis errors derived via DA schemes, are influenced by the same assumptions as those used to create the analysis fields themselves, and the presumed but unknown correlation between analysis and forecast errors make the problem difficult. In this paper, an approach is introduced for the unbiased estimation of analysis and forecast errors. The method is independent of any assumption or tuning parameter used in DA schemes. The method combines information from differences between forecast and analysis fields (‘perceived forecast errors’) with prior knowledge regarding the time evolution of (1) forecast error variance and (2) correlation between errors in analyses and forecasts. The quality of the error estimates, given the validity of the prior relationships, depends on the sample size of independent measurements of perceived errors. In a simulated forecast environment, the method is demonstrated to reproduce the true analysis and forecast error within predicted error bounds. The method is then applied to forecasts from four leading numerical weather prediction centres to assess the performance of their corresponding DA and modelling systems. Error variance estimates are qualitatively consistent with earlier studies regarding the performance of the forecast systems compared. The estimated correlation between forecast and analysis errors is found to be a useful diagnostic of the performance of observing and DA systems. In case of significant model-related errors, a methodology to decompose initial value and model-related forecast errors is also proposed and successfully demonstrated.

Geodynamo model and error parameter estimation using geomagnetic data assimilation

S U M M A R Y We have developed a new geomagnetic data assimilation approach which uses the minimum variance’ estimate for the analysis state, and which models both the forecast (or model output) and observation errors using an empirical approach and parameter tuning. This system is used in a series of assimilation experiments using Gauss coefficients (hereafter referred to as observational data) from the GUFM1 and CM4 fieldmodels for the years 1590–1990.We show that this assimilation system could be used to improve our knowledge of model parameters, model errors and the dynamical consistency of observation errors, by comparing forecasts of the magnetic field with the observations every 20 yr. Statistics of differences between observation and forecast (O − F) are used to determine how forecast accuracy depends on the Rayleigh number, forecast error correlation length scale and an observation error scale factor. Experiments have been carried out which demonstrate that a Rayleigh number of 30 times the critical Rayleigh number produces better geomagnetic forecasts than lower values, with an Ekman number of E = 1.25 × 10−6, which produces a modified magnetic Reynolds number within the parameter domain with an ‘Earth like’ geodynamo. The optimal forecast error correlation length scale is found to be around 90 per cent of the thickness of the outer core, indicating a significant bias in the forecasts. Geomagnetic forecasts are also found to be highly sensitive to estimates of modelled observation errors: Errors that are too small do not lead to the gradual reduction in forecast error with time that is generally expected in a data assimilation system while observation errors that are too large lead to model divergence. Finally, we show that assimilation of L ≤ 3 (or large scale) gauss coefficients can help to improve forecasts of the L > 5 (smaller scale) coefficients, and that these improvements are the result of corrections to the velocity field in the geodynamo model.

Management Insights

Management Insights

Sources of earnings variability and their effect on earnings forecasts

AbstractPrevious research shows that analysts’ forecasts of earnings do not fully incorporate information contained in reported earnings variability. This study investigates whether the inefficient forecast is because of a failure to incorporate observable information on two components of earnings variability: variability in operating performance and income smoothing. Our results show that analysts’ forecasts fully incorporate information contained in earnings variability for firms with high income smoothing and for firms with low operating variability. A smaller serial correlation of forecast errors is observed for firms with low operating variability, which suggests that analysts recognize the permanence in earnings for such firms.

Investigation of statistical structure of temperature short-range forecast errors in the atmospheric boundary layer for the purpose of objective analysis

The procedure is described of the estimation of statistical structure of short-range forecast errors of temperature field in the atmospheric boundary layer for the purpose of objective analysis. The numerical experiments on the estimation of forecast error covariances were carried out for the WRF (Weather Research and Forecast, NCEP, USA) model. The comparative experiments were implemented on the estimation of the influence of stability type in the atmospheric boundary layer on the variability of vertical and three-dimensional covariances. It is demonstrated that the variance and radius of correlation of temperature field forecast errors along the vertical as well as the variability of three-dimensional covariance functions in the atmospheric boundary layer differ considerably at different stability types. The results are cited of the numerical experiments on the estimation of the WRF model temperature field forecast error covariances in the atmospheric boundary layer for summer and winter periods.

Reading the Tea Leaves: Why Serial Correlation Patterns in Analysts' Forecast Errors are not Evidence of Inefficient Information Processing

We put forward a model in which analysts are uncertain about a firm's earnings process. Faced with the possibility of using a misspecified model, analysts issue forecasts that are robust to model misspecification. We estimate that this mechanism explains approximately 60% of the autocorrelation in analysts' forecast errors. The remainder stems from the cross-sectional variation in mean forecast errors and in analysts' estimation errors of the persistence of earnings growth shocks. Consistent with our model, we find that analysts learn about some features of the earnings process but not others, and this learning reduces, but does not eliminate, the auto- correlation of forecast errors as firms age. Other potential explanations for the autocorrelation of analyst's forecast errors are rejected. Our model of robust forecasting applies not only to analysts' forecasts but to all model-based forecasts.

Disagreement and Biases in Inflation Expectations

Disagreement in inflation expectations observed from survey data varies systematically over time in a way that reflects the level and variance of current inflation. This paper offers a simple explanation for these facts based on asymmetries in the forecasters' costs of over- and under-predicting inflation. Our model implies (i) biased forecasts; (ii) positive serial correlation in forecast errors; (iii) a cross-sectional dispersion that rises with the level and the variance of the inflation rate; and (iv) predictability of forecast errors at different horizons by means of the spread between the short- and long-term variance of inflation. We find empirically that these patterns are present in inflation forecasts from the Survey of Professional Forecasters. A constant bias component, not explained by asymmetric loss and rational expectations, is required to explain the shift in the sign of the bias observed for a substantial portion of forecasters around 1982.

Disagreement and Biases in Inflation Expectations

Disagreement in inflation expectations observed from survey data varies systematically over time in a way that reflects the level and variance of current inflation. This paper offers a simple explanation for these facts based on asymmetries in the forecasters' costs of over‐ and underpredicting inflation. Our model implies (i) biased forecasts, (ii) positive serial correlation in forecast errors, (iii) a cross‐sectional dispersion that rises with the level and the variance of the inflation rate, and (iv) predictability of forecast errors at different horizons by means of the spread between the short‐ and long‐term variance of inflation. We find empirically that these patterns are present in inflation forecasts from the Survey of Professional Forecasters. A constant bias component, not explained by asymmetric loss and rational expectations, is required to explain the shift in the sign of the bias observed for a substantial portion of forecasters around 1982.

Model Uncertainty and Serial Correlation in Forecast Errors

Model Uncertainty and Serial Correlation in Forecast Errors

Serial Correlation in Management Earnings Forecast Errors*

We examine whether management earnings forecast errors exhibit serial correlation and how analysts understand the serial correlation property of management forecast errors. Management forecast errors should not exhibit serial correlation if managers efficiently process information in prior forecast errors and truthfully convey their earnings expectations through management forecasts. However, for long-horizon management forecasts of annual earnings, we find significantly positive serial correlation in management forecast errors, and sample self-selection does not seem to drive this phenomenon. Further analyses suggest that managers’ unintentional information processing bias contributes to this positive serial correlation. Analysts anticipate the inter-temporal persistence of management forecast errors but underestimate the persistence level when reacting to management forecasts. Our findings have implications for market participants who rely on management forecasts to form earnings expectations, and also shed light on the efficiency of managerial decision making.

Estimation of the local diffusion tensor and normalization for heterogeneous correlation modelling using a diffusion equation

AbstractAs the background‐error covariance matrix is a key component of any assimilation system, its modelling is an important step. Usually, this matrix is decomposed into correlations and standard deviations matrices. An interesting method for modelling the correlation matrix of the background error for complex geometry, like an ocean grid, consists in computing correlation functions using a diffusion operator. The background‐error correlation functions can be estimated for example from an ensemble of perturbed forecasts. The diffusion operator is able to represent heterogeneous correlation functions at a reasonable numerical cost. But a first challenge resides in the determination of the local diffusion tensor corresponding to the local correlation function. Then the second challenge resides in the determination of the normalization to make sure that the matrix modelled through the diffusion operator is a correlation matrix. In this article, we propose to build a background‐error correlation matrix using a diffusion operator based on a local diffusion tensor. The estimation of this local tensor is performed using an ensemble of perturbed forecasts. A validation within a randomization method illustrates the feasibility and the accuracy of the proposed method. In particular, it is shown that the local geographical variations of diagnosed correlation functions (through an ensemble of perturbed forecast) are well represented.This is first illustrated in an analytical one‐dimensional framework. In that context, the diffusion field and the normalization field are deduced from a given correlation length‐scale field. The resulting length‐scales are shown to correspond to the initial length‐scale when the given length‐scale field spectrum is red. The approximate normalization, computed from the local length‐scale, is close to the true normalization under the same condition of a red spectrum.Then, the method is illustrated in a real context using an ensemble of perturbed forecasts from the MOCAGE–PALM assimilation system. In that case, length‐scale and anisotropy diagnosis reveal the complexity of the correlation of stratospheric ozone forecast errors. The local diffusion tensor deduced from these diagnoses is shown be able to represent such an existing heterogeneity and anisotropy. As in the one‐dimensional case, the approximate normalization, based on the local diffusion tensor, appears to be a really good approximation of the true normalization. Copyright © 2008 Royal Meteorological Society