Mean Square Forecast Error Research Articles

Evaluation of the ArcIOPS sea ice forecasts during 2021–2023

The operational sea ice forecasts from the Arctic Ice Ocean Prediction System (ArcIOPS) during 2021–2023 are validated against satellite-retrieved sea ice concentration and drift data, in situ and reanalyzed sea ice thickness data. The results indicate that the ArcIOPS has a reliable capacity on the Arctic sea ice forecasts for the future 7 days. Over the validation period, the root mean square error (RMSE) of the ArcIOPS sea ice concentration forecasts at a lead time of up to 168 h ranges between 8% and 20%, and the integrated ice edge error (IIEE) is lower than 1.6 × 106 km2 with respect to the Hai Yang 2B (HY-2B) sea ice concentration data. Compared to the Pan-Arctic Ice Ocean Modeling and Assimilation System (PIOMAS), sea ice volume evolution from the ArcIOPS forecasts is closer to that derived from the CS2SMOS sea ice thickness observations, which have been assimilated into the ArcIOPS. Sea ice thickness comparisons at three locations in the Beaufort Sea between the ArcIOPS forecasts and in situ mooring observations also prove that the sea ice thickness forecasts are credible, which sets a solid basis for supporting ice-breaker navigation in the Arctic thick ice zone. The sea ice drift deviations between the ArcIOPS forecasts and the National Snow and Ice Data Center (NSIDC) data are lower than 4 cm/s in most of the months. Future work will emphasize on developing multi-variable data assimilation scheme and fully coupled air‒ice‒ocean forecasting system for the Arctic sea ice forecasts.

Forecasting the price of oil: A cautionary note

We study the out-of-sample predictability of monthly crude oil prices using forecast combinations constructed from several individual predictor forecasts. Our empirical results indicate that combination forecasts of monthly average oil prices are more accurate than the no-change forecast with statistically significant reductions in mean square forecast errors (MSFE) and significant directional accuracy at every horizon up to 24 months, consistent with earlier evidence that forecast combinations greatly enhance the forecastability of oil prices. In contrast, we find no significant MSFE reductions or directional accuracy for forecasts of end-of-month oil prices at almost all horizons. Furthermore, we document that end-of-month forecasts when used to guide investment and hedging decisions of investors, statistically, do not deliver superior economic value to investors. Overall, the implication of our results is that the statistical and economic significance of forecasts of oil prices is heavily influenced by the construction of the underlying oil price series and provide a cautionary note on which oil price series to use in forecasting.

Pivot Clustering to Minimize Error in Forecasting Aggregated Demand Streams Each Following an Autoregressive Moving Average Model

In this paper, we compare the effects of forecasting demand using individual (disaggregated) components versus first aggregating the components either fully or into several clusters. Demand streams are assumed to follow autoregressive moving average (ARMA) processes. Using individual demand streams will always lead to a superior forecast compared to any aggregates; however, we show that if several aggregated clusters are formed in a structured manner, then these subaggregated clusters will lead to a forecast with minimal increase in mean-squared forecast error. We show this result based on theoretical MSFE obtained directly from the models generating the clusters as well as estimated MSFE obtained directly from simulated demand observations. We suggest a pivot algorithm, which we call Pivot Clustering, to create these clusters. We also provide theoretical results to investigate sub-aggregation, including for special cases, such as aggregating demand generated by MA(1) models and aggregating demand generated by ARMA models with similar or the same parameters.

Enhancing Nigerian Oil Price Forecasting: A Comprehensive Analysis of Model Averaging Techniques

Numerous fields of endeavour have benefited greatly from statistical forecasting, which has aided decision-making by planners and policy makers. In this study, Bayesian Model Averaging (BMA) and Dynamic Model Averaging (DMA) are employed to forecast oil prices in Nigeria. It aimed at predicting the oil prices in Nigeria. Essentially, there are lot of model uncertainties in empirical growth researches. The predictive performance value considering the Mean Squared Forecast Error (MSFE) for BMA and DMA were 920.23 & 540.40 respectively. The DMA predicted the model better than the BMA. High levels of model uncertainties were indeed accounted for, in conformity with the theoretical knowledge.

Forecasting Levels in Loglinear Unit Root Models

This article considers unbiased prediction of levels when data series are modeled as a random walk with drift and other exogenous factors after taking natural logs. We derive the unique unbiased predictors for growth and its variance. Derivation of level forecasts is more involved because the last observation enters the conditional expectation and is highly correlated with the parameter estimates, even asymptotically. This leads to conceptual questions regarding conditioning on endogenous variables. We prove that no conditionally unbiased forecast exists. We derive forecasts that are unconditionally unbiased and take into account estimation uncertainty, non linearity of the transformations, and the correlation between the last observation and estimate, which is quantitatively more important than estimation uncertainty and future disturbances together. The exact unbiased forecasts are shown to have lower Mean Squared Forecast Error (MSFE) than usual forecasts. The results are applied to Bitcoin price levels and a disaggregated eight sector model of UK industrial production.

A probabilistic perspective on predictability of solar irradiance using bootstrapped correlograms and ensemble predictability error growth

Recently in “Correlogram, predictability error growth, and bounds of mean square er-ror of solar irradiance forecasts” [Renew. Sustain. Energy Rev. 167 (2022) 112736], a new measure of predictability, applicable to solar irradiance in particular, was proposed. That predictability measure is expressed as one minus the ratio of the smallest attainable and highest tolerable root mean square errors (RMSEs) for a specific forecasting situation. Whereas the smallest attainable RMSE can be derived from the predictability error growth (PEG) of a numerical weather prediction model, the highest tolerable RMSE is computed from the forecasts of a naïve standard of reference. Despite that proposal has pioneered our understanding on the predictability of solar irradiance, answers to some technical challenges remain opaque. One of those technical challenges is the lack of uncertainty quantification of the predictability estimate, which is what this work seeks to resolve. Two conceptions, namely, ensemble PEG and bootstrapped correlograms, are herein introduced to facilitate the probabilistic representations of the smallest attainable and highest tolerable RMSEs, respectively. It is found that the uncertainty in predictability estimates is very small as compared to the range over which predictability varies across climatic conditions and forecast horizons.

Post-Pandemic Steel Production Scenarios for Poland Based on Forecasts of Annual Steel Production Volume

Abstract The paper presents the results of forecasts made for the volume of steel production in Poland based on actual data for the period from 2006 to 2021 with forecasting until 2026. The actual data used for the forecasts included annual steel production volumes in Poland (crude steel) in millions of tons. Basic adaptive methods were used to forecast the volume of steel production for the next five years. When selecting the methods, the course of the trend of the studied phenomenon was taken into account. In order to estimate the level of admissibility of the adopted forecasting methods, as well as to select the best forecasts, the errors of apparent forecasts (ex post) were calculated. Errors were calculated in the work: RMSE Root Mean Square Error being the square root of the mean square error of the ex-post forecasts yt for the period 2006-2021; ? as the mean value of the relative error of expired forecasts y*t (2006-2021) – this error informs about the part of the absolute error per unit of the real value of the variable yt. Optimization of the forecast values was based on the search for the minimum value of one of the above-mentioned errors, treated as an optimization criterion. In addition, the value of the point forecast (for 2022) obtained on the basis of the models used was compared with the steel production volume obtained for 3 quarters of 2022 in Poland with the forecast for the last quarter. Forecasting results obtained on the basis of the forecasting methods used, taking into account the permissible forecast errors, were considered as the basis for determining steel production scenarios for Poland until 2026. To determine the scenarios, forecast aggregation was used, and so the central forecasts were determined separately for decreasing trends and for increasing trends, based on the average values of the forecasts obtained for the period 2022-2026. The central forecasts were considered the baseline scenarios for steel production in Poland in 2022-2026 and the projected production volumes above the baseline forecasts with upward trends were considered an optimistic scenario, while the forecasted production volumes below the central scenario for downward trends were considered a pessimistic scenario for the Polish steel industry.

Deep learning of sea-level variability and flood for coastal city resilience

Due to climate change, it is important to study the relationship between floods and sea-level rise in coastal city resilience. In this research sea surface temperature (SST) from MODIS, wind speed, precipitation, and sea-level rise from satellite altimetry are investigated for dynamic sea-level variability. An annual SST increase of 0.1C° is observed around the Gothenburg coast. Also in the middle of the North Sea, an annual increase of about 0.2C° is evident. The annual sea surface height (SSH) trend is 3 mm on the Gothenburg coast. We have a strong positive spatial correlation between SST and SSH near the Gothenburg coast. In the next step, dynamic sea-level variability is predicted with a convolution neural network and long short term memory. Root mean square error of wind speed, precipitation, SST, and mean sea-level forecasts are ±0.84 m/s, ±48.75 mm, ±3.48C° and ±24 mm, respectively. The 5-year trends of mean seal level show a significant increase from 28 mm/year to 46 mm/year in the last 5 year periods and the rate of increase has doubled. In the final step, the water rise of 5–10 m in Gothenburg city was simulated, and in the worst scenario, more than 50 % of the city will be damaged.

Correlogram, predictability error growth, and bounds of mean square error of solar irradiance forecasts

Solar forecast verification must incorporate a notion of “relativity.” Particularly for deterministic forecasts, one wishes to know the upper and lower bounds of the mean square error (MSE) for the forecasting situation of concern, such that the performance of some forecasts of interest can be quantified relative to these bounds. This work proposes a method for estimating these bounds. For the upper bound of MSE, the theory builds upon the premise that no forecast worse than that provided by the standard of reference should be used. Under some mild assumptions, the MSE of the standard of reference, which is herein defined to be the optimal convex combination of climatology and persistence, varies solely with the lag-h autocorrelation of the clear-sky index. Then, by fitting a theoretical isotropic correlogram with a nugget effect to the empirical autocorrelations, the upper bound of MSE, as a function of forecast horizon, can be derived. For the lower bound of MSE, it can be approximated by the predictability error growth, which is the difference between the controlled and perturbed forecasts from a perfect dynamical weather model. The empirical part of this work considers the irradiance forecasts from the European Centre for Medium-Range Weather Forecasts, at seven locations in United States, over a period of two years (2019–2020). Various MSEs reported in the literature are put in perspective of the estimated bounds of MSE. The proposed bounds have a profound impact on predictability quantification and skill score computation, which are essential for comparative forecast verification.

Forecasting of short-term photovoltaic power generation using combined interval type-2 Takagi-Sugeno-Kang fuzzy systems

For the purpose of environmental protection and economic development, photovoltaic (PV) power generation is becoming increasingly popular, but the intermittence and uncertainty in PV power generation make grid-connected PV systems face severe challenges, accurate forecast is an effective way to solve this problem. Motived by the expectancy to process uncertainty and improve the prediction accuracy, in this study, a new prediction model combines interval type-2 Takagi-Sugeno-Kang (TSK) fuzzy neural network (type-2 TSKFNN) model optimized by extended Kalman filter (EKF) and SOM is proposed. A possible clustering interval is first determined according to the distribution of PV power generation, then the meteorological data are clustered using SOM, and the optimal size of categories is determined by Davies-Bouldin index (DBI). In each data partition, the decoupled extended Kalman filter (DKEF) is used to optimize the parameters of type-2 TSKFNN, including two Gaussian models with uncertain mean and uncertain standard deviation, and the independent fuzzy prediction models are established. The actual prediction analysis verifies the performance of proposed SOM-EKTSK model by using the data collected from Yulara Solar System in Australia, and the comparisons with other fuzzy logic system (FLS) models are also completed. The results show that the proposed SOM-EKTSK model has the highest prediction accuracy. Compared with other contrast models, the root mean square error (RMSE) of forecasts is reduced by 16.47% in spring, 4.75% in summer, 23.15% in autumn and 44.36% in winter, on average.© 2017 Elsevier Inc. All rights reserved.

Simultaneous prediction of functionally dependent random variables by maximum likelihood estimation

The paper presents a fundamental parametric approach to simultaneous forecasting of a vector of functionally dependent random variables. The motivation behind the proposed method is the following: each random variable at interest is forecasted by its own model and then adjusted in accordance with the functional link. The method incorporates the assumption that models’ errors are independent or weekly dependent. Proposed adjustment is explicit and extremely easy-to-use. Not only does it allow adjusting point forecasts, but also it is possible to adjust the expected variance of errors, that is useful for computation of confidence intervals. Conducted thorough simulation and empirical testing confirms, that proposed method allows to achieve a steady decrease in the mean-squared forecast error for each of predicted variables.

Forecasting residential gas demand: machine learning approaches and seasonal role of temperature forecasts

The one-day-ahead forecasting of residential gas demand at country level is investigated by implementing and comparing five models: ridge regression, Gaussian process (GP), k-nearest neighbour, artificial neural network (ANN) and torus model. Italian demand data are used for testing the proposed algorithms. The most significant aspects of the pre-processing and feature engineering are discussed, lending particular attention to the role of one-day-ahead temperature forecasts. Our best model, in terms of root mean square error (RMSE), is the ANN; if the mean absolute error (MAE) is considered, the GP becomes the best model, although by a narrow margin. A novel contribution is the development of a model describing the propagation of temperature forecast errors to gas forecasting errors that is successfully validated on data. On the Italian data, it is shown that temperature errors account for 18% of the mean square error of gas demand forecasts. [Received: October 3, 2019; Accepted: November 9, 2019]

On the Potential of Improving WRF Model Forecasts by Assimilation of High-Resolution GPS-Derived Water-Vapor Maps Augmented with METEOSAT-11 Data

Improving the accuracy of numerical weather predictions remains a challenging task. The absence of sufficiently detailed temporal and spatial real-time in-situ measurements poses a critical gap regarding the proper representation of atmospheric moisture fields, such as water vapor distribution, which are highly imperative for improving weather predictions accuracy. The estimated amount of the total vertically integrated water vapor (IWV), which can be derived from the attenuation of global positioning systems (GPS) signals, can support various atmospheric models at global, regional, and local scales. Currently, several existing atmospheric numerical models can estimate the IWV amount. However, they do not provide accurate results compared with in-situ measurements such as radiosondes. Here, we present a new strategy for assimilating 2D IWV regional maps estimations, derived from combined GPS and METEOSAT satellite imagery data, to improve Weather Research and Forecast (WRF) model predictions accuracy in Israel and surrounding areas. As opposed to previous studies, which used point measurements of IWV in the assimilation procedure, in the current study, we assimilate quasi-continuous 2D GPS IWV maps, combined with METEOSAT-11 data. Using the suggested methodology, our results indicate an improvement of more than 30% in the root mean square error (RMSE) of WRF forecasts after assimilation relative standalone WRF, when both are compared to the radiosonde measured data near the Mediterranean coast. Moreover, significant improvements along the Jordan Rift Valley and Dead Sea Valley areas are obtained when compared to 2D IWV regional maps estimations. Improvements in these areas suggest the impact of the assimilated high resolution IWV maps, with initialization times which coincide with the Mediterranean Sea Breeze propagation from the coastline to highland stations, as the distance to the Mediterranean Sea shore, along with other features, dictates its arrival times.

On the Predictability of the Equity Premium Using Deep Learning Techniques

Deep learning is drawing keen attention in contemporary financial research. In this article, the authors investigate the statistical predictive power and economic significance of financial stock market data by using deep learning techniques. In particular, the authors use the equity premium as the response variable and financial variables as predictors. The deep learning techniques used in this study provide useful evidence of statistical predictability and economic significance. Considering the statistical predictive performance of the deep learning models, H2O deep learning (H2ODL) gives the smallest mean-squared forecast error (MSFE), with the corresponding highest cumulative return (CR) and Sharpe ratio (SR) in each of the out-of-sample periods. Specifically, the H2ODL with Rectifier used as the activation function outperformed the other models in this article. In the fusion results, the SAE-with-H2O using the Maxout activation function yields the smallest MSFE with the corresponding highest CR and SR in all of the out-of-sample periods. It is worth noting that the higher the CR, the higher the SR and the lower the MSFE, which concords with a rule of thumb. Overall, the empirical analysis in this study revealed that the SAE-with-H2O using the Maxout activation function produced the best statistically predictive and economically significant results with robustness across all out-of-sample periods. <b>TOPICS:</b>Big data/machine learning, performance measurement, quantitative methods, simulations, statistical methods <b>Key Findings</b> ▪ In this article, the authors use deep learning models to predict the equity premium, employing a plethora of well-known predictors. ▪ The authors employ deep learning models such as deep neural networks, a stacked autoencoder, and long short-term memory models. ▪ The statistical and economic significance of the proposed models is examined and back tested in three out-of-sample periods.

Tail-Heaviness, Asymmetry, and Profitability Forecasting by Quantile Regression

We show that quantile regression is better than ordinary-least-squares (OLS) regression in forecasting profitability for a range of profitability measures following the conventional setup of the accounting literature, including the mean absolute forecast error (MAFE) evaluation criterion. Moreover, we perform both a simulated-data and an archival-data analysis to examine how the forecasting performance of quantile regression against OLS changes with the shape of the profitability distribution. Considering the MAFE and mean squared forecast error (MSFE) criteria together, we see that the quantile regression is more accurate relative to OLS when the profitability to be forecast has a heavier-tailed distribution. In addition, the asymmetry of the profitability distribution has either a U-shape or an inverted-U-shape effect on the forecasting accuracy of quantile regression. An application of the distributional shape analysis framework to cash flow forecasting demonstrates the usefulness of the framework beyond profitability forecasting, providing additional empirical evidence on the positive effect of tail-heaviness and supporting the notion of an inverted-U-shape effect of asymmetry. This paper was accepted by Shiva Rajgopal, accounting.

Predicting Interest Rates Using Shrinkage Methods, Real-Time Diffusion Indexes, and Model Combinations

In the context of predicting the term structure of interest rates, we explore the marginal predictive content of real-time diffusion indexes extracted from a “data rich” real-time dataset, when used in dynamic Nelson-Siegel (NS) models of the variety discussed in Diebold and Li (DNS: 2007) and Svensson (NSS: 1994). We find that the indexes have significant predictive content for sample periods ranging from 2001 through 2010. Additionally, DNS and NSS type models that include these indexes are the mean square forecast error (MSFE) “best” performers, when compared with various other econometric specifications, for the same period. In our top ranked models, diffusion indexes are sometimes optimally constructed used un-targeted principal component analysis (PCA), and sometimes using targeted PCA in conjunction with elastic net and least absolute shrinkage operators. The news is not all good for models utilizing indexes constructed from real-time datasets, however. In particular, after 2010 relatively few “data-rich” prediction models “beat” a random walk benchmark. Also, forecast combinations that utilize models that exclude real-time diffusion indexes yield the lowest overall MSFEs, dominating all other combination and individual models, across all sample periods, forecast horizons and bond maturities. Two key conclusions from our analysis are the following. First, fully revised data may have an important confounding effect upon results obtained when instead carrying out real-time prediction experiments. Second, real-time diffusion indexes matter when comparing the predictive performance of individual models, indicating the presence of unspanned macroeconomic risks in the term structure of interest rates. However, there appear to be two different ways to “capture” these unspanned risks. One is to use data rich real-time diffusion indexes, and another is to simply combine predictions from many non-data rich models.

Forecasting core inflation: the case of South Africa

ABSTRACT Underlying, or core, inflation is likely the most important variable for monetary policy. It is considered to be the optimal nominal anchor as it is stable, excludes relative price shocks, and reflects underlying trends in the behaviour of price-setters and demand conditions in the economy. Despite its importance, there is sparse literature on estimating and forecasting core inflation in South Africa, with the focus still on measuring it. This paper emphasizes predicting core inflation from time-varying parameter vector autoregressive models (TVP-VARs), factor-augmented VARs (FAVAR), and structural break models using quarterly data from 1981Q1 to 2013Q4. We use mean squared forecast errors (MSFE) and predictive likelihoods to evaluate the forecasts. In general, we find that (i) time-varying parameter models consistently outperform constant coefficient models (ii) small TVP-VARs outperform all other models; (iii) models with heteroscedastic errors do better than models with homoscedastic errors; and (iv) allowing for structural breaks does not improve the predictability of core inflation. Overall, our results imply that additional information on the growth rate of the economy and the interest rate is sufficient to forecast core inflation accurately, but the relationship between these three variables needs to be modelled in a time-varying fashion.

Analysis of the performance of competing models for aggregate demand forecasting using observable data characteristics

This paper provides an analysis of the performance of two simplified approaches for forecasting the aggregate demand for a product family consisting of two interrelated items. The correlated demand characteristic of the product family is modelled by bivariate vector IMA(1, 1) process. The forecasting approaches chosen for evaluation are univariate scheme and aggregate scheme. The performances of these methods are analysed by theoretically determining the forecast mean square error (MSE) in terms of observable data characteristics. Further, subsequent evaluation of percentage changes of these MSEs with respect to the MSE of original bivariate model has been conducted to determine how far the forecast MSE values deviate from the original model. Through numerical experimentation, the necessary and sufficient conditions in terms of observable data characteristics are determined for the two approaches to be equal in terms of forecast MSE of the original bivariate model.

A combined random effect and fixed effect forecast for panel data models

When some of the regressors in a panel data model are correlated with the random individual effects, the random effect (RE) estimator becomes inconsistent while the fixed effect (FE) estimator is consistent. Depending on the various degree of such correlation, we can combine the RE estimator and FE estimator to form a combined estimator which can be better than each of the FE and RE estimators. In this paper, we are interested in whether the combined estimator may be used to form a combined forecast to improve upon the RE forecast (forecast made using the RE estimator) and the FE forecast (forecast using the FE estimator) in out-of-sample forecasting. Our simulation experiment shows that the combined forecast does dominate the FE forecast for all degrees of endogeneity in terms of mean squared forecast errors (MSFE), demonstrating that the theoretical results of the risk dominance for the in-sample estimation carry over to the out-of-sample forecasting. It also shows that the combined forecast can reduce MSFE relative to the RE forecast for moderate to large degrees of endogeneity and for large degrees of heterogeneity in individual effects.

A Horse Race in High Dimensional Space

In this paper, we study the predictive power of dense and sparse estimators in a high dimensional space. We propose a new forecasting method, called Elastically Weighted Principal Components Analysis (EWPCA) that selects the variables, with respect to the target variable, taking into account the collinearity among the data using the Elastic Net soft thresholding. Then, we weight the selected predictors using the Elastic Net regression coefficient, and we finally apply the principal component analysis to the new “elastically” weighted data matrix. We compare this method to common benchmark and other methods to forecast macroeconomic variables in a data-rich environment, dived into dense representation, such as Dynamic Factor Models and Ridge regressions and sparse representations, such as LASSO regression. All these models are adapted to take into account the linear dependency of the macroeconomic time series. Moreover, to estimate the hyperparameters of these models, including the EWPCA, we propose a new procedure called “brute force”. This method allows us to treat all the hyperparameters of the model uniformly and to take the longitudinal feature of the time-series data into account. Our findings can be summarized as follows. First, the “brute force” method to estimate the hyperparameters is more stable and gives better forecasting performances, in terms of MSFE, than the traditional criteria used in the literature to tune the hyperparameters. This result holds for all samples sizes and forecasting horizons. Secondly, our two-step forecasting procedure enhances the forecasts’ interpretability. Lastly, the EWPCA leads to better forecasting performances, in terms of mean square forecast error (MSFE), than the other sparse and dense methods or naive benchmark, at different forecasts horizons and sample sizes.