Forecasting Exercise Research Articles

Combining density forecast accuracy tests: an application to agricultural, energy, and metal commodities

Abstract This study develops a new methodology for combining density forecast accuracy tests and assessing the relevance of psychological indicators in predicting commodity returns. Density forecasts provide a complete description of the uncertainty associated with a prediction and are highly requested by policymakers, central bankers, and financial operators to define policy actions, manage financial risks, and assess portfolio selection. The proposed methodology combines different tests and derives the p-value of the resulting test statistic by Monte Carlo simulations. To assess the power of the proposed methodology, we implement a set of experiments for several data-generating processes. Based on an empirical forecasting exercise applied to agricultural, energy, and metal commodities, we find that sentiment variables and psychological factors improve the density forecasts of commodity futures returns, especially for agricultural commodities. Additionally, combinations of sentiment variables are more powerful in predicting returns than considering them separately.

Forecasts of thermal coal prices through Gaussian process regressions

Given thermal coal's significance as a tactical energy source, price projections for the commodity are crucial for investors and decision-makers alike. The goal of the current work is to determine whether Gaussian process regressions are useful for this forecast problem using a dataset of closing prices of thermal coal traded on the China Zhengzhou Commodity Exchange from January 4, 2016, to December 31, 2020. This is a significant financial index that has not received enough attention in the literature in terms of price forecasting. Our forecasting exercises make use of Bayesian optimizations and cross-validation. The price from January 02, 2020, to December 31, 2020 is successfully predicted by the generated models, with the out-of-sample relative root mean square error of 0.4210%. Gaussian process regressions are shown to be useful for the thermal coal price forecast problem. The outcomes of this projection might be used as independent technical forecasts or in conjunction with other forecasts for policy research that entails developing viewpoints on price patterns.

Wasserstein barycenter regression: application to the joint dynamics of regional GDP and life expectancy in Italy

AbstractWe propose to investigate the joint dynamics of regional gross domestic product and life expectancy in Italy through Wasserstein barycenter regression derived from optimal transport theory. Wasserstein barycenter regression has the advantage of being flexible in modeling complex data distributions, given its ability to capture multimodal relationships, while maintaining the possibility of incorporating uncertainty and priors, other than yielding interpretable results. The main findings reveal that regional clusters tend to emerge, highlighting inequalities in Italian regions in economic and life expectancy terms. This suggests that targeted policy actions at a regional level fostering equitable development, especially from an economic viewpoint, might reduce regional inequality. Our results are validated by a robustness check on a human mobility dataset and by an illustrative forecasting exercise, which confirms the model’s ability to estimate and predict joint distributions and produce novel empirical evidence.

Forecasting Bitcoin returns: Econometric time series analysis vs. machine learning

AbstractWe study the statistical properties of the Bitcoin return series and provide a thorough forecasting exercise. Also, we calibrate state‐of‐the‐art machine learning techniques and compare the results with econometric time series models. The empirical assessment provides evidence that the application of machine learning techniques outperforms econometric benchmarks in terms of forecasting precision for both in‐ and out‐of‐sample forecasts. We find that both deep learning architectures as well as complex layers, such as LSTM, do not increase the precision of daily forecasts. Specifically, a simple recurrent neural network describes a sensible choice for forecasting daily return series.

The time-varying Multivariate Autoregressive Index model

Many economic variables are characterized by changes in their conditional mean and volatility, and time-varying Vector Autoregressive Models are often used to handle such complexity. Unfortunately, as the number of series grows, they present increasing estimation and interpretation issues. This paper tries to address this problem by proposing a Multivariate Autoregressive Index model that features time-varying mean and volatility. Technically, we develop a new estimation methodology that mixes switching algorithms with the forgetting factors strategy of Koop and Korobilis (2012). This substantially reduces the computational burden and allows one to select or weigh the number of common components, and other data features, in real-time without additional computational costs. Using US macroeconomic data, we provide a forecast exercise that shows the feasibility and usefulness of this model.

Bayesian Markov switching model for BRICS currencies' exchange rates

AbstractExchange rate modeling has always fascinated researchers because of its complex macroeconomic dynamics. This study documents the exchange rate dynamics of major emerging economies after accounting for their macroeconomic cycles and explores the Bayesian Vector Error Correction Model (VECM) Markov Regime switching model, which uses time‐varying transition probabilities. The main objective is to study the exchange rate dynamics of Brazil, Russia, India, China, and South Africa (BRICS) vis‐à‐vis the US dollar. The Bayesian setup uses two hierarchal shrinkage priors, the normal‐gamma (NG) prior and the Litterman prior, for parameters' estimation. These shrinkage priors allow for a more comprehensive assessment of the regime‐specific coefficients. The model performed well in differentiating between the two regimes for all currencies. The Russian ruble was identified to be the most depreciated currency, whereas the African Rand was the most appreciated. The evaluation of model features revealed that many regime‐specific coefficients differed significantly from their common mean. A forecasting exercise was then performed for the out‐of‐sample period to assess the model's performance. A significant improvement was observed over the basic random walk (RW) model and the linear Bayesian vector autoregression (BVAR) model.

Explaining long-term bond yields synchronization dynamics in Europe

We present a novel empirical assessment of the determinants of sovereign yield synchronization dynamics in the European Monetary Union. This topic has been seldom addressed in the existing macroeconometric literature. We use a time-varying measure of government bond yields synchronization and Bayesian Model Averaging methods to show that the persistence of synchronization measures differs significantly between GIIPS countries (Greece, Ireland, Italy, Portugal and Spain) and the rest of the monetary union, as well as across periods characterized by whether the zero lower bound of interest rates was binding or not. The degree of synchronization in inflation rates with the rest of the currency area is a robust predictor of the synchronization of sovereign yields, as opposed to economic fundamentals describing the fiscal positions of individual countries. An out-of-sample forecasting exercise reveals that accounting for the most relevant economic fundamentals can lead to improvements in the directional accuracy of the forecasts of yield synchronization rates for GIIPS countries.

Did the COVID-19 pandemic permanently impact e-commerce in the US market?

The pandemic compelled many individuals, initially hesitant about online shopping, to overcome their reservations, acquire essential skills, and transition to online retail. This provided a natural experiment to assess whether the barriers to online shopping and the comfort of traditional in-store habits have hindered a broader shift to e-commerce. This paper uses the US retail data to analyze e-commerce activities before, during, and after the pandemic to determine whether the pandemic has permanently altered the pattern of the activities to determine whether the pandemic has permanently altered the pattern of the activities by utilizing structural break detection tools. Additionally, we carry out a forecasting exercise for post-pandemic based on pre-pandemic data to confirm our findings. Results suggest that while e-commerce activities surged during lockdown, they have predominantly reverted to pre-pandemic patterns. Our findings caution both investors and companies against overreaction in the face of exuberant changes in the market to avoid painful corrections afterward.

Predicting tail risks and the evolution of temperatures

This paper explores a range of simple models to study the relationship between global temperature anomalies and climate forcings. In particular, we consider quantile regression models with potentially time-varying parameters (TVP), implemented by Bayesian methods. In its most general specification, this approach is flexible in that it models distinct regions of distribution of global temperature anomalies, while also allowing us to investigate changes in the relationship between (natural and anthropogenic) climate forcings and temperatures. Our results indicate that there is indeed considerable variation over time in the relationship between temperatures and its drivers, and that these effects may be heterogeneous across different quantiles. We then perform a long-range forecasting exercise for temperatures, which suggests that incorporating TVP or explicitly modelling quantile levels or the combination of both features can improve prediction for different parts of the temperature distribution. In addition, we produce forecasts for 2030 considering the intermediate RCP 4.5 scenario: given that no single specification dominates, we account for model uncertainty by considering forecast averaging across all specifications. Our approach allows us to make statements about the probability of temperature levels — for instance, we find that a scenario of +1.8 °C will occur with a non-negligible probability under RCP 4.5.

Toward high-resolution projection of electricity prices: A machine learning approach to quantifying the effects of high fuel and CO2 prices

The growing share of renewables, the retirement of coal generation, and the increasing significance (and price) of carbon emissions continue to reshape electricity market dynamics. Ongoing power sector transformations, along with recurrent energy price shocks, bring new challenges to electricity price prediction and projection. Adept at capturing nonlinear and time-varying dynamics of the power market, machine learning (ML) methods can significantly enhance modeling and projection capabilities. This paper leverages ML modeling to (1) analyze the day-ahead electricity price dynamics and (2) develop future price projections under varying market conditions. First, we offer a methodology for identifying and ranking the factors determining day-ahead power price behavior. Guided by the understanding of power price drivers, we then develop an ensemble ML model for high-resolution (i.e., for each day of the next year) electricity price projection. Hence, the first part of our analysis helps relate fundamental modeling approaches to ML studies, and the second part extends the existing literature, which is limited to high aggregation level (e.g., yearly basis) projections. High-resolution projections provide valuable insights into within-year price dynamics and help answer highly debated questions raised by the growing share of renewables and the current energy market turmoil. We demonstrate our methodology on the German power market, using data on 80 explanatory variables from 2015 to 2021. We identify the most critical power price drivers and develop an ML model to study the within-year impact of high fuel and carbon prices. Our findings indicate that the often-neglected interaction variables may significantly impact price projections and should be included in day-ahead electricity price modeling. We confirm the efficacy of our approach with a forecasting exercise based on day-ahead electricity prices in the first two quarters of 2022.

Reservoir computing for macroeconomic forecasting with mixed-frequency data

Macroeconomic forecasting has recently started embracing techniques that can deal with large-scale datasets and series with unequal release periods. Mixed-data sampling (MIDAS) and dynamic factor models (DFMs) are the two main state-of-the-art approaches to modeling series with non-homogeneous frequencies. We introduce a new framework, called the multi-frequency echo state network (MFESN), based on a relatively novel machine learning paradigm called reservoir computing. Echo state networks (ESNs) are recurrent neural networks formulated as nonlinear state-space systems with random state coefficients where only the observation map is subject to estimation. MFESNs are considerably more efficient than DFMs and can incorporate many series, as opposed to MIDAS models, which are prone to the curse of dimensionality. All methods are compared in extensive multistep forecasting exercises targeting U.S. GDP growth. We find that our MFESN models achieve superior or comparable performance over MIDAS and DFMs at a much lower computational cost.

Forecasting individual exercise sweat losses from forecast air temperature and energy expenditure.

Recent success in predicting individual sweat losses from air temperature and energy expenditure measurements suggests a potential for forecasting individual sweat losses for future combinations of environment and exercise. The purpose of this study is to determine the plausibility of accurately forecasting exercise sweat losses from meteorological air temperature forecasts and individual running energy expenditure forecasts. The potential impact on plasma sodium is also estimated when setting drinking rates equal to forecast sweat losses. Individual exercise sweat losses (equated to water needs) and energy expended while running were measured in 33 participants along with air temperature and compared with forecasts of the same. Forecast inputs were used in a web app to forecast exercise sweat losses for comparison with observed values. The bias between forecast and observed exercise sweat losses was used to calculate the potential drinking impact on plasma sodium. The concordance correlation coefficient between forecast and observed values was 0.95, 0.96, and 0.91 for air temperature, energy expenditure, and exercise sweat losses, respectively, indicating excellent agreement and no significant differences observed via t-test. Perfect matching of water intake to sweat losses would lower plasma sodium concentrations from 140 to 138 mmol/L; calculations using the 95% limits of agreement for bias showed that drinking according to forecast exercise sweat losses would alter plasma sodium concentrations from 140 to between 136 and 141 mmol/L. The outcomes support the strong potential for accurately forecasting exercise sweat losses from commonly available meteorological air temperature forecasts and energy expenditure from forecast running distance.

Large Order-Invariant Bayesian VARs with Stochastic Volatility

Many popular specifications for Vector Autoregressions (VARs) with multivariate stochastic volatility are not invariant to the way the variables are ordered due to the use of a lower triangular parameterization of the error covariance matrix. We show that the order invariance problem in existing approaches is likely to become more serious in large VARs. We propose the use of a specification which avoids the use of this lower triangular parameterization. We show that the presence of multivariate stochastic volatility allows for identification of the proposed model and prove that it is invariant to ordering. We develop a Markov chain Monte Carlo algorithm which allows for Bayesian estimation and prediction. In exercises involving artificial and real macroeconomic data, we demonstrate that the choice of variable ordering can have non-negligible effects on empirical results when using the nonorder invariant approach. In a macroeconomic forecasting exercise involving VARs with 20 variables we find that our order-invariant approach leads to the best forecasts and that some choices of variable ordering can lead to poor forecasts using a conventional, non-order invariant, approach.

An Iterative Approach toward Development of Ensemble Visualization Techniques for High-Impact Winter Weather Hazards: Part II: Product Evaluation

Abstract We developed five prototype convection-allowing model ensemble visualization products with the goal of improving depictions of the timing of winter weather hazards. These products are interactive, web-based plots visualizing probabilistic onset times and durations of intense snowfall rates, probabilities of heavy snow at rush hour, periods of heightened impacts, and mesoscale snowband probabilities. Prototypes were evaluated in three experimental groups coordinated by the Weather Prediction Center (WPC) Hydrometeorological Testbed (HMT), with a total of 53 National Weather Service (NWS) forecasters. Forecasters were asked to complete a simple forecast exercise for a snowfall event, with a control group using the Storm Prediction Center’s (SPC) High-Resolution Ensemble Forecast (HREF) system viewer, and an experimental group using both the HREF viewer and the five experimental graphics. Forecast accuracy was similar between the groups, but the experimental group exhibited smaller mean absolute error for snowfall duration forecasts. A series of Likert-scale questions saw participants respond favorably to all of the products and indicated that they would use them in operational forecasts and in communicating information to core partners. Forecasters also felt that the new products improved their comprehension of ensemble spread and reduced the time required to complete the forecasting exercise. Follow-up plenary discussions reiterated that there is a high demand for ensemble products of this type, though a number of potential improvements, such as greater customizability, were suggested. Ultimately, we demonstrated that social science methods can be effectively employed in the atmospheric sciences to yield improved visualization products.

Forecasting Future Crime Rates

Despite its importance for testing criminological theories and informing crime control policy, forecasting crime rates has all but disappeared from criminology. We argue for a resurgence of crime forecasting in the study of crime trends. As an example of the value, as well as the challenges, of forecasting, we engage in a forecasting exercise based on data from New York City. We discuss each of the steps taken to forecast New York’s violent and property crime rates to 2024, from preparing the data for reliable analysis, specifying the forecasting model, selecting the forecasting method, and validating the results. The results of autoregressive integrated moving average (ARIMA) forecast models show a rise in New York’s violent and property crime rates in 2022 and 2023 before flattening in 2024. Renewed attention to forecasting can help to secure the future of the study of crime trends.

Do extreme range estimators improve realized volatility forecasts? Evidence from G7 Stock Markets

This paper investigates whether range estimators contain important information in forecasting future realized volatility. We use widely applied range-based estimators: Parkinson, Garman-Klass, Roger-Satchell, and Yang-Zhang within a HAR-RV-X framework. Overnight volatility and close-to-close volatility estimators are also included, and the forecasting exercise is applied to G7 stock markets using a rolling window. Using QLIKE, HMSE and MCS forecast criteria, several noteworthy points are reported. The overall findings suggest that while no single model dominates, overnight return volatility achieves the most consistent performance. For example, HAR-RV model forecasts for CAC and DAX indices are improved only by overnight volatility, with some evidence also for SPX. For other indices, forecasts are improved by Parkinson and/or Garman-Klass volatility estimators. Of note, simpler range estimators outperform more complex range estimators. The findings could be important for investors in managing portfolio risk.

Firm-level business uncertainty and the predictability of the aggregate U.S. stock market volatility during the COVID-19 pandemic

In this paper, we analyze the predictive role of firm-level business expectations and uncertainties derived from a panel survey of U.S. 1750 business executives from 50 states for the realized variance (sum of daily squared log-returns over a month) of the S&P500 index over the monthly period of September 2016 to July 2021. Unlike standard models, our predictive framework adopts a time-varying approach due to the existence of multiple structural breaks in the relationship between volatility and the predictors in the model, which in turn leads to statistically insignificant causal effects in a constant parameter set-up. Our time-varying results suggest that the predictive power of firm-level business uncertainty is concentrated during the early part of the sample associated with the U.S.-China trade war and towards the end of our data coverage in the wake of the outbreak of the COVID-19 pandemic. Since in-sample predictability does not guarantee the same over an out-sample, we also conducted a full-fledged forecasting exercise to show that subjective expectations and uncertainties associated with sales growth rates and employment produce statistically significant predictability gains over January 2020 to July 2021, given an in-sample of September 2016 to December 2019. Our results suggest that subjective measures of business uncertainty at the firm level indeed capture predictive information regarding aggregate stock market uncertainty, which has important implications for investors and economic projections at the policy level.

Forecasting the stock-cryptocurrency relationship: Evidence from a dynamic GAS model

The impact of cryptocurrency on other assets has become a subject of intense research, given the rise of digital currency over the last decade. However, unlike traditional assets, cryptocurrency has been subject to extreme movements in price and volatility. As a result, it has become important for investors and risk managers to model and forecast volatility and correlation between digital currency and other assets. This paper utilises a multivariate generalised autoregressive score (GAS) model to study the time-varying dependence between stock prices (S&P500, NASDAQ, Dow Jones Industrial) and cryptocurrencies (Bitcoin and Ethereum). The results show that the GAS framework outperforms the traditional DCC-GARCH model, capturing the volatility persistence and non-linearity between stock and cryptocurrency. Regarding the correlations, while we identify a time-varying relationship, the strength of this relationship is in the low-to-moderate range. In addition, our forecasting exercise shows that the GAS specification has superior forecasting ability beyond certain horizon days compared to the DCC-GARCH model.

Stochastic debt sustainability analysis using time-varying fiscal reaction functions - an agnostic approach to fiscal forecasting

ABSTRACT This paper presents a model-based approach for stochastic primary balance and public debt simulations to assess fiscal sustainability in selected OECD countries. Fiscal behaviour is modelled by means of a fiscal reaction function with time-varying coefficients, which is then, together with a time-varying coefficient vector autoregression, embedded in a stochastic debt sustainability analysis framework. In a pseudo-out-of-sample forecasting exercise using vintage datasets, the model is evaluated against its frequently used fixed coefficient pendant and the European Commission’s Economic Forecasts at different horizons. The results indicate that stochastic debt sustainability analyses based on time-varying fiscal reaction functions and vector autoregressions perform competitively in terms of mean squared error and forecast bias at different horizons, especially with respect to public debt as well as short-term primary balance forecasts. Thus, models of this sort should be considered for complementary use at policy institutions, using them jointly with more ‘discretionary’ approaches to fiscal sustainability analysis.

The future(s) of unpaid work: How susceptible do experts from different backgrounds think the domestic sphere is to automation?

The future of work has become a prominent topic for research and policy debate. However, the debate has focused entirely on paid work, even though people in industrialized countries on average spend comparable amounts of time on unpaid work. The objectives of this study are therefore (1) to expand the future of work debate to unpaid domestic work and (2) to critique the main methodology used in previous studies. To these ends, we conducted a forecasting exercise in which 65 AI experts from the UK and Japan estimated how automatable are 17 housework and care work tasks. Unlike previous studies, we applied a sociological approach that considers how experts' diverse backgrounds might shape their estimates. On average our experts predicted that 39 percent of the time spent on a domestic task will be automatable within ten years. Japanese male experts were notably pessimistic about the potentials of domestic automation, a result we interpret through gender disparities in the Japanese household. Our contributions are providing the first quantitative estimates concerning the future of unpaid work and demonstrating how such predictions are socially contingent, with implications to forecasting methodology.