Forecast Combination Research Articles

Correction to: Optimal and robust combination of forecasts via constrained optimization and shrinkage

We provide a correction to Proposition 1 in Optimal and robust combination of forecasts via constrained optimization and shrinkage, published in the International Journal of Forecasting 38(1):97-116 (2021). This correction has no impact on any other result (neither theoretical nor empirical) provided in the above paper.

Convective‐scale numerical weather prediction and warnings over Lake Victoria: Part II—Can model output support severe weather warning decision‐making?

AbstractThe greater Lake Victoria region was the focus of the ‘HIGHWeather impact lAke sYstem’ project, with the aim of developing an early warning capability for hazardous weather. Strong thunderstorm outflows can cause boats to capsize and lead to drowning. Lightning is also a hazard to fishermen, and over land flash flooding from heavy rainfall is a significant hazard. Between March and October 2019, 16 regional warnings were issued, and 14 days had reported impacts. Only one of the issued warnings aligned with any reported impacts. The issued warnings were verified using three different methodologies and found to have a very low frequency bias, confirming that too few warnings are being issued, especially for precipitation. The skill of the warnings based on the probability of detection and symmetric extremal dependency index was also found to be low. Potential warnings days are derived from individual and multivariate combinations of numerical weather prediction forecasts of mean wind speed, precipitation and a new lightning diagnostic from the 4.4 km Tropical Africa configuration of the Met Office Unified Model. The methodology identified 5–7 times the issued number of warnings, including 50% of days with reported impacts and 25% of days with issued warnings. Overall, 70%–80% of the identified potential warning days appear to be justified, suggesting that for this period not enough warnings were issued. It is shown that a daily aggregate of the lightning diagnostic is a skilful addition to any guidance supporting a decision‐making process.

Combination model for day-ahead solar forecasting using local and global model input

Numerical Weather Prediction (NWP) models over limited areas enable the simulation of local atmospheric processes in more detail and with a higher degree of accuracy when compared to global models. Limited-area NWP models can outperform their global counterparts due to higher resolution (ability to explicitly simulate processes) and tailored physics (global models, unless run as a physics ensemble, have one set of parameterization schemes for the whole globe). However, increased accuracy from an NWP model is not guaranteed and can vary based on the location and variable of interest. In this paper, we present a method for combining the output of a limited-area NWP model, the Weather Research and Forecasting model (WRF) and its global model—the European Center for Medium Range Weather Forecasting (ECMWF) deterministic model. We simulate day-ahead global horizontal irradiance for a location in Qinghai, China. WRF model configurations optimized by the type of day (cloud amount) are then implemented based on the ECMWF model forecast of cloud amount. A regression model to combined ECMWF and WRF model forecasts is then trained. The optimized coefficients (weights) of ECMWF and WRF show increasing WRF importance with higher cloud amounts and the combination out-performs the ECMWF input by 5.2% and the best WRF configuration by 7.2% on a 2.5-month testing set. The performance of the combined model increased with observed cloud amount where the combined model out-performed the ECMWF model by 12.6% for cloudy days indicating the relative importance of physical downscaling for the simulation of clouds.

Can machine learning improve small area population forecasts? A forecast combination approach

Generating accurate small area population forecasts is vital for governments and businesses as it provides better grounds for decision making and strategic planning of future demand for services and infrastructure. Small area population forecasting faces numerous challenges, including complex underlying demographic processes, data sparsity, and short time series due to changing geographic boundaries. In this paper, we propose a novel framework for small area forecasting which combines proven demographic forecasting methods, an exponential smoothing based algorithm, and a machine learning based forecasting technique. The proposed forecasting combination contains four base models commonly used in demographic forecasting, a univariate forecasting model specifically suitable for forecasting yearly data, and a globally trained Light Gradient Boosting Model (LGBM) that exploits the similarities between a collection of population time series. In this study, three forecast combination techniques are investigated to weight the forecasts generated by these base models. We empirically evaluate our method, by preparing small area population forecasts for Australia and New Zealand. The proposed framework is able to achieve competitive results in terms of forecasting accuracy. Moreover, we show that the inclusion of the LGBM model always improves the accuracy of combination models on both datasets, relative to combination models which only include the demographic models. In particular, the results indicate that the proposed combination framework decreases the prevalence of relatively poor forecasts, while improving the reliability of small area population forecasts.

Out-of-sample forecasting of cryptocurrency returns: A comprehensive comparison of predictors and algorithms

Existing evidence shows that daily cryptocurrency returns are predictable by publicly available variables. However, a majority of evidence relies on potentially over-fitted in-sample estimation. This paper provides a comprehensive comparison of predictors and forecasting methods in the literature for out-of-sample return predictions of Bitcoin, Ethereum, and Ripple. We find that (1) well-known in-sample predictors such as investor attention and trading volume fail to produce statistically significant out-of-sample predictability, (2) a change in stochastic correlation with stock markets is the only meaningful predictor with out-of-sample R2 up to 2.69%, 1.71%, and 2.12% for Bitcoin, Ethereum, and Ripple, respectively, and (3) forecasting methods greatly differ in their performances; methods that are inspired by economic mechanism outperform universal forecasting methods such as shrinkage estimators, combination forecasts, monitoring forecasts, and various machine learning algorithms that are commonly used in practice.

Forecasting the Chinese Stock Market Volatility with G7 Stock Market Volatilities: A Scaled PCA Approach

ABSTRACT In this study, we forecast the realized volatility (RV) of the Chinese stock market using the heterogeneous autoregressive (HAR) model and various extended models. To extract the volatility information from the G7 stock markets, we employ a newly proposed approach, the scaled principal component analysis (SPCA), to produce a diffusion index and extend the HAR benchmark (HAR-SPCA). To validate the effectiveness of the SPCA approach, we employ three other dimension reduction approaches, the kitchen sink model, and five popular forecast combinations to deal with multivariate information and make competing forecasts. The results suggest that the combined volatility information from the G7 stock markets significantly predicts Chinese stock market volatility. More importantly, the forecasts from the HAR-SPCA model are steadily more accurate than the benchmark and other competing models under various evaluation criteria. Finally, our results are persistent to various robustness checks and the evaluation of portfolio performance.

192 Engineering Strategies and Solutions to Improve Swine Resilience and Alleviate Physiological Stress in a Changing Climate

Abstract Heat stress negatively affects a variety of production parameters in the swine industry including growth, carcass composition, and reproduction. The combination of climate forecasts and improved genetic performance, indicate increasingly negative impacts of heat stress for the future. Therefore, future heat stress mitigation approaches will need engineered solutions that also consider expected economic risk and corresponding uncertainty. Physically modifying the environment is currently the best approach by utilizing the heat dissipation (albeit, poor) mechanisms of the pig, that is, via the 4 modes of heat transfer, conduction, convection, radiation, and evaporation. A temperature gradient for sensible or vapor pressure gradient for latent heat loss are needed to drive heat exchange. Strategies can then be grouped into (a) modifying the thermodynamic properties of the incoming air via the ventilation system; (b) building characteristics or features; (c) animal zone environment modification; and (d) changes in management. Examples of each strategy include (a) evaporative cooling pads, earth-air-heat exchanger; (b) insulation, spatial orientation; (c) fogging, cooling the drinking water, increasing air velocity; and (d) stocking density. Engineering of future controlled swine environments will need enhanced modeling of heat transfer between pigs and their environment, advanced design/analysis methods of environmental control, and creation of innovative technology that improves sensing, controlling, and modifying the environment. Much of the engineering process starts with and relies on established standards that describe key analytical methods for design, data to enable design computations, and criteria/goals for different environments. There is and will continue to be a need for consistent updates to these data to ensure modern intensive housing is able to provide the proper thermoneutral zone, ventilation rates, etc. for prolific pigs in a changing climate.

Combining Predictions of Auto Insurance Claims

This paper aims to better predict highly skewed auto insurance claims by combining candidate predictions. We analyze a version of the Kangaroo Auto Insurance company data and study the effects of combining different methods using five measures of prediction accuracy. The results show the following. First, when there is an outstanding (in terms of Gini Index) prediction among the candidates, the “forecast combination puzzle” phenomenon disappears. The simple average method performs much worse than the more sophisticated model combination methods, indicating that combining different methods could help us avoid performance degradation. Second, the choice of the prediction accuracy measure is crucial in defining the best candidate prediction for “low frequency and high severity” (LFHS) data. For example, mean square error (MSE) does not distinguish well between model combination methods, as the values are close. Third, the performances of different model combination methods can differ drastically. We propose using a new model combination method, named ARM-Tweedie, for such LFHS data; it benefits from an optimal rate of convergence and exhibits a desirable performance in several measures for the Kangaroo data. Fourth, overall, model combination methods improve the prediction accuracy for auto insurance claim costs. In particular, Adaptive Regression by Mixing (ARM), ARM-Tweedie, and constrained Linear Regression can improve forecast performance when there are only weak learners or when no dominant learner exists.

How macro-variables drive crude oil volatility? Perspective from the STL-based iterated combination method

Based on the GARCH-MIDAS framework, this article mainly explores whether the Seasonal and Trend decomposition using Loess (STL decomposition) and the iterated combination approach can improve the prediction accuracy of crude oil price volatility by using macro variables. We introduce five macro variables, that is, consumer price index, interest rate, producer price index, industrial production index, and unemployment rate, into the long-term component in the GARCH-MIDAS model. Though the model including raw macro variables performs better than the model containing each STL-based single subsequence, we find that the iterated combination forecasts using the forecasts obtained by each subsequence are superior to the corresponding standard ones. Besides, when mean-variance investors perform asset allocation, they can obtain more certainty-equivalent-returns by applying our new iterated combination approach. Finally, various robustness checks can verify the above-mentioned findings. In short, this article may provide a new perspective for economic empirical applications and theoretical research.

Estimation of Metal Oxide Surge Arresters' Useful Life Based on Time Series Forecasts

Estimating equipment useful life has been a challenge over the years. In this work, a methodology based on the application of a combined time series forecast model composed of the adaptive neuro-fuzzy inference system (ANFIS) and support vector machine for regression (SVR) is proposed to be used in the determination of metal oxide surge arresters’ (MOSAs’) useful life. For this purpose, field measurements were performed in order to build a time series database composed of values of the leakage current third harmonic component, considering that this component is one of the indicators of the MOSAs’ degradation level most used in arresters’ monitoring. Subsequently, the forecasting models (ANFIS and SVR) were implemented. Then a combined model was proposed using the obtained results of those models. The performance of each implemented models was evaluated using three types of errors: MSE, MAE, and MAPE. After of the models’ evaluation, the useful life estimation of the MOSA was carried out using the proposed model. Considering the obtained results, the forecasts using the combined model were more accurate than those provided by the models based on ANFIS and SVR. Thus, the proposed model was used to estimate the MOSA's useful life.

Interval forecasts of weekly incident and cumulative COVID-19 mortality in the United States: A comparison of combining methods.

A combined forecast from multiple models is typically more accurate than an individual forecast, but there are few examples of studies of combining in infectious disease forecasting. We investigated the accuracy of different ways of combining interval forecasts of weekly incident and cumulative coronavirus disease-2019 (COVID-19) mortality. We considered weekly interval forecasts, for 1- to 4-week prediction horizons, with out-of-sample periods of approximately 18 months ending on 8 January 2022, for multiple locations in the United States, using data from the COVID-19 Forecast Hub. Our comparison involved simple and more complex combining methods, including methods that involve trimming outliers or performance-based weights. Prediction accuracy was evaluated using interval scores, weighted interval scores, skill scores, ranks, and reliability diagrams. The weighted inverse score and median combining methods performed best for forecasts of incident deaths. Overall, the leading inverse score method was 12% better than the mean benchmark method in forecasting the 95% interval and, considering all interval forecasts, the median was 7% better than the mean. Overall, the median was the most accurate method for forecasts of cumulative deaths. Compared to the mean, the median's accuracy was 65% better in forecasting the 95% interval, and 43% better considering all interval forecasts. For all combining methods except the median, combining forecasts from only compartmental models produced better forecasts than combining forecasts from all models. Combining forecasts can improve the contribution of probabilistic forecasting to health policy decision making during epidemics. The relative performance of combining methods depends on the extent of outliers and the type of models in the combination. The median combination has the advantage of being robust to outlying forecasts. Our results support the Hub's use of the median and we recommend further investigation into the use of weighted methods.

Empirically-transformed linear opinion pools

The linear opinion pool (LOP) produces potentially non-Gaussian combination forecast densities. In this paper, we propose a computationally convenient transformation for the LOP to mirror the non-Gaussianity exhibited by the target variable. Our methodology involves a Smirnov transform to reshape the LOP combination forecasts using the empirical cumulative distribution function. We illustrate our empirically transformed opinion pool (EtLOP) approach with an application examining quarterly real-time forecasts for U.S. inflation evaluated on a sample from 1990:1 to 2020:2. EtLOP improves performance by approximately 10% to 30% in terms of the continuous ranked probability score across forecasting horizons.

An accurate and fully-automated ensemble model for weekly time series forecasting

Many businesses and industries require accurate forecasts for weekly time series nowadays. However, the forecasting literature does not currently provide easy-to-use, automatic, reproducible and accurate approaches dedicated to this task. We propose a forecasting method in this domain to fill this gap, leveraging state-of-the-art forecasting techniques, such as forecast combination, meta-learning, and global modelling. We consider different meta-learning architectures, algorithms, and base model pools. Based on all considered model variants, we propose to use a stacking approach with lasso regression which optimally combines the forecasts of four base models: a global Recurrent Neural Network (RNN) model, Theta, Trigonometric Box–Cox ARMA Trend Seasonal (TBATS), and Dynamic Harmonic Regression ARIMA (DHR-ARIMA), as it shows the overall best performance across seven experimental weekly datasets on four evaluation metrics. Our proposed method also consistently outperforms a set of benchmarks and state-of-the-art weekly forecasting models by a considerable margin with statistical significance. Our method can produce the most accurate forecasts, in terms of mean sMAPE, for the M4 weekly dataset among all benchmarks and all original competition participants.

Shrinking return forecasts

AbstractWe develop a new approach that shrinks a given model forecast to the benchmark model forecast in order to improve forecasting performance. Simulation results show the superior performance of our approach, relative to popular methods such as forecast combination and the robustness to model misspecification. We apply our method to forecasting the returns on the S&P 500 index and find significant predictability when shrinking the principal component (PC) regression forecasts based on statistical and economic evaluation criteria. The forecast improvement from our shrinkage approach can be explained by the ability of its hyperparameters to be better predict real economic changes.

Forecasting crude oil volatility with uncertainty indicators: New evidence

This paper uses multiple uncertainty indicators to forecast monthly WTI crude oil volatility and compare the predictive performance of combination forecast methods, dimension reduction techniques and two least absolute shrinkage and selection operator augmented MIDAS (MIDAS-LASSO and MS-MIDAS-LASSO) models. Some noteworthy findings are observed by using the MIDAS-RV extensions. First, among all uncertainty indicators, the U.S. petroleum market equity market volatility tracker index (PMEMV) statistically has the best short-term predictive power for the volatility of crude oil market, especially during low volatility, non-crisis and economic expansion periods. However, the geopolitical risk index (GPR) performs better in predicting long-term crude oil volatility than other uncertainty indicators, and it also performs better than other uncertainty indicators in forecasting short-term high volatility of crude oil market. In addition, the financial stress index (FSI) has better predictive ability than other uncertainty indicators during periods of crisis and economic recession. Finally, the newly constructed MS-MIDAS-LASSO and MIDAS-LASSO models always have much higher forecasting accuracy than combination forecast methods, dimension reduction techniques as well as the best MIDAS-RV-X models, with MS-MIDAS-LASSO model having greater forecasting accuracy than the MIDAS-LASSO model in most cases. This empirical finding is confirmed by a variety of robustness checks.

Combination forecasting using multiple attribute decision making in tourism demand

Pronóstico combinado utilizando la toma de decisiones de atributos múltiples en la demanda turísticaPropósitoLa combinación de pronósticos en este estudio abordó tres cuestiones importantes para la situación del turismo: Reducir las restricciones que surgen con respecto a las estadísticas de datos disponibles, evaluar los pesos con un solo pronóstico, y considerar las relaciones no lineales entre las combinaciones con un único modelo de pronóstico.Diseño/metodología/enfoqueSe propuso un marco metodológico de tres etapas basado en MADM. Un solo pronóstico fue generado mediante modelos de predicción grises para la primera etapa. Se aplicó VIKOR para desarrollar un esquema de ponderación en la segunda etapa, y la integral de Choquet se usó para combinar los pronósticos de manera no lineal en la tercera etapa.RecomendacionesLos resultados empíricos de la demanda turística entrante en Taiwán mostraron que el método propuesto puede mejorar efectivamente la precisión en mayor medida que otros métodos combinados. Una buena práctica del pronóstico puede proporcionar aún más, mediante las previsiones y la estimación exante de pronósticos posteriores al COVID-19.Implicaciones practicesLos sectores públicos y privados de las economías con alta dependencia del turismo pueden beneficiarse del método propuesto al usar los pronósticos para ayudarlos a formular estrategias turísticas.Autenticidad/valorEste estudio contribuye a presentar un marco basado en MADM que avanza en el desarrollo de un método de combinación más preciso para la previsión del turismo.

Development and Evaluation of Global Korean Aviation Turbulence Forecast Systems Based on an Operational Numerical Weather Prediction Model and In Situ Flight Turbulence Observation Data

Abstract A global Korean deterministic aviation turbulence guidance (G-KTG) system and a global Korean probabilistic turbulence forecast (G-KPT) system are developed using outputs from the operational Global Data Assimilation and Prediction System of the Korea Meteorological Administration, and the performance skill of the systems are evaluated against in situ flight eddy dissipation rates (EDRs) recorded for one year (September 2018–August 2019). G-KTG and G-KPT consider clear-air turbulence (CAT) and mountain wave turbulence diagnostics, while G-KTG additionally considers near-cloud turbulence (NCT) diagnostics. In the G-KTG system, the various combinations of deterministic EDR forecasts are tested by different ensemble means of individual turbulence diagnostics. In the G-KPT system, the probabilistic forecast is established by counting the number of diagnostics that exceed a certain threshold for strong intensity turbulence on the given model grid. The evaluation results of the G-KTG system based on the area under the relative operating characteristic curve (AUC) reveal that G-KTG, which consists of CAT and NCT diagnostics, shows the highest AUC value among the various G-KTG combinations; in addition, the summertime performance is significantly improved when NCT diagnostics are included. In the evaluation results of the G-KTG system over the globe, U.S., and East Asia regions, the recent graphical turbulence guidance system–based G-KTG shows better performance than the regional KTG–based G-KTG for all three regions. For all altitude bands, the G-KPTs with 40% probability as the minimal threshold for alerting forecasters of strong turbulence show higher values of true skill statistic than the G-KTGs.

Forecasting crude oil futures market returns: A principal component analysis combination approach

To improve the predictability of crude oil futures market returns, this paper proposes a new combination approach based on principal component analysis (PCA). The PCA combination approach combines individual forecasts given by all PCA subset regression models that use all potential predictor subsets to construct PCA indexes. The proposed method can not only guard against over-fitting by employing the PCA technique but also reduce forecast variance due to extensive forecast combinations, thus benefiting from both the combination of information and the combination of forecasts. Showing impressive out-of-sample forecasting performance, the PCA combination approach outperforms a benchmark model and many related competing models. Furthermore, a mean–variance investor can realize sizeable utility gains by using the PCA combination forecasts relative to the competing forecasts from an asset allocation perspective.

Predictability of bull and bear markets: A new look at forecasting stock market regimes (and returns) in the US

The empirical literature of stock market predictability mainly suffers from model uncertainty and parameter instability. To meet this challenge, we propose a novel approach that combines dimensionality reduction, regime-switching models, and forecast combination to predict excess returns on the S&P 500. First, we aggregate the weekly information of 146 popular macroeconomic and financial variables using different principal component analysis techniques. Second, we estimate Markov-switching models with time-varying transition probabilities using the principal components as predictors. Third, we pool the models in forecast clusters to hedge against model risk and to evaluate the usefulness of different specifications. Our weekly forecasts respond to regime changes in a timely manner to participate in recoveries or to prevent losses. This is also reflected in an improvement of risk-adjusted performance measures as compared to several benchmarks. However, when considering stock market returns, our forecasts do not outperform common benchmarks. Nevertheless, they do add statistical and, in particular, economic value during recessions or in declining markets.

Forecasts of growth in US durables spending: assessing the usefulness of disaggregated consumer survey data

This study aims to improve the accuracy of the Federal Reserve forecasts of growth in durables spending using disaggregated consumer survey data. Test results for 1988–2016 indicate that these forecasts do (do not) contain past information in consumer durables-buying (home-buying) attitudes of 35–54-year-old participants, participants with a college degree, male participants, and participants with the top 33% income. Using real-time data on durables spending and information in consumer home-buying attitudes and expectations, we construct a knowledge model (KM) to generate comparable forecasts of growth in durables spending. Our results indicate that the one- and four-quarter-ahead KM forecasts can potentially help improve the accuracy of Federal Reserve forecasts. Further results indicate that the one- and four-quarter-ahead combined Federal Reserve and KM forecasts show significant reductions in forecast errors, meaning that there are accuracy gains from using disaggregated consumer survey data. The practical implication is that forecasters should pay special attention to consumer home-buying attitudes and expectations about future business conditions, and policymakers should make use of such survey measures in monitoring the economy in real time.