Model Confidence Set Research Articles

CNN-CBAM-LSTM: Enhancing Stock Return Prediction Through Long and Short Information Mining in Stock Prediction

Deep learning, a foundational technology in artificial intelligence, facilitates the identification of complex associations between stock prices and various influential factors through comprehensive data analysis. Stock price data exhibits unique time-series characteristics; models emphasizing long-term data may miss short-term fluctuations, while those focusing solely on short-term data may not capture cyclical trends. Existing models that integrate long short-term memory (LSTM) and convolutional neural networks (CNNs) face limitations in capturing both long- and short-term dependencies due to LSTM’s gated transmission mechanism and CNNs’ limited receptive field. This study introduces an innovative deep learning model, CNN-CBAM-LSTM, which integrates the convolutional block attention module (CBAM) to enhance the extraction of both long- and short-term features. The model’s performance is assessed using the Australian Standard & Poor’s 200 Index (AS51), showing improvement over traditional models across metrics such as RMSE, MAE, R2, and RETURN. To further confirm its robustness and generalizability, Diebold–Mariano (DM) tests and model confidence set experiments are conducted, with results indicating the consistently high performance of the CNN-CBAM-LSTM model. Additional tests on six globally recognized stock indices reinforce the model’s predictive strength and adaptability, establishing it as a reliable tool for forecasting in the stock market.

Time‐Varying US Government Spending Anticipation in Real Time

ABSTRACTDue to legislation and implementation lags, forward‐looking economic agents anticipate changes in fiscal policy variables before they actually occur. The literature shows that this foresight poses a challenge to the econometric analysis of fiscal policies. While most of the literature uses fully revised data to investigate the degree of fiscal foresight, we use forecasts from the Survey of Professional Forecasters (SPF), the Greenbook/Tealbook from the Federal Reserve, and the Real‐Time Data Set for Macroeconomists. Furthermore, we distinguish between federal as well as state and local consumption & investment expenditures. We find that real‐time data matter. Using the first release, the SPF nowcast was able to predict 43% of the out‐of‐sample fluctuation in federal government spending growth (only 24% using the most recent release). Moreover, the SPF was able to predict 60% and 52% of the cumulated growth in federal and state & local government spending growth over a 1‐year horizon. We use the Diebold–Mariano tests and model confidence sets to investigate whether SPF forecasts significantly outperform the Greenbook projections and forecasts from purely backward‐looking time series models. Compared to the SPF and Greenbook projections, the time series models perform inferior at most forecast horizons. In addition, so‐called information advantage regressions reveal that most forecasts could be improved by using the information of the SPF. Using rolling windows, we document remarkable time‐variation in the degree of fiscal foresight of the SPF and its information advantage against (augmented) autoregressive models and the Greenbook. Particularly during the 1980s and 2000s, we find a strong degree of anticipation for government spending at the federal level by the SPF and the central bank.

The Conditional Autoregressive F-Riesz Model for Realized Covariance Matrices

Abstract We introduce a new model for the dynamics of fat-tailed (realized) covariance-matrix-valued time-series using the F-Riesz distribution. The model allows for heterogeneous tail behavior across the coordinates of the covariance matrix via two vector-valued degrees of freedom parameters, thus generalizing the familiar Wishart and matrix-F distributions. We show that the filter implied by the new model is invertible and that a two-step targeted maximum likelihood estimator is consistent. Applying the new F-Riesz model to U.S. stocks, both tail heterogeneity and tail fatness turn out to be empirically relevant: they produce significant in-sample and out-of-sample likelihood increases, ex-post portfolio standard deviations that are in the global minimum variance model confidence set, and economic differences that are either in favor of the new model or competitive with a range of benchmark models.

Enhanced Foreign Exchange Volatility Forecasting Using Ceemdan with Optuna-Optimized Ensemble Deep Learning Model

Foreign Exchange (FX) is the largest financial market in the world, with a daily trading volume that significantly exceeds that of stock and futures markets. The prediction of FX volatility is a critical financial concern that has garnered significant attention from researchers and practitioners due to its far-reaching implications in the financial markets. This paper presents a novel hybrid ensemble forecasting model integrating a decomposition strategy and three deep learning (DL) models: Long Short-Term Memory (LSTM), Bidirectional LSTM (BiLSTM), and Convolutional Neural Network (CNN). This combination addresses individual models' limitations and further improves the accuracy and stability of FX volatility forecasting. The proposed approach utilizes the CEEMDAN technique to decompose volatility into multiple distinct intrinsic mode functions (IMFs) and merges these IMFs with GARCH and EGARCH volatilities to form the input dataset for the DL models. In addition, we employed an attention mechanism to improve the effectiveness of the DL techniques. Furthermore, the hyperparameters for the DL models are optimized using the Optuna algorithm. Finally, a hybrid ensemble model for forecasting exchange rate volatility is developed by combining the predictions of three distinct DL models. The proposed approach is evaluated against various benchmark models using evaluation measures such as MSE, MAE, HMSE, HMAE, RMSE, Q-LIKE, and the model confidence set (MCS) approach. The results demonstrate that our proposed approach provides accurate and reliable forecasts of FX volatility under different forecasting regimes, making it a valuable tool for financial practitioners and researchers.

A GARCH model selection and estimation method based on neural network with the loss function of mean square error and model confidence set

AbstractThis paper proposes a method that uses mean square error (MSE) and model confidence set (MCS) as the loss function of back‐propagation neural network (BPNN), aiming to train and find a generalized autoregressive conditional heteroskedastic (GARCH) model that has the best forecasting performance of a time series. Combining MSE and the p‐value of MCS can not only estimate better parameters for the GARCH models but also find the best GARCH model to forecast the volatility of a time series. Meanwhile, we divide a time series into several parts and use each part as the input of the BPNN. Through the BPNN, each part of the time series will be turned into several forecasting values. These values will be used to calculate the MSE and the p‐value of MCS, which will then be used to update the parameters of the BPNN. In the end, we use MCS to choose the best GARCH model among the trained GARCH models and compare this method with maximum likelihood estimation (MLE) and the generalized least squares estimation (GLS). The result shows that the p‐value of MCS of the best model estimated by this method is higher than the p‐value of MCS of the best model estimated by MLE and GLS. According to the theory of MCS, a model that has a larger p‐value does have a better forecasting performance. The method proposed by this paper can provide a potential application of neural network in GARCH model forecasting and estimation.

A novel robust method for estimating the covariance matrix of financial returns with applications to risk management

This study introduces the dynamic Gerber model (DGC) and evaluates its performance in the prediction of Value at Risk (VaR) and Expected Shortfall (ES) compared to alternative parametric, non-parametric and semi-parametric methods for estimating the covariance matrix of returns. Based on ES backtests, the DGC method produces, overall, accurate ES forecasts. Furthermore, we use the Model Confidence Set procedure to identify the superior set of models (SSM). For all the portfolios and VaR/ES confidence levels we consider, the DGC is found to belong to the SSM.

Deep Learning Wind Power Prediction Model Based on Attention Mechanism-Based Convolutional Neural Network and Gated Recurrent Unit Neural Network

Accurate prediction of wind power is crucial for the efficient operation and risk management of wind farms. This paper introduces a deep learning model for wind power prediction that integrates an Attention mechanism with a convolutional neural network (CNN) and a gated recurrent unit (GRU) neural network. Addressing the randomness, intermittency, volatility and uncertainty of wind speed, we first apply swarm decomposition (SWD) to preprocess the original wind power data into subsequences. Subsequently, the CNN extracts spatial features, and the GRU identifies temporal correlations. The Attention mechanism enhances feature significance, further optimizing prediction accuracy. Complex error sequences generated by the CNN–GRU–Attention (CGA) model are corrected using the autoregressive integrated moving average (ARIMA). We evaluated the model’s performance using three wind power datasets against 16 other models, employing six evaluation indices (MSE, RMSE, MAPE, Theil’s [Formula: see text], TIC and SPL) and the Diebold–Mariano (DM) test and model confidence set (MCS) for model testing. Our results demonstrate the proposed model’s superior accuracy and efficiency in predicting wind power.

Improving millimetre-wave path loss estimation using automated hyperparameter-tuned stacking ensemble regression machine learning

Path loss prediction is a crucial aspect of designing and operating wireless communication systems, especially in the millimetre-waves (mmWaves) frequency bands. However, these bands are associated with climate-related challenges: rain attenuation, and free space path loss. To address these challenges, an advanced stacking ensemble-regression machine learning (SEML) model with automated hyperparameter tuning (AHT) was proposed. The AHT-SEML model leverages multiple base regressors integrated with a meta-regressor. The model's performance was optimised using the AHT tuning technique. The AHT-SEML model's efficiency was tested using simulated path loss data from a Composite 3D Raytracing-Image-Method propagation model across four sub-Saharan cities, at mmWaves frequencies. The AHT-SEML model's performance was compared to three empirical path loss models, namely Close-In (CI), Floating Intercept (FI), and Alpha-Beta-Gamma (ABG), using evaluation metrics such as Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE). AHT-SEML outperformed other models in the four cities across all frequencies and scenarios with the highest Index of Agreement and lowest Bayesian information criterion. Model confidence set (MCS) analysis with CI benchmark indicates that all the models except AHT-SEML performed below the critical t-value of 2.3530 at 95% confidence level with a degree of freedom of 3, implying no significant differences in their MAEs compared to the CI. However, AHT-SEML's t-statistic values exceed this critical t-value, indicating statistically significant differences and better performance than the CI benchmark models. Similarly, F-statistics of 29.45 and 26.54 correspond to p-values of 1.91×10−14 and 2.50×10−13 for MAE and RMSE, respectively, corroborating significant differences in the AHT-SEML's performance.

Oil price volatility prediction using out-of-sample analysis – Prediction efficiency of individual models, combination methods, and machine learning based shrinkage methods

In this study, we compare the efficacy of forecast combination methods against machine learning-based shrinkage techniques in predicting oil price volatility. Our analysis is based on heterogeneous autoregressive (HAR) model framework. We employ eight individual HAR models and their variations, alongside five distinct combination methods for aggregating forecasts derived from HAR models and their variants. Additionally, we incorporate two widely recognized machine learning-based shrinkage methods, namely the elastic net and the lasso. Machine learning (ML) techniques, including elastic net and lasso, exhibit promise in estimating individual extended HAR models and combination sampled approaches. Meanwhile, model confidence set (MCS) estimation techniques demonstrate notably superior out-of-sample forecasting performance for the chosen sample. Our empirical findings reveal that both the elastic net and the lasso exhibit superior out-of-sample prediction accuracy in comparison to the individual HAR models and their variants, as well as the five combination techniques. Furthermore, we provide statistical evidence demonstrating the notably higher directional accuracy achieved by the elastic net and lasso methodologies. Importantly, our results remain statistically consistent across a range of robustness analyses. These findings hold significance for investors and policymakers, as they suggest potential economic benefits derived from allocating portfolios in alignment with oil price volatility estimates.

Hybrid forecasting of crude oil volatility index: The cross‐market effects of stock market jumps

AbstractFrom the cross‐market perspective, this paper investigates crude oil volatility index (OVX) forecasts by proposing a hybrid method, which combines the data‐driven SVR technique and parametric models. In terms of parametric models, we utilize GARCH‐type models with jumps, and the forecasting effects of five non‐parametric jumps (including interday and intraday jump tests) of stock market are also explored. Empirical results show that our approach can substantially increase forecasting accuracy. In addition, the model confidence set test and robust test reaffirm the superiority of the novel hybrid method. From the assessment of economic significance, the advantages of the hybrid method for volatility index forecasting are further confirmed. All these findings imply that jumps of stock market can be helpful in forecasting OVX, especially after the introduction of the hybrid method. Our work can certainly provide a new insight for volatility forecasting and cross‐market research.

Multivariate realized volatility: an analysis via shrinkage methods for Brazilian market data

IntroductionRealized volatility analysis of assets in the Brazilian market within a multivariate framework is the focus of this study. Despite the success of volatility models in univariate scenarios, challenges arise due to increasing dimensionality of covariance matrices and lower asset liquidity in emerging markets.MethodsIn this study, we utilize intraday stock trading data from the Brazilian Market to compute daily covariance matrices using various specifications. To mitigate dimensionality issues in covariance matrix estimation, we implement penalization restrictions on coefficients through regressions with shrinkage techniques using Ridge, LASSO, or Elastic Net estimators. These techniques are employed to capture the dynamics of covariance matrices.ResultsComparison of covariance construction models is performed using the Model Confidence Set (MCS) algorithm, which selects the best models based on their predictive performance. The findings indicate that the method used for estimating the covariance matrix significantly impacts the selection of the best models. Additionally, it is observed that more liquid sectors demonstrate greater intra-sectoral dynamics.DiscussionWhile the results benefit from shrinkage techniques, the high correlation between assets presents challenges in capturing stock or sector idiosyncrasies. This suggests the need for further exploration and refinement of methods to better capture the complexities of volatility dynamics in emerging markets like Brazil.

Riemannian‐geometric regime‐switching covariance hedging

AbstractThis study develops a regime‐switching Riemannian‐geometric covariance framework for futures hedging. The covariance of conventional regime‐switching BEKK (Baba, Engle, Kraft and Kroner) (RSBEKK) evolves on flat spaces that exclude a prior the possibility of inherent geometric covariance dynamic. A Riemannian‐geometric regime‐switching BEKK (RG‐RSBEKK) is proposed such that the covariance moves along a trajectory on Riemannian manifolds. RG‐RSBEKK is applied to China Securities Index 300 futures for hedging the stock sector exposures. Empirical results reveal that specifying covariance dynamic on curved spaces enhances hedging effectiveness based on the model confidence set with loss measures of variance, utility, value‐at‐risk, and Frobenius distance.

Forecasting the volatility of crude oil basis: Univariate models versus multivariate models

The studies on investigating and forecasting crude oil volatility primarily focus on oil price volatility. This research aims to forecast the volatility of the oil futures basis using generalized auto regressive conditional heteroskedasticity (GARCH) and its extended models. Specifically, this study compares the forecasting performance of nine econometric models, including two univariate and seven multivariate models. The empirical results show that simple univariate models outperform the multivariate models in forecasting basis volatility, as indicated by the model confidence set (MCS). Moreover, from an economic perspective, the empirical results also demonstrate that the univariate models generate higher Sharpe ratios than the multivariate ones, consistent with the statistical evaluation results on forecasting accuracy.

Forecasting volatility of stock indices: Improved GARCH-type models through combined weighted volatility measure and weighted volatility indicators

This paper proposes an unbiased combined weighted (CW) volatility measure and weighted volatility indicators (WVI) that integrates the return- and range-based volatility measures to model the dynamics volatility of stock returns. The main feature of the CW measure is that it is formulated based on the weighted inter- and intra-price information to quantify the volatility directly, while the WVI effectively identifies signals on the shift of volatility. Empirical analysis using five stock indices demonstrates that the CW measure, utilising squared returns in combination with range-based Garman-Klass volatility measure, exhibits the lowest losses based on root mean squared error and quasi-likelihood when compared to 5-minute realised volatility as a proxy for true volatility. Furthermore, we investigate the feasibility of incorporating the CW measure and WVI as the exogenous variable(s) in the generalised autoregressive conditional heteroscedasticity (GARCH)-type models to enhance the forecasting performance. The findings indicate that the GARCH-CW-WVI and EGARCH-CW-WVI models exhibit superior in-sample model fit based on the Akaike information criterion than the existing GARCH and EGARCH models. Moreover, our proposed models also offer the best out-of-sample forecasts evaluated using various loss functions and further tested using Hansen’s model confidence set based on the mean squared error loss. Different risk levels of value-at-risk (VaR) and expected shortfall (ES) forecasts based on GARCH-CW-WVI and EGARCH-CW-WVI models are computed and examined with various backtests to confirm the accuracies of VaR and ES forecasts.

Model Selection Path and Construction of Model Confidence Set under High-Dimensional Variables

Model selection uncertainty has drawn a lot of attention from academics recently because it significantly affects parameter estimation and prediction. Scholars are currently addressing and quantifying uncertainty in model selection by concentrating on model combining and model confidence sets. In this paper, we present a new approach for building model confidence sets, which we call AMac. We provide a theoretical lower bound on the degree of confidence in the model confidence sets that AMac has built. Furthermore, we discuss how the implementation of current model confidence set construction methods becomes difficult when dealing with high-dimensional variables. To address this problem, we suggest building model selection paths (MSP) as a solution. We develop an algorithm for building MSP and show its effectiveness by utilizing the theories of adaptive lasso and lars. We perform an extensive set of simulation experiments to compare the performances of Mac and AMac methods. According to the results, AMac is more stable when there are fluctuations in noise levels. The model confidence sets built by AMac, in particular, achieve coverage rates that are closer to the desired confidence level, especially in the presence of high noise levels. To further confirm that MSP can successfully generate model confidence sets that maintain the given confidence level as the sample size increases, we conduct extensive simulation tests with high-dimensional variables. Ultimately, we hope that the strategies and concepts discussed in this work will improve results in subsequent research on the uncertainty of model selection.

A hybrid forecasting framework based on MCS and machine learning for higher dimensional and unbalanced systems

Forecasting methods and theories have been widely researched and applied in complex systems and fields such as statistical physics, econophysics, material crystals, etc. However, challenges persist in applying these methods to complex systems characterized by high dimensionality, data imbalance, and single prediction evaluation. To address these issues, we propose a novel hybrid forecasting approach that integrates the model confidence set (MCS) with machine learning (ML) models. We introduce Principal Component Analysis (PCA) to reduce dimensionality of the data, reduce data imbalance through a combination of random undersampling and oversampling, and introduce several metrics to evaluate the machine learning model set. We also introduce the MCS to select the optimal model from the set of ML models and propose a new combinatorial approach, the MCS-ML combinatorial model. An empirical study is conducted using the example of abnormal transactions in the Bitcoin blockchain. The empirical results show that the proposed MCS-ML combinatorial model has better predictive performance than the models in the ML model set under different data structures.

Crude oil volatility index forecasting: New evidence based on positive and negative jumps from Chinese stock market

This article investigates the crude oil volatility index (OVX) forecasting from the perspective of cross-market asymmetric effects of Chinese stock market jumps. We calculate six kinds of positive and negative jumps based on the high-frequency data of stock returns which are used to represent the asymmetric shocks of stock markets. Principal component analysis (PCA) and momentum of predictability (MoP) strategy are employed separately to synthesize the information of asymmetric jumps. Our empirical results find that considering the positive and negative jumps in Chinese stock market helps to improve the forecasting ability of OVX, especially under the MoP strategy. The out-of-sample model confidence set (MCS) tests and Diebold-Mariano (DM) tests, the evaluation of economic significance and the robustness tests further verify our results.

Forecasting GDP growth: The economic impact of COVID‐19 pandemic

AbstractThe primary goal of this study is to effectively measure the impact of a severe random shock, such as the COVID‐19 pandemic on aggregate economic activity in Greece, seven other euro area economies, three Scandinavian countries, and the United States. The class of linear and quantile predictive regression models is proposed for the analysis of real gross domestic product (GDP) growth, and a Bayesian approach for model selection is developed, by using a computationally flexible Markov chain Monte Carlo stochastic search algorithm that explores the posterior distribution of linear and quantile models, and identifies the relevant predictor variables. Penalized likelihood regression models are also implemented to tackle the issue of model selection. The model confidence set approach is applied and verifies that the selected models identified by the stochastic search algorithm belong to the set of superior models. Our analysis confirms that the outbreak of the pandemic had a profound effect on the economies under study, and reveals that different predictor variables are able to explain different quantiles of the underlying real GDP growth distribution for analyzed countries, suggesting that the quantile modeling approach improves the ability to adequately explain real GDP series compared with the standard conditional mean approach that explains only the average of the relationship between real GDP growth and several predictor variables.

The short-term predictability of returns in order book markets: A deep learning perspective

This paper uses deep learning techniques to conduct a systematic large-scale analysis of order book-driven predictability in high-frequency returns. First, we introduce a new and robust representation of the order book, the volume representation. Next, we conduct an extensive empirical experiment to address various questions regarding predictability. We investigate if and how far ahead there is predictability, the importance of a robust data representation, the advantages of multi-horizon modeling, and the presence of universal trading patterns. We use model confidence sets, which provide a formalized statistical inference framework well suited to answer these questions. Our findings show that at high frequencies, predictability in mid-price returns is not just present but ubiquitous. The performance of the deep learning models is strongly dependent on the choice of order book representation, and in this respect, the volume representation appears to have multiple practical advantages.

Climate Change and Volatility Forecasting: Novel Insights from Sectoral Indices

This study investigates the impact of climate policy uncertainty (CPU) on sectoral indices and clean energy exchange-traded funds (ETFs) by using a GARCH-MIDAS model. The stock indices include renewable energy (NEX), transportation, mining, industrial, real estate, green economy US, green economy Europe, green economy Asia index, and ETFs clean energy (PBW), global clean energy (PBD). The research also evaluates the forecasting power of uncertainty factors, including equity market volatility (EMV), economy policy uncertainty (EPU), trade policy uncertainty (TPU), fiscal policy uncertainty (FPU), global economy policy uncertainty (GEPU), and geopolitical risk (GPR), to predict the volatility. We obtained some notable results. First, the out-of-sample findings show that CPU index information is useful to predict the volatility of the NEX renewable energy, green economy US, transportation, energy index for the US, and clean energy (PBW), global clean energy (PBD) ETFs. Second, EMV, EPU, and GPR also contain valuable information for the real estate, industrial, energy, and NEX renewable energy index. Additionally, we find evidence during low and high volatility and upheaval of the COVID-19 pandemic. The Roos2 square and model confidence set (MCS) tests verify each model's out-of-sample forecasting performance.