Superior Predictive Ability Test Research Articles

The out-of-sample performance of carry trades

We carry out a large-scale investigation of the reliability of the profitability of carry trade strategies, using foreign exchange data for 48 countries over 36 years, employing reality check, superior predictive ability test, and stepwise tests to correct for data-snooping bias (the factor of luck in model selection). Carry trade strategies chosen as profitable in one period are generally not profitable in an ensuing out-of-sample sample period, especially after correcting for data-snooping and even after allowing for learning and stop-loss strategies. Any evidence of consistency in carry trade profitability that is found is concentrated in a relatively brief historical period, 1998-2005. We further investigate particular currency pairs that may drive the out-of-sample profitability during this period, and find their performance to be unstable in general. Our findings thus highlight the instability and the importance of chance or luck in generating profits from carry trades.

The roles of liquidity and delay in financial markets based on an optimal forecasting model.

We investigate the roles of liquidity and delay in financial markets through our proposed optimal forecasting model. The efficiency and liquidity of the financial market are examined using stochastic models that incorporate information delay. Based on machine learning, we estimate the in-sample and out-of-sample forecasting price performances of the six proposed methods using the likelihood function and Bayesian methods, and the out-of-sample prediction performance is compared with the benchmark model ARIMA-GARCH. We discover that the forecasting price performance of the proposed simplified delay stochastic model is superior to that of the benchmark methods by the test methods of a variety of loss function, superior predictive ability test (SPA), Akaike information criterion (AIC), and Bayesian information criterion (BIC). Using data from the Chinese stock market, the best forecasting model assesses the efficiency and liquidity of the financial market while accounting for information delay and trade probability. The rise in trade probability and delay time affects the stability of the return distribution and raises the risk, according to stochastic simulation. The empirical findings show that empirical and best forecasting approaches are compatible, that company size and liquidity (delay time) have an inverse relationship, and that delay time and liquidity have a nonlinear relationship. The most efficient have optimal liquidity.

The predictive ability of technical trading rules: an empirical analysis of developed and emerging equity markets

We investigate the predictability of leading equity indices of 23 developed and 18 emerging markets with a set of 6406 technical trading rules over up to 66 years. Using a state-of-the-art test for superior predictive ability to control for data snooping bias, we find in-sample evidence for technical heuristics with significant outperformance over a simple buy-and-hold strategy in the majority of markets. The proportion of heuristics with superior performance is much higher among emerging market indices, and the predictability diminishes drastically over time in all markets. In particular, markets turn unpredictable in the last years of our sample. Moreover, the results are very sensitive to the introduction of moderate transaction costs. An out-of-sample analysis shows that the performance of technical rules is not persistent in the sense that recently best-performing rules perform significantly worse than simple buy-and-hold strategies in the future. Overall, our results cast serious doubt on whether investors could have earned any excess profits using the broad range of considered technical trading rules.

Cryptocurrencies and Lucky Factors: The value of technical and fundamental analysis

AbstractThis study explores the effectiveness of technical and fundamental analysis in predicting and trading the returns of 12 cryptocurrencies, namely Bitcoin, Ethereum, Ripple, Dash, Cardano, Avalanche, Binance Coin, Dogecoin, Polkadot, Litecoin, Terra and Solana. A universe of 7846 technical rules, five log moving average‐based ratios and 59 fundamental factors are used to test predictability and profitability through the Lucky Factors methodology and Superior Predictive Ability test. We observe predictability for a small set of technical and fundamental rules, while only the short‐term log moving average‐based ratio and Hashrate Index demonstrate genuine in‐sample and out‐of‐sample profitability. Our findings question the value of both technical and fundamental analysis on cryptocurrencies.

ON THE SIZE CONTROL OF THE HYBRID TEST FOR SUPERIOR PREDICTIVE ABILITY

This article analyzes the theoretical properties of the hybrid test for superior predictive ability. A simple example reveals that the test may not be size-controlled at common significance levels with rejection rates exceeding $11\%$ at a $5\%$ nominal level. Generalizing this observation, the main results show the pointwise asymptotic invalidity of the hybrid test under reasonable conditions. Monte Carlo simulations support these theoretical findings.

Conditional evaluation of predictive models: The cspa command

In this article, we introduce a new command, cspa, that implements the conditional superior predictive ability test developed in Li, Liao, and Quaedvlieg (2022, Review of Economic Studies 89: 843–875). With the conditional performance of predictive methods measured nonparametrically by the conditional expectation functions of their predictive losses, we test the null hypothesis that a benchmark model weakly outperforms a collection of competitors uniformly across the conditioning space. The proposed command can implement this test for both independent cross-sectional data and serially dependent time-series data. Confidence sets for the most superior model can be obtained by inverting the test, for which the cspa command also offers a convenient implementation.

Forecasting Industrial Production Using Its Aggregated and Disaggregated Series or a Combination of Both: Evidence from One Emerging Market Economy

In this paper, we address whether using a disaggregated series or combining an aggregated and disaggregated series improves the forecasting of the aggregated series compared to using the aggregated series alone. We used econometric techniques, such as the weighted lag adaptive least absolute shrinkage and selection operator, and Exponential Triple Smoothing (ETS), as well as the Autometrics algorithm to forecast industrial production in Brazil one to twelve months ahead. This is the novelty of the work, as is the use of the average multi-horizon Superior Predictive Ability (aSPA) and uniform multi-horizon Superior Predictive Ability (uSPA) tests, used to select the best forecasting model by combining different horizons. Our sample covers the period from January 2002 to February 2020. The disaggregated ETS has a better forecast performance when forecasting horizons that are more than one month ahead using the mean square error, and the aggregated ETS has better forecasting ability for horizons equal to 1 and 2. The aggregated ETS forecast does not contain information that is useful for forecasting industrial production in Brazil beyond the information already found in the disaggregated ETS forecast between two and twelve months ahead.

Dynamic forecasting performance and liquidity evaluation of financial market by Econophysics and Bayesian methods

In a complex financial system, what is the forecasting performance of macro and micro evolution models of Econophysics on asset prices? For this problem, from the perspective of machine learning, we study the dynamic forecasting and liquidity assessment of financial markets, based on econophysics and Bayesian methods. We establish eight dynamic prediction methods, based on our proposed likelihood estimation and Bayesian estimation methods of macro and micro evolution models of econophysics. Combined machine learning thinking and real data, we empirically study and simulate the out-of-sample dynamic forecasting analysis of eight proposed methods and compare with the benchmark GARCH model. A variety of loss functions, superior predictive ability test (SPA), Akaike and Bayesian information criterion (AIC and BIC) methods are introduced to further evaluate the forecasting performance of our proposed methods. The research of out of sample prediction shows that (1) the method of the simplified stochastic model with Bayesian method for only sample return has the best forecasting performance; (2) the method of the stochastic model with Bayesian method for only return samples has the worst forecasting performance. For the liquidity assessment problem, there is a strong correlation between the trading probability evaluated by the proposed eight methods and the real turnover rate, and an increase of liquidity is correspond to the increase of asset risk. In other words, it suggests that all proposed methods can well evaluate market liquidity.

Autoregressive conditional duration models for high frequency financial data: an empirical study on mid cap exchange traded funds

PurposeThe purpose of this study is to identify appropriate autoregressive conditional duration (ACD) models that can capture the dynamics of tick-by-tick mid-cap exchange traded funds (ETFs) for the period July 2017 to December 2017 and accurately predict future trade duration values. The forecasted durations are then used to demonstrate the practical usefulness of the ACD models in quantifying an intraday time-based risk measure.Design/methodology/approachThrough six functional forms and six error distributions, 36 ACD models are estimated for eight mid-cap ETFs. The Akaike information criterion and Bayesian information criterion and the Ljung-Box test are used to evaluate goodness-of-fit while root mean square error and the Superior predictive ability test are applied to assess forecast accuracy.FindingsThe Box-Cox ACD (BACD), augmented Box-Cox ACD (ABACD) and additive and multiplicative ACD (AMACD) extensions are among the best fits. The results obtained prove that higher degrees of flexibility do not necessarily enhance goodness of fit and forecast accuracy does not always depend on model adequacy. BACD and AMACD models based on the generalised-F distribution generate the best forecasts, irrespective of the trading frequencies of the ETFs.Originality/valueTo the best of the authors’ knowledge, this is the first study that analyses the empirical performance of ACD models for high-frequency ETF data. Additionally, in comparison to previous works, a wider range of ACD models is considered on a reasonably longer sample period. The paper will be of interest to researchers in the area of market microstructure and to practitioners engaged in high-frequency trading.

Short-term volatility forecasting with kernel support vector regression and Markov switching multifractal model

In volatility forecasting literature, Markov switching multifractal (MSM) models are well known for capturing many important stylized facts such as long memory and fat tails. MSM delivers stronger performance both in- and out-of-sample than GARCH-type models in long-term forecasts. However, the literature shows that MSM forecasts only slightly improve on GARCH(1,1) at short-term intervals. This indicates that there may exist certain patterns to be discovered in the innovation part . To enhance MSM's prediction accuracy at the short-term level with higher frequency data, a hybrid model of the MSM model and support vector regression (SVR) is proposed, in which a particle swarm optimization (PSO) algorithm is applied to optimize hyperparameters of the support vector regression in the scope of constraint permission. The method is referred to as MSM-PSO-SVR. Further, we introduce the Fourier kernel MSM-PSO-SVR and evaluate the performance of various MSM-PSO-SVR models in terms of mean absolute error (MAE) and the mean squared error (MSE) with one-minute data of the exchange traded fund (ETF) SPDR S&P 500 Trust ETF (ticker symbol: SPY). The experimental results show that the proposed approach outperforms the other competing peer models and in particular, the selection of SVR kernel might yield significant boosts in forecasting ability. Results of Hansen's Superior Predictive Ability test further validate the conclusion.

Conditional Superior Predictive Ability

Abstract This article proposes a test for the conditional superior predictive ability (CSPA) of a family of forecasting methods with respect to a benchmark. The test is functional in nature: under the null hypothesis, the benchmark’s conditional expected loss is no more than those of the competitors, uniformly across all conditioning states. By inverting the CSPA tests for a set of benchmarks, we obtain confidence sets for the uniformly most superior method. The econometric inference pertains to testing conditional moment inequalities for time series data with general serial dependence, and we justify its asymptotic validity using a uniform non-parametric inference method based on a new strong approximation theory for mixingales. The usefulness of the method is demonstrated in empirical applications on volatility and inflation forecasting.

Are multifractal processes suited to forecasting electricity price volatility? Evidence from Australian intraday data

Abstract We analyze Australian electricity price returns and find that they exhibit volatility clustering, long memory, structural breaks, and multifractality. Consequently, we let the return mean equation follow two alternative specifications, namely (i) a smooth transition autoregressive fractionally integrated moving average (STARFIMA) process, and (ii) a Markov-switching autoregressive fractionally integrated moving average (MSARFIMA) process. We specify volatility dynamics via a set of (i) short- and long-memory GARCH-type processes, (ii) Markov-switching (MS) GARCH-type processes, and (iii) a Markov-switching multifractal (MSM) process. Based on equal and superior predictive ability tests (using MSE and MAE loss functions), we compare the out-of-sample relative forecasting performance of the models. We find that the (multifractal) MSM volatility model keeps up with the conventional GARCH- and MSGARCH-type specifications. In particular, the MSM model outperforms the alternative specifications, when using the daily squared return as a proxy for latent volatility.

Economic indicators and stock market volatility in an emerging economy

By analyzing the daily realized volatility series calculated from intraday stock price observations, this study examines the direct causality between one-day-ahead aggregate stock market volatility and several economic and financial indicators in the Korean market, a leading emerging market. Using the predictive regression and superior predictive ability tests, we find that the model-free implied volatility index (VKOSPI) and stock market indicators both lead the daily market volatility. However, daily economic indicators provide no predictive information beyond that contained in historical volatility. Though in-sample causality does not guarantee a better out-of-sample forecasting performance, the VKOSPI and combinations of predictors exhibit significant predictive ability regardless of the time period. Our study verifies the information role of the VKOSPI as an indicator of daily market risk.

Forecasting crude oil and refined products volatilities and correlations: New evidence from fractionally integrated multivariate GARCH models

Abstract The relationship between the prices of crude oil and its refined products is at the heart of the oil industry. Crude oil and refined products volatilities and correlations have been modelled extensively using short-memory multivariate GARCH models. This paper investigates the potential benefits from using fractionally integrated multivariate GARCH models from a forecasting and a risk management perspective. Several models for the spot returns on three major oil-related markets are compared. In-sample results show significant evidence of long-memory decay and leverage effects in volatilities and of time-varying autocorrelations. The forecasting performance of the models is assessed by means of three approaches: the Superior Predictive Ability test, the Model Confidence Set and the Value-at-Risk. The results indicate that the multivariate models incorporating long-memory outperform the short-memory benchmarks in forecasting the conditional covariance matrix and associated risk magnitudes. The paper makes an innovative contribution to the analysis of the relationship between crude oil and its refined products providing refiners, physical oil traders, non-commercial oil traders and other energy markets agents with significant insights for hedging and risk management operations.

Conditional Superior Predictive Ability

This paper proposes a test for the conditional superior predictive ability (CSPA) of a family of forecast methods with respect to a benchmark. The test is functional in nature: Under the null hypothesis, the benchmark’s conditional expected loss is no more than those of the competitors, uniformly across all conditioning states. By inverting the CSPA tests for a set of benchmarks, we obtain confidence sets for the uniformly most superior method. The econometric inference pertains to testing a system of conditional moment inequalities for time series data with general serial dependence, and we justify its asymptotic validity using a uniform nonparametric inference method based on a new strong approximation theory for mixingales. The usefulness of the method is demonstrated in empirical applications on volatility and inflation forecasting.

Forecasting Energy Price Volatilities and Correlations: New Evidence From Fractionally Integrated Multivariate Garch Models

Energy price volatilities and correlations have been modeled extensively using short-memory multivariate GARCH models. This paper investigates the potential benefits from using multivariate fractionally integrated GARCH models from a forecasting and a risk management perspective. Several multivariate GARCH models for the spot returns on three major energy markets are compared. Our in-sample results show significant evidence of long-memory decay in energy price returns volatilities, leverage effects and time-varying auto-correlations. The one-step ahead forecasting performance of the models is assessed using several robust matrix loss functions by means of three approaches: the Superior Predictive Ability test, the Model Confidence Set and the Value-at-Risk. The results indicate that the multivariate models incorporating long-memory outperform the short-memory benchmarks in forecasting the conditional co-variance matrix and associated risk magnitudes.

Forecasting crude oil price volatility via a HM-EGARCH model

This paper compares uni-regime GARCH-type models, GARCH-type models with Markov and hidden Markov (HM) switching regimes on their forecasting abilities in WTI and Daqing crude oil markets, respectively. Empirical results indicate a HM-EGARCH model outperforms the competitive models, namely the regular GARCH-type models and Markov regime-switching models as well as the other models with hidden Markov regimes through results of six loss functions and the superior predictive ability (SPA) test. More significantly, we find the HM-EGARCH not only performs well in developed crude oil markets, but also in emerging crude oil markets. Therefore, the HM-EGARCH model can be regard as an effective measure of volatility when accounting for different volatility states in the time-changing process.

Modeling unbiased extreme value volatility estimator in presence of heterogeneity and jumps: A study with economic significance analysis

Based on the heterogeneous autoregressive (HAR) model, the study proposes the frameworks (HAR-AddRS and HAR-AddRS-J) to incorporate the impact of heterogeneity and volatility jumps in modeling the AddRS volatility estimator (Kumar & Maheswaran, 2014a, 2014b). The forecasting performance of the HAR-AddRS and HAR-AddRS-J models in comparison to the returns-based conditional volatility models is evaluated using the error statistic approach, Hansen (2005) superior predictive ability (SPA) test, and Hansen, Lunde et al. (2011) model confidence set (MCS) approach. The findings suggest that more accurate forecasts of daily volatility are obtained based on the HAR-AddRS and HAR-AddRS-J models than based on the returns-based conditional volatility models. The economic significance analysis results show that a substantial economic gain is achieved when the volatility forecasts based on the HAR-AddRS-J model are used to implement the trading strategies, however, the same is not true when the volatility forecasts are based on the returns-based conditional volatility models.

Data driven value-at-risk forecasting using a SVR-GARCH-KDE hybrid

Appropriate risk management is crucial to ensure the competitiveness of financial institutions and the stability of the economy. One widely used financial risk measure is Value-at-Risk (VaR). VaR estimates based on linear and parametric models can lead to biased results or even underestimation of risk due to time varying volatility, skewness and leptokurtosis of financial return series. The paper proposes a nonlinear and nonparametric framework to forecast VaR that is motivated by overcoming the disadvantages of parametric models with a purely data driven approach. Mean and volatility are modeled via support vector regression (SVR) where the volatility model is motivated by the standard generalized autoregressive conditional heteroscedasticity (GARCH) formulation. Based on this, VaR is derived by applying kernel density estimation (KDE). This approach allows for flexible tail shapes of the profit and loss distribution, adapts for a wide class of tail events and is able to capture complex structures regarding mean and volatility. The SVR-GARCH-KDE hybrid is compared to standard, exponential and threshold GARCH models coupled with different error distributions. To examine the performance in different markets, one-day-ahead and ten-days-ahead forecasts are produced for different financial indices. Model evaluation using a likelihood ratio based test framework for interval forecasts and a test for superior predictive ability indicates that the SVR-GARCH-KDE hybrid performs competitive to benchmark models and reduces potential losses especially for ten-days-ahead forecasts significantly. Especially models that are coupled with a normal distribution are systematically outperformed.

Heterogeneous market hypothesis approach for modeling unbiased extreme value volatility estimator in presence of leverage effect: An individual stock level study with economic significance analysis

This paper explores the role of heterogeneity and leverage effect on the predictability of the AddRS volatility estimator (Kumar & Maheswaran, 2014a) using daily, weekly and monthly volatility components. Similar to the model setting of heterogeneous autoregressive (HAR) model (Corsi, 2009), we introduce the frameworks (HAR – AddRS and HAR – AddRS – L) to incorporate the impact of heterogeneity and leverage effect in modeling the AddRS volatility estimator. We find that the heterogeneity and leverage effect significantly impact the volatility prediction and when taken together produce a better in-sample fit. To evaluate the out-of-sample performance of our new volatility models, we compare the forecasting performance of our models with that of other traditional benchmark models forecasts using the error statistic approach and Hansen (2005) superior predictive ability (SPA) test. The results show that the HAR – AddRS and HAR – AddRS – L models provide more accurate volatility forecasts for the out-of-sample. We also undertake the economic significance analysis to highlight that a substantial economic gain is achieved when the volatility forecasts based on the HAR – AddRS – L model are used to implement various trading strategies, however, the same is not true when the volatility forecasts are based on the traditional returns-based conditional volatility models.