Long-horizon Predictability Research Articles

Period-aggregated transformer for learning latent seasonalities in long-horizon financial time series.

Fluctuations in the financial market are influenced by various driving forces and numerous factors. Traditional financial research aims to identify the factors influencing stock prices, and existing works construct a common neural network learning framework that learns temporal dependency using a fixed time window of historical information, such as RNN and LSTM models. However, these models only consider the short-term and point-to-point relationships within stock series. The financial market is a complex and dynamic system with many unobservable temporal patterns. Therefore, we propose an adaptive period-aggregation model called the Latent Period-Aggregated Stock Transformer (LPAST). The model integrates a variational autoencoder (VAE) with a period-to-period attention mechanism for multistep prediction in the financial time series. Additionally, we introduce a self-correlation learning method and routing mechanism to handle complex multi-period aggregations and information distribution. Main contributions include proposing a novel period-aggregation representation scheme, introducing a new attention mechanism, and validating the model's superiority in long-horizon prediction tasks. The LPAST model demonstrates its potential and effectiveness in financial market prediction, highlighting its relevance in financial research and predictive analytics.

Predicting vehicle trajectory via combination of model-based and data-driven methods using Kalman filter

Predicting future trajectories of surrounding vehicles accurately benefits the decision-making and motion-planning of autonomous vehicles (AVs). Physical-model-based prediction methods are generally accurate in short horizon prediction, but often fail in long-horizon prediction owing to the simplicity of the models and the complexity of the environment. Data-driven prediction methods can achieve promising performance in complex traffic scenarios and long-horizon prediction tasks. However, they might not be reliable on some occasions compared with model-based approaches, mainly owing to their uncertainty, poor interpretability and unsatisfying ability to learn the physical constraints of vehicles. In this paper, a novel cooperative vehicle trajectory prediction strategy exploiting the advantages of the two categories of prediction methods is proposed. The prediction results of the data-driven and kinematic models together with their uncertainties are described by the bivariate Gaussian model, and subsequently combined with an innovatively designed Kalman filter. The fusion strategy provides both physical and environmental constraints on the two types of methods for better prediction performance in both short and long horizons. Experiments applying different data-driven methods were carried out on the public BEV High-D dataset, as well as our specially-collected first-point-of-view highway dataset. The results indicate that the proposed method outperforms both the sub-models adopted in terms of prediction accuracy.

Taking stock of long-horizon predictability tests: Are factor returns predictable?

This study provides a critical assessment of long-horizon return predictability tests using highly persistent regressors. We show that the commonly used statistics are typically oversized, leading to spurious inference. Instead, we propose a Wald statistic, which accommodates multiple predictors of (unknown) arbitrary persistence degree within the I(0)-I(1) range. The test statistic, based on an adaptation of the IVX procedure to a long-horizon regression framework, is shown to have a standard chi-squared asymptotic distribution (regardless of the stochastic properties of the regressors used as predictors) and to exhibit excellent finite-sample size and power properties. Employing this test statistic, we find evidence of predictability for “old” and “new” pricing factors with monthly returns, but this becomes weaker as the predictive horizon increases. The predictability evidence substantially weakens with annual data. Overall, we question the incremental value of using long-horizon predictive regressions.

Transformed regression-based long-horizon predictability tests

We propose new tests for long-horizon predictability based on IVX estimation of a transformed regression which explicitly accounts for the over-lapping nature of the dependent variable in the long-horizon regression arising from temporal aggregation. To improve efficiency, we moreover incorporate the residual augmentation approach recently used in the context of short-horizon predictability testing by Demetrescu and Rodrigues (2022). Our proposed tests improve on extant tests in the literature in a number of ways. First, they allow practitioners to remain ambivalent over the strength of the persistence of the predictors. Second, they are valid under much weaker conditions on the innovations than extant long-horizon predictability tests; in particular, we allow for general forms of conditional and unconditional heteroskedasticity in the innovations, neither of which are tied to a parametric model. Third, unlike the popular Bonferroni-based methods in the literature, our proposed tests can handle multiple predictors, and can be easily implemented as either one or two-sided hypotheses tests. Monte Carlo analysis suggests that our preferred tests offer improved finite sample properties compared to the leading tests in the literature. We report results from an empirical application investigating the use of real exchange rates for predicting nominal exchange rates and inflation.

Real-time temperature predictions via state-space model and parameters identification within rack-based cooling data centers

Current rack-based cooling architecture of data centers (DCs) is a promising method since it simplifies airflow distribution and provides fast cooling regulation. Real-time on-demand control of the cooling system is an effective way to improve its operational energy efficiency without sacrificing the thermal security of IT equipment. Accurate and fast temperature distribution prediction serves as one of the bases for ensuring the superior performance of advanced control algorithms. Thus, this study proposed a novel grey-box state-space model, to rapidly predict the dynamic temperature distribution for rack-based cooling DCs. Various heat transfer physics in the rack-based cooling DCs, including heat production caused by servers and heat movements by airflows, were modeled as a state-space structure using the zonal modeling approach. The coefficient matrices were identified through the prediction-error method (PEM), in order to avoid the extremely time-consuming process of obtaining accurate physical parameters regarding the system. This developed model was validated with an experimentally validated mechanistic model and computational fluid dynamics (CFD) simulations. Additionally, the impact of the prediction horizon's size and IT workload transient changes on the proposed model's prediction accuracy were investigated as well. Through simulation, the developed model achieves sufficient accuracy with an average root mean square error (RMSE) equal to 0.19 °C and less than 3% relative error for predicting 1-min dynamic temperature evolutions. Also, the developed model shows satisfying performances for the long-horizon prediction and transient scenario, which will facilitate advanced control techniques for DC cooling systems.

Constrained Gaussian Processes With Integrated Kernels for Long-Horizon Prediction of Dense Pedestrian Crowd Flows

In this paper, we present a novel approach for predicting pedestrian crowd dynamics over longer time horizons (30s). In dense environments over long time horizons, the number of pedestrian interactions is high, leading to the degradation of traditional pedestrian trajectory estimation techniques. Alternatively, we consider the macroscopic properties of the crowd as a whole, focusing on the flow of density. This approach benefits from not considering pedestrians individually, and can probabilistically estimate the existence of previously unobserved individuals. We propose a novel approach to imposing a physical constraint on the crowd density flow. Initially, a coarse resolution prediction is generated by a Convolutional Recurrent Neural Network (ConvRNN), and subsequently smoothly interpolated by a Gaussian Process (GP). Using the linearity properties of GPs, a continuous representation of the crowd is produced that complies with both the ConvRNN's prediction and a conservation of density constraint. The approach is trained and analysed on the dense ATC dataset, where we show the advantages of the approach and the improvements from our contributions.

A Multiple-Output Hybrid Ship Trajectory Predictor With Consideration for Future Command Assumption

Onboard sensors contribute to data-driven understanding of complex and nonlinear ship dynamics in real time. By using sensors, precise ship trajectory prediction plays a key role in intelligent collision avoidance. A hybrid predictor makes prediction based on a mathematical model of which error is compensated by a black-box model. A Multiple-output Hybrid Predictor (MHP), which makes a long-horizon prediction at a time based on onboard sensor data, was developed in the previous study. However, it cannot handle a time series of future command assumption. This limitation hinders an MHP from being applied to the evaluation of future command assumption in the predictive decision making. A novel architecture of MHP presented in this study converts a long time series of future command assumption into a fixed-length model-based-predicted vessel state; then, it is included in inputs of a black-box error compensator. This idea is robust to multidimensionality of commands and long control horizon. Assuming a low-fidelity vessel model and a limited data are available, simulation experiments are conducted. The effect of environmental disturbances and maneuverings on the prediction performance is examined comprehensively for the first time. The present study successfully incorporates a long time series of future command assumption in an MHP. It reduces the mean error by 81.8% compared to a model-based predictor and by 45.6% compared to a data-driven predictor under Beaufort wind force scale 4 wave, wind, and ocean current. Present study expands the application of MHP to the predictive decision making of future autonomous ships.

On LASSO for predictive regression

Explanatory variables in a predictive regression typically exhibit low signal strength and various degrees of persistence. Variable selection in such a context is of great importance. In this paper, we explore the pitfalls and possibilities of the LASSO methods in this predictive regression framework. In the presence of stationary, local unit root, and cointegrated predictors, we show that the adaptive LASSO cannot asymptotically eliminate all cointegrating variables with zero regression coefficients. This new finding motivates a novel post-selection adaptive LASSO, which we call the twin adaptive LASSO (TAlasso), to restore variable selection consistency. Accommodating the system of heterogeneous regressors, TAlasso achieves the well-known oracle property. In contrast, conventional LASSO fails to attain coefficient estimation consistency and variable screening in all components simultaneously. We apply these LASSO methods to evaluate the short- and long-horizon predictability of S&P 500 excess returns.

Long-horizon predictability and information decay in equity markets

We use machine learning to predict stock returns at forward horizons from 1 month ahead to 120 months ahead. Stock return predictability declines with the forecast horizon; it follows an asymptotic exponential decay process consisting of a permanent component (c. 20 bp/month) and a transient component (c. 240 bp/month) which decays at around 6% per month. Persistent but declining predictability at increasing horizons can be explained by persistent but declining benchmark risk factor exposures at each horizon. Limits to arbitrage linked to risk (but not implementation costs) also explains declining profitability.

Discount Shock, Price-Rent Dynamics, and the Business Cycle

The price-rent ratio is highly volatile and predicts future returns for commercial real estate. Price-rent variations in commercial real estate also tend to comove with investment and output. We develop a general equilibrium model that explicitly introduces a rental market and incorporates collateral constraints on production as a key ingredient. Our estimation identifies discount-rate shocks as the most important factor in (1) driving price-rent variations, (2) producing the long-horizon predictability of real estate returns, and (3) linking the dynamics in commercial real estate to those in the production sector.

Long-Horizon Predictability: A Cautionary Tale

Long-horizon return regressions effectively have small sample sizes. Using overlapping long-horizon returns provides only marginal benefit. Adjustments for overlapping observations have greatly overstated t-statistics. The evidence from regressions at multiple horizons is often misinterpreted. As a result, much less statistical evidence of long-horizon return predictability exists than is implied by research, which casts doubt on claims about forecasts based on stock market valuations and factor timing.

Taking Stock of Long-Horizon Predictability Tests: Are Factor Returns Predictable?

This study provides a critical assessment of long-horizon return predictability tests using highly persistent regressors. We show that the most commonly used test statistics are typically oversized, leading to spurious inference. As a remedy, we propose a simple Wald statistic, which can accommodate multiple predictors, exhibits excellent finite-sample properties regardless of the length of the predictive horizon, and is robust to the (unobservable) exact time series properties of the employed predictor(s). Employing this test statistic and examining a small set of variables that have been commonly used as proxies for business cycle conditions, we find evidence of predictability for old and new pricing factors with monthly returns. However, this evidence becomes weaker, not stronger, as the predictive horizon increases and disappears for most of the factors with annual returns. Overall, we cast doubt on the incremental value of using long-horizon predictive regressions.

Commodity Markets, Long-Run Predictability, and Intertemporal Pricing

This paper shows that commodity portfolios that capture the backwardation and contango phases exhibit in-sample and out-of-sample predictive power for the first two moments of the distribution of long-horizon aggregate equity market returns, and for the business cycle. It also demonstrates that a pricing model based on the corresponding backwardation and contango risk factors explains relatively well a wide cross-section of equity portfolios. The cross-sectional “hedging” risk prices are economically consistent with the direction of long-horizon predictability. Backwardation and contango thus act as plausible investment opportunity state variables in the context of Merton’s (1973) intertemporal CAPM.

Commodity Risk Factors and Intertemporal Asset Pricing

This paper shows that commodity portfolios that capture the backwardation and contango phases exhibit in-sample and out-of-sample predictive power for the first two moments of the distribution of long-horizon aggregate equity market returns, and for the business cycle. It also demonstrates that a pricing model based on the corresponding backwardation and contango risk factors explains relatively well a wide cross-section of equity portfolios. The cross-sectional “hedging” risk prices are economically consistent with the direction of long-horizon predictability. Backwardation and contango thus act as plausible investment opportunity state variables in the context of Merton’s (1973) intertemporal CAPM.

The existence and source of stock return predictability: Evidence from dividend, output and consumption ratios

This article considers stock return predictability and its source using ratios derived from stock prices, dividends, output and consumption. We analyse 29 stock markets (sampled quarterly) and s17 stock markets (sampled annually). One-period ahead predictive regressions provide some support for predictability of returns although there is also evidence supporting dividend and consumption growth predictability. Greater evidence for predictable stock market returns is found when estimating panel regressions and through consideration of long-horizon predictability. Furthermore, examining long horizons allows us to comment on the source of predictability. Our results suggest that predictability arises from both time variation in expected returns (risk appetite) and cash flow.

Exchange Rates and Fundamentals: A New Look at the Evidence on Long-Horizon Predictability

A widely held view in finance is that there is predictability in stock returns, bond returns, and exchange rates and that this predictability increases with the forecast horizon. Conventional tests for long horizon predictability may reject the null too frequently when the predictor variable is highly persistent and endogenous and there are overlapping observations creating unintended autocorrelation errors. We use a recently developed econometric technique to investigate predictability in short versus long horizon exchange rates. This procedure negates the historical bane of overlapping data i.e., the autocorrelation problem. This metric has strong and stable asymptotic properties even in small samples. We conduct extensive examination across a broad spectrum of time horizons, and find that the evidence for predictability is not as strong as previously stated.

Robust Econometric Inference for Stock Return Predictability

This study examines stock return predictability via lagged financial variables with unknown stochastic properties. We propose a novel testing procedure that (1) robustifies inference to regressors' degree of persistence, (2) accommodates testing the joint predictive ability of financial variables in multiple regression, (3) is easy to implement as it is based on a linear estimation procedure, and (4) can be used for long-horizon predictability tests. We provide some evidence in favor of short-horizon predictability during the 1927-2012 period. Nevertheless, this evidence almost entirely disappears in the post�1952 period. Moreover, predictability becomes weaker, not stronger, as the predictive horizon increases.

Ambiguity Aversion and Asset Prices in Production Economies

We examine a production-based asset-pricing model with an unobservable mean growth rate following a two-state Markov chain and with an ambiguity-averse representative agent. Our model requires a low coefficient of relative risk aversion to produce: (i) a high equity premium and volatile equity returns, (ii) a low and smooth risk-free rate, (iii) smooth consumption growth and volatile investment growth, (iv) countercyclical equity premium and market price of risk, (v) conditional heteroscedasticity in returns, and (vi) long-horizon predictability of excess returns.

The Power of Dynamic Asset Allocation

This article re-assesses the evidence and practical relevance of asset returns’ long-horizon predictability, investigating whether practitioners can profitably exploit predictability patterns by using relatively simple, dynamic asset allocation strategies. The analysis shows forward-looking models that rely on steady-state equations for equities and initial yields to maturity for bonds are far better predictors of markets’ long-run direction than is the industry’s conventional approach, which involves extrapolating from historical averages. Using a long-term U.S. sample from 1926 to 2010, the authors find that predictability translates into significantly better risk-adjusted performance from dynamic asset allocation strategies that rely on forward-looking inputs.

The Power of Dynamic Asset Allocation

This article re-assesses the evidence and practical relevance of asset returns’ long-horizon predictability, investigating whether practitioners can profitably exploit predictability patterns by using relatively simple, dynamic asset allocation strategies. The analysis shows forward-looking models that rely on steady-state equations for equities and initial yields to maturity for bonds are far better predictors of markets’ long-run direction than is the industry’s conventional approach, which involves extrapolating from historical averages. Using a long-term U.S. sample from 1926 to 2010, the authors find that predictability translates into significantly better risk-adjusted performance from dynamic asset allocation strategies that rely on forward-looking inputs.