Forecasting the dynamics of correlations in complex systems

Abstract

Investigating the correlations between time series is a fundamental approach to reveal the hidden mechanisms in complex systems. However, the estimated correlations often show time-dependent behaviors, which may create uncertainty for decision-making in various scenarios. Thus, forecasting the evolution of these varying correlations may be helpful, but it is still unsolved entirely. We bridge this gap by proposing a data-driven framework: (a) we first embed all the pairwise correlations within a complex system into multivariate correlation-based series by sliding windows; (b) we then identify two different low-dimensional representations of multivariate correlation-based series through delay embedding and dimensionality reduction; (c) finally, multistep ahead predictions of varying correlations can be achieved by training a mapping between two low-dimensional representations. Both model and real-world systems are used to illustrate our framework, including finance, neuroscience, and climate. Our framework is robust and has the potential to be used for other complex systems. Hopefully, forecasting the evolution of correlations in complex systems can be a useful complementary, since existing works mainly focus on the predictions of components within the systems.