High-dimensional Time Series Research Articles

Robustness and the general dynamic factor model with infinite-dimensional space: Identification, estimation, and forecasting

General dynamic factor models have demonstrated their capacity to circumvent the curse of dimensionality in the analysis of high-dimensional time series and have been successfully considered in many economic and financial applications. As second-order models, however, they are sensitive to the presence of outliers—an issue that has not been analyzed so far in the general case of dynamic factors with possibly infinite-dimensional factor spaces (Forni et al. 2000, 2015, 2017). In this paper, we consider this robustness issue and study the impact of additive outliers on the identification, estimation, and forecasting performance of general dynamic factor models. Based on our findings, we propose robust versions of identification, estimation, and forecasting procedures. The finite-sample performance of our methods is evaluated via Monte Carlo experiments and successfully applied to a classical data set of 115 US macroeconomic and financial time series.

Segmentation of High Dimensional Time-Series Data Using Mixture of Sparse Principal Component Regression Model with Information Complexity.

This paper presents a new and novel hybrid modeling method for the segmentation of high dimensional time-series data using the mixture of the sparse principal components regression (MIX-SPCR) model with information complexity () criterion as the fitness function. Our approach encompasses dimension reduction in high dimensional time-series data and, at the same time, determines the number of component clusters (i.e., number of segments across time-series data) and selects the best subset of predictors. A large-scale Monte Carlo simulation is performed to show the capability of the MIX-SPCR model to identify the correct structure of the time-series data successfully. MIX-SPCR model is also applied to a high dimensional Standard & Poor’s 500 (S&P 500) index data to uncover the time-series’s hidden structure and identify the structure change points. The approach presented in this paper determines both the relationships among the predictor variables and how various predictor variables contribute to the explanatory power of the response variable through the sparsity settings cluster wise.

Modeling high-dimensional unit-root time series

This paper proposes a new procedure to build factor models for high-dimensional unit-root time series by postulating that a p-dimensional unit-root process is a nonsingular linear transformation of a set of unit-root processes, a set of stationary common factors that are dynamically dependent, and some idiosyncratic white noise components. For the stationary components, we assume that the factor process captures the temporal dependence, and that the idiosyncratic white noise series explains, jointly with the factors, the cross-sectional dependence. The estimation of nonsingular linear loading spaces is carried out in two steps. First, we use an eigenanalysis of a nonnegative definite matrix of the data to separate the unit-root processes from the stationary ones, and a modified method to specify the number of unit roots. We then employ another eigenanalysis and a projected principal component analysis to identify the stationary common factors and the white noise series. We propose a new procedure to specify the number of white noise series and, hence, the number of stationary common factors. We establish asymptotic properties of the proposed method for both fixed and diverging p as the sample size n increases, and use a simulation and a real example to demonstrate the performance of the proposed method in finite samples. We also compare our method with some commonly used ones in the literature regarding the forecast ability of the extracted factors, and find that the proposed method performs well in out-of-sample forecasting of a 508-dimensional PM2.5 series in Taiwan.

Inference of Break-Points in High-Dimensional Time Series

We consider a new procedure for detecting structural breaks in mean for high-dimensional time series. We target breaks happening at unknown time points and locations. In particular, at a fixed time point our method is concerned with either the biggest break in one location or aggregating simultaneous breaks over multiple locations. We allow for both big or small sized breaks, so that we can 1), stamp the dates and the locations of the breaks, 2), estimate the break sizes and 3), make inference on the break sizes as well as the break dates. Our theoretical setup incorporates both temporal and cross-sectional dependence, and is suitable for heavy-tailed innovations. We derive the asymptotic distribution for the sizes of the breaks by extending the existing powerful theory on local linear kernel estimation and high dimensional Gaussian approximation to allow for trend stationary time series with jumps. A robust long-run covariance matrix estimation is proposed, which can be of independent interest. An application on detecting structural changes of the US unemployment rate is considered to illustrate the usefulness of our method.

Hypothesis testing for high-dimensional time series via self-normalization

Self-normalization has attracted considerable attention in the recent literature of time series analysis, but its scope of applicability has been limited to low-/fixed-dimensional parameters for low-dimensional time series. In this article, we propose a new formulation of self-normalization for inference about the mean of high-dimensional stationary processes. Our original test statistic is a U-statistic with a trimming parameter to remove the bias caused by weak dependence. Under the framework of nonlinear causal processes, we show the asymptotic normality of our U-statistic with the convergence rate dependent upon the order of the Frobenius norm of the long-run covariance matrix. The self-normalized test statistic is then constructed on the basis of recursive subsampled U-statistics and its limiting null distribution is shown to be a functional of time-changed Brownian motion, which differs from the pivotal limit used in the low-dimensional setting. An interesting phenomenon associated with self-normalization is that it works in the high-dimensional context even if the convergence rate of original test statistic is unknown. We also present applications to testing for bandedness of the covariance matrix and testing for white noise for high-dimensional stationary time series and compare the finite sample performance with existing methods in simulation studies. At the root of our theoretical arguments, we extend the martingale approximation to the high-dimensional setting, which could be of independent theoretical interest.

Autoreservoir computing for multistep ahead prediction based on the spatiotemporal information transformation

We develop an auto-reservoir computing framework, Auto-Reservoir Neural Network (ARNN), to efficiently and accurately make multi-step-ahead predictions based on a short-term high-dimensional time series. Different from traditional reservoir computing whose reservoir is an external dynamical system irrelevant to the target system, ARNN directly transforms the observed high-dimensional dynamics as its reservoir, which maps the high-dimensional/spatial data to the future temporal values of a target variable based on our spatiotemporal information (STI) transformation. Thus, the multi-step prediction of the target variable is achieved in an accurate and computationally efficient manner. ARNN is successfully applied to both representative models and real-world datasets, all of which show satisfactory performance in the multi-step-ahead prediction, even when the data are perturbed by noise and when the system is time-varying. Actually, such ARNN transformation equivalently expands the sample size and thus has great potential in practical applications in artificial intelligence and machine learning.

Two sample tests for high-dimensional autocovariances

The problem of testing for the equality of autocovariances of two independent high-dimensional time series is studied. Tests based on the suprema or sums of suitable averages across the dimensions are adapted from the available literature. Another test based on principal component analysis (PCA) is introduced and studied in theory. An extension is also considered to the setting of testing for the equality of autocovariances of two populations, having multiple individual high-dimensional series from the two populations. The proposed methodologies are assessed on simulated data, with the performance of the introduced PCA testing being superior overall. An application using fMRI data from individuals experiencing two different emotional states is provided.

Long-range forecasting in feature-evolving data streams

Accurate long-range forecasting in high-dimensional feature-evolving time series is a pressing issue with multiple applications in optimal resource allocation, monitoring and budget planning. Stochastic nature of feature-evolving time series coupled with their temporal dependency pose a great challenge in their forecasting. This is because the length of input sequence (rows) may vary as data points evolve with their feature values (columns) changing. In high-dimensional feature-evolving heterogeneous time series, it is impractical to train a forecasting model per single time series across millions of metrics, leave alone space required to maintain the forecasting model and evolving time series in memory for timely streaming processing. Thus this paper proposes O̲ne sketchF̲itsA̲llT̲ime series algorithm, which is a stochastic deep neural network framework to address stated problems collectively. Extensive experiments on real-life datasets and rigorous evaluation showcases that OFAT is fast, robust, accurate and superior to the state-of-the-art methods.

Detecting changes in the second moment structure of high-dimensional sensor-type data in a K-sample setting

The K sample problem for high-dimensional vector time series is studied, especially focusing on sensor data streams, in order to analyze the second moment structure and detect changes across samples and/or across variables cumulated sum (CUSUM) statistics of bilinear forms of the sample covariance matrix. In this model, K independent vector time series are observed over a time span which may correspond to K sensors (locations) yielding d-dimensional data as well as K locations where d sensors emit univariate data. Unequal sample sizes are considered as arising when the sampling rate of the sensors differs. We provide large-sample approximations and two related change point statistics, a sum of squares and a pooled variance statistic. The resulting procedures are investigated by simulations and illustrated by analyzing a real data set.

Robust Estimation of the Mean and Covariance Matrix for High Dimensional Time Series

Robust Estimation of the Mean and Covariance Matrix for High Dimensional Time Series

Blind Source Separation for Compositional Time Series

Many geological phenomena are regularly measured over time to follow developments and changes. For many of these phenomena, the absolute values are not of interest, but rather the relative information, which means that the data are compositional time series. Thus, the serial nature and the compositional geometry should be considered when analyzing the data. Multivariate time series are already challenging, especially if they are higher dimensional, and latent variable models are a popular way to deal with this kind of data. Blind source separation techniques are well-established latent factor models for time series, with many variants covering quite different time series models. Here, several such methods and their assumptions are reviewed, and it is shown how they can be applied to high-dimensional compositional time series. Also, a novel blind source separation method is suggested which is quite flexible regarding the assumptions of the latent time series. The methodology is illustrated using simulations and in an application to light absorbance data from water samples taken from a small stream in Lower Austria.

Data driven governing equations approximations using attention based multistep neural networks

During the past decade, deep learning has represented the biggest research trend in the field of machine learning, which provides new powerful tools to interrogate high dimensional time series data in a way that has not been possible before. With the recent success in natural language processing, one would expect widespread adaptation to problems like time series forecasting and classification. After all, both involve processing sequential data. However, to this point, research on their adaptation to time series problems has remained limited. Recently, a multi-step time-stepping scheme without the need of direct access to temporal gradients has been proposed, which can accurately identify nonlinear dynamical systems from time series data. In this paper, we combined an attention mechanism with a deep model in a multi-step time-stepping scheme to perform nonlinear system identification and forecasting tasks. To our knowledge, this is the first paper to use attention based models to deal with nonlinear system identification and forecasting problems. The attention weights on rows select those variables that are helpful for forecasting, which can enhance the information across multiple time steps to capture temporal information. The experiment results indicate that the attention based model in a multi-step time-stepping scheme has better identification and prediction performance for nonlinear time series identification and forecasting problems.

A Metric Learning-Based Univariate Time Series Classification Method

High-dimensional time series classification is a serious problem. A similarity measure based on distance is one of the methods for time series classification. This paper proposes a metric learning-based univariate time series classification method (ML-UTSC), which uses a Mahalanobis matrix on metric learning to calculate the local distance between multivariate time series and combines Dynamic Time Warping(DTW) and the nearest neighbor classification to achieve the final classification. In this method, the features of the univariate time series are presented as multivariate time series data with a mean value, variance, and slope. Next, a three-dimensional Mahalanobis matrix is obtained based on metric learning in the data. The time series is divided into segments of equal intervals to enable the Mahalanobis matrix to more accurately describe the features of the time series data. Compared with the most effective measurement method, the related experimental results show that our proposed algorithm has a lower classification error rate in most of the test datasets.

High-dimensional changepoint detection via a geometrically inspired mapping

High-dimensional changepoint analysis is a growing area of research and has applications in a wide range of fields. The aim is to accurately and efficiently detect changepoints in time series data when both the number of time points and dimensions grow large. Existing methods typically aggregate or project the data to a smaller number of dimensions, usually one. We present a high-dimensional changepoint detection method that takes inspiration from geometry to map a high-dimensional time series to two dimensions. We show theoretically and through simulation that if the input series is Gaussian, then the mappings preserve the Gaussianity of the data. Applying univariate changepoint detection methods to both mapped series allows the detection of changepoints that correspond to changes in the mean and variance of the original time series. We demonstrate that this approach outperforms the current state-of-the-art multivariate changepoint methods in terms of accuracy of detected changepoints and computational efficiency. We conclude with applications from genetics and finance.

Dynamic Semiparametric Factor Model With Structural Breaks

For the change-point analysis of a high-dimensional time series, we consider a semiparametric model with dynamic structural break factors. With our model, the observations are described by a few low-dimensional factors with time-invariant loading functions of the covariates. Regarding the structural break, the factors are assumed to be nonstationary and follow a vector autoregression process with a change in the parameter values. In addition, to account for the known spatial discrepancies, we introduce discrete loading functions. We study the theoretical properties of the estimates of the loading functions and the factors. Moreover, we provide both the consistency and the asymptotic normality for making an inference on the estimated breakpoint. Importantly, our results hold for both large and small breaks in the factor dependency structure. The estimation precision is further illustrated via a simulation study. Finally, we present two empirical applications in modeling the dynamics of the minimum wage policy in China and analyzing a limit order book dataset. Supplementary materials for this article are available online.

Efficient Dynamic Latent Variable Analysis for High-Dimensional Time Series Data

Dynamic-inner canonical correlation analysis (DiCCA) extracts dynamic latent variables from high-dimensional time series data with a descending order of predictability in terms of $R^2$ . The reduced dimensional latent variables with rank-ordered predictability capture the dynamic features in the data, leading to easy interpretation and visualization. In this article, numerically efficient algorithms for DiCCA are developed to extract dynamic latent components from high-dimensional time series data. The numerically improved DiCCA algorithms avoid repeatedly inverting a covariance matrix inside the iteration loop of the numerical DiCCA algorithms. A further improvement using singular value decomposition converts the generalized eigenvector problem into a standard eigenvector problem for the DiCCA solution. Another improvement in model efficiency in this article is the dynamic model compaction of the extracted latent scores using autoregressive integrated moving average (ARIMA) models. Integrating factors, if existed in the latent variable scores, are made explicit in the ARIMA models. Numerical tests on two industrial datasets are provided to illustrate the improvements.

Nonparametric Spectral Analysis of Multivariate Time Series

Spectral analysis of multivariate time series has been an active field of methodological and applied statistics for the past 50 years. Since the success of the fast Fourier transform algorithm, the analysis of serial auto- and cross-correlation in the frequency domain has helped us to understand the dynamics in many serially correlated data without necessarily needing to develop complex parametric models. In this work, we give a nonexhaustive review of the mostly recent nonparametric methods of spectral analysis of multivariate time series, with an emphasis on model-based approaches. We try to give insights into a variety of complimentary approaches for standard and less standard situations (such as nonstationary, replicated, or high-dimensional time series), discuss estimation aspects (such as smoothing over frequency), and include some examples stemming from life science applications (such as brain data).

A Survey of Time Series Data Visualization Research

Time series data visualization integrates data analysis and mining, computer graphics, interaction design and other technologies and methods. This paper first analyzes the characteristics of time series data, including time and data attributes. Secondly, two kinds of visualization methods are summarized: one is the visualization method of time attribute, including spiral chart, calendar view, theme river view and dynamic view; the other is the visualization method of high dimensional time series data, which mainly summarizes four kinds of parallel coordinate methods. And the visual interaction design method is analyzed. Finally, the visualization of time series data is summarized and prospected.

Threshold factor models for high-dimensional time series

We consider a threshold factor model for high-dimensional time series in which the dynamics of the time series is assumed to switch between different regimes according to the value of a threshold variable. This is an extension of threshold modeling to a high-dimensional time series setting under a factor structure. Specifically, within each threshold regime, the time series is assumed to follow a factor model. The regime switching mechanism creates structural changes in the factor loading matrices. It provides flexibility in dealing with situations that the underlying states may be changing over time, as often observed in economic time series and other applications. We develop the procedures for the estimation of the loading spaces, the number of factors and the threshold value, as well as the identification of the threshold variable, which governs the regime change mechanism. The theoretical properties are investigated. Simulated and real data examples are presented to illustrate the performance of the proposed method.

Testing serial correlations in high-dimensional time series via extreme value theory

This paper proposes a simple test for detecting serial correlations in high-dimensional time series. The proposed test makes use of the robust properties of Spearman’s rank correlation and the theory of extreme values. Asymptotic properties of the test statistics are derived under some minor conditions as both the sample size and dimension go to infinity. The test is not sensitive to the underlying distribution of the time series so long as the data are continuously distributed. In particular, the existence of finite-order moments of the underlying distribution is not required, and asymptotic critical values of the test statistics are available in closed form. In finite samples, we correct biases of the sample autocorrelations and conduct simulations to study the performance of the proposed test statistics. Simulation results show that the proposed test statistics enjoy good properties of size and power in finite samples. We apply the proposed test to a 92-dimensional series of asset returns. Finally, a simple R code is available to obtain finite-sample critical values of the test statistics if needed.