High-dimensional Time Series Data Research Articles

The Analysis of Enterprise Improvement in Global Commodity Price Prediction Based on Deep Learning

The article expects to solve the traditional econometric statistical model, shallow machine learning algorithm, and many limitations in learning the nonlinear relationship of related indicators affecting commodity futures price trend. This article proposes a neural network commodity futures price prediction model by the mixture of convolutional neural networks (CNN) and gated recurrent unit (GRU). Firstly, the dimension reduction algorithm of multidimensional data by principal component analysis (PCA) is used. Through linear transformation, the original variables with correlation are transformed into a set of new linear irrelevant variables, and the high-dimensional time series data of commodity futures are reduced. Secondly, the variable features are extracted from the CNN network module in the CNN-GRU model, and the GRU network module learns the periodicity and trend of the original data. Finally, the full connection layer outputs the forecast results of commodity futures price.

Rate-optimal robust estimation of high-dimensional vector autoregressive models

High-dimensional time series data appear in many scientific areas in the current data-rich environment. Analysis of such data poses new challenges to data analysts because of not only the complicated dynamic dependence between the series, but also the existence of aberrant observations, such as missing values, contaminated observations, and heavy-tailed distributions. For high-dimensional vector autoregressive (VAR) models, we introduce a unified estimation procedure that is robust to model misspecification, heavy-tailed noise contamination, and conditional heteroscedasticity. The proposed methodology enjoys both statistical optimality and computational efficiency, and can handle many popular high-dimensional models, such as sparse, reduced-rank, banded, and network-structured VAR models. With proper regularization and data truncation, the estimation convergence rates are shown to be almost optimal in the minimax sense under a bounded (2+2ϵ)th moment condition. When ϵ≥1, the rates of convergence match those obtained under the sub-Gaussian assumption. Consistency of the proposed estimators is also established for some ϵ∈(0,1), with minimax optimal convergence rates associated with ϵ. The efficacy of the proposed estimation methods is demonstrated by simulation and a U.S. macroeconomic example.

Lite and Efficient Deep Learning Model for Bearing Fault Diagnosis Using the CWRU Dataset

Bearing defects are a common problem in rotating machines and equipment that can lead to unexpected downtime, costly repairs, and even safety hazards. Diagnosing bearing defects is crucial for preventative maintenance, and deep learning models have shown promising results in this field. On the other hand, the high complexity of these models can lead to high computational and data processing costs, making their practical implementation challenging. Recent studies have focused on optimizing these models by reducing their size and complexity, but these methods often compromise classification performance. This paper proposes a new approach that reduces the dimensionality of input data and optimizes the model structure simultaneously. A much lower input data dimension than that of existing deep learning models was achieved by downsampling the vibration sensor signals used for bearing defect diagnosis and constructing spectrograms. This paper introduces a lite convolutional neural network (CNN) model with fixed feature map dimensions that achieve high classification accuracy with low-dimensional input data. The vibration sensor signals used for bearing defect diagnosis were first downsampled to reduce the dimensionality of the input data. Next, spectrograms were constructed using the signals of the minimum interval. Experiments were conducted using the vibration sensor signals from the Case Western Reserve University (CWRU) dataset. The experimental results show that the proposed method could be highly efficient in terms of computation while maintaining outstanding classification performance. The results show that the proposed method outperformed a state-of-the-art model for bearing defect diagnosis under different conditions. This approach is not limited to the field of bearing failure diagnosis, but could be applied potentially to other fields that require the analysis of high-dimensional time series data.

Forecasting upper respiratory tract infection burden using high-dimensional time series data and forecast combinations.

Upper respiratory tract infections (URTIs) represent a large strain on primary health resources. To mitigate URTI transmission and public health burdens, it is important to pre-empt and provide forward guidance on URTI burden, while taking into account various facets which influence URTI transmission. This is so that appropriate public health measures can be taken to mitigate strain on primary care resources. This study describes a new approach to forecasting URTIs which can be used for national public health resource planning. Specifically, using environmental and disease data comprising more than 1000 dimensions, we developed sub-models which optimizes model explainability, in-sample model fit, predictive accuracy and combines many weaker predictors over a 2-month time horizon to generate direct, point forecasts over a 1-8 week ahead forecast horizon. Predictive performance was evaluated using rolling out-of-sample forecast assessment within both periods with/without structural breaks in transmission over the period of 2012-2022. We showed that forecast combinations of 5 other forecasting models had better and more consistent predictive performance than other modelling approaches, over periods with and without structural breaks in transmission dynamics. Furthermore, epidemiological analysis on high dimensional data was enabled using post-selection inference, to show the dynamic association between lower temperature, increases in past relative humidity and absolute humidity and increased URTIs attendance. The methods proposed can be used for outbreak preparedness and guide healthcare resource planning, in both stable periods of transmission and periods where structural breaks in data occur.

A Review of Outlier Detection and Robust Estimation Methods for High Dimensional Time Series Data

Diagnostic procedures for finding outliers in high dimensional multivariate time series and robust estimation methods for these data are reviewed. First, methods for searching for outliers assuming that the data have been generated by a Dynamic Factor Model are presented. Then, other existing methods for detecting different types of multivariate time series outliers are analyzed. They include identifying outlying series and, also, looking for segments, or periods of time, where the series have unusual dynamics. Second, robust estimation methods are considered. Dynamic Principal Components, as a very general procedure to estimate the dynamic in a high dimensional data set, is introduced and different types of robust estimation of these components are reviewed. Dynamic Principal Components can be applied for robust estimation of Generalized Dynamic Factor models and some results are given. Finally, other methods proposed for robust estimation of high dimensional VAR models and other multivariate time series problems are also briefly discussed.

Forecasting Infectious and Parasitic Disease Emergency Department Attendances Using High-Dimensional Time Series Data

Forecasting Infectious and Parasitic Disease Emergency Department Attendances Using High-Dimensional Time Series Data

Time series regression based on Bayesian model averaging and principal component analysis

This paper proposed an adaptive prediction model for high-dimensional time series data based on model averaging method and principal component analysis. Specifically, this paper considers the case where the response variable is a scalar and the predictor variable is a time series. Firstly, the high-dimensional time series data is extracted information by principal component analysis. Secondly, the Bayesian model averaging method is used to perform the forecast task based on the principal component projection matrix. The proposed method can effectively deal with the unsupervised nature of PCA and avoid the problem of selecting the number of PCA. It is demonstrated that the proposed method is competitive compared with the lasso regression and the ridge regression by real data analyses.

An explainable knowledge distillation method with XGBoost for ICU mortality prediction

Background and Objective:Mortality prediction is an important task in intensive care unit (ICU) for quantifying the severity of patients’ physiological condition. Currently, scoring systems are widely applied for mortality prediction, while the performance is unsatisfactory in many clinical conditions due to the non-specificity and linearity characteristics of the used model. As the availability of the large volume of data recorded in electronic health records (EHRs), deep learning models have achieved state-of-art predictive performance. However, deep learning models are hard to meet the requirement of explainability in clinical conditions. Hence, an explainable Knowledge Distillation method with XGBoost (XGB-KD) is proposed to improve the predictive performance of XGBoost while supporting better explainability. Methods:In this method, we first use outperformed deep learning teacher models to learn the complex patterns hidden in high-dimensional multivariate time series data. Then, we distill knowledge from soft labels generated by the ensemble of teacher models to guide the training of XGBoost student model, whose inputs are meaningful features obtained from feature engineering. Finally, we conduct model calibration to obtain predicted probabilities reflecting the true posterior probabilities and use SHapley Additive exPlanations (SHAP) to obtain insights about the trained model. Results:We conduct comprehensive experiments on MIMIC-III dataset to evaluate our method. The results demonstrate that our method achieves better predictive performance than vanilla XGBoost, deep learning models and several state-of-art baselines from related works. Our method can also provide intuitive explanations. Conclusions:Our method is useful for improving the predictive performance of XGBoost by distilling knowledge from deep learning models and can provide meaningful explanations for predictions.

Data-driven discovery of intrinsic dynamics

Dynamical models underpin our ability to understand and predict the behaviour of natural systems. Whether dynamical models are developed from first-principles derivations or from observational data, they are predicated on our choice of state variables. The choice of state variables is driven by convenience and intuition, and, in data-driven cases, the observed variables are often chosen to be the state variables. The dimensionality of these variables (and consequently the dynamical models) can be arbitrarily large, obscuring the underlying behaviour of the system. In truth these variables are often highly redundant and the system is driven by a much smaller set of latent intrinsic variables. In this study we combine the mathematical theory of manifolds with the representational capacity of neural networks to develop a method that learns a system’s intrinsic state variables directly from time-series data, as well as predictive models for their dynamics. What distinguishes our method is its ability to reduce data to the intrinsic dimensionality of the nonlinear manifold they live on. This ability is enabled by the concepts of charts and atlases from the theory of manifolds, whereby a manifold is represented by a collection of patches that are sewn together—a necessary representation to attain intrinsic dimensionality. We demonstrate this approach on several high-dimensional systems with low-dimensional behaviour. The resulting framework provides the ability to develop dynamical models of the lowest possible dimension, capturing the essence of a system. Learning minimal representations of dynamical systems is essential for mathematical modelling and prediction in science and engineering. Floryan and Graham propose a deep learning framework able to estimate accurate global dynamical models by sewing together multiple local representations learnt from high-dimensional time-series data.

SSP-WGAN-Based Data Enhancement and Prediction Method for Cement Clinker f-CaO

Aiming at the problem of low prediction accuracy of traditional prediction models due to the limited labeled sample data and the imbalance of multitimescale sample data in the cement production process, a cement clinker-free calcium oxide (f-CaO) data enhancement and prediction model based on semisupervised prediction Wasserstein generative adversarial networks (SSP-WGANs) is proposed in this article. The model is constructed by WGAN and a prediction model. In the generator of WGAN, the traditional noise input is replaced by time-series matrices composed of related variables affecting the f-CaO content of cement clinker. The generator maps the input high-dimensional time-series data into low-dimensional labeled values through its internal convolutional layer, which can fill in the missing labeled values of the input unlabeled samples. The generated labels are spliced with the related variables affecting the f-CaO content of clinker according to the timescale relationship and are uniformly sent to the discriminator with the real sample pairs for discrimination, thus eliminating the influence of multiple timescales. The generated labeled data are matched with the unlabeled data and used to expand the training set of the prediction model composed of CNN-GRU, which significantly improves the prediction accuracy of the model. The results show that the SSP-WGAN model with data enhancement has higher accuracy and stronger robustness.

ADT: AI-Driven network Telemetry processing on routers

Network monitoring is a pivotal part of network management and operations. It is responsible for monitoring the behavior of the network to assure its functionality within expectation and to guarantee a smooth-running environment for enabling of various services. Therefore, operators are interested in gaining a comprehensive assessment of their network elements and tracking operational changes to facilitate timely correction of any deviation. Commonly, this assessment is achieved by performing regular manual checks of different operational counters and defining expert rules from known root causes. The common approach requires the maintenance of a regularly updated set of rules and only goes as far as the operator’s pre-gained knowledge of the system. With the growing complexity of the networks as well as the availability of more data, a more efficient monitoring approach is necessary to address the emerging network monitoring requirements.In this paper, a novel unsupervised approach is proposed that is capable of exploring a broader set of counters (not limited to the handpicked Key Performance Indicators (KPIs)). The goal is to leverage the dependencies between the counters in order to discover complex state changes that might have otherwise slipped the operator’s view. This paper proposes ADT, an AI-driven telemetry processing solution that facilitates monitoring of a larger set of counters. The Detector block of ADT is known as DESTIN, a multivariate unsupervised change detection for high dimensional time-series data of originally low effective dimension, which provides near real-time state assessment of network devices. The efficiency of the proposed approach is demonstrated and compared with well-known methodologies on an experimental test-bed. The method’s performance is also explored extensively considering different criteria such as traffic type, device and the type of events to identify its potentials and limitations. The datasets used for the evaluation are made publicly available.

TPD: a web tool for tipping-point detection based on dynamic network biomarker.

Tipping points or critical transitions widely exist during the progression of many biological processes. It is of great importance to detect the tipping point with the measured omics data, which may be a key to achieving predictive or preventive medicine. We present the tipping point detector (TPD), a web tool for the detection of the tipping point during the dynamic process of biological systems, and further its leading molecules or network, based on the input high-dimensional time series or stage course data. With the solid theoretical background of dynamic network biomarker (DNB) and a series of computational methods for DNB detection, TPD detects the potential tipping point/critical state from the input omics data and outputs multifarious visualized results, including a suggested tipping point with a statistically significant P value, the identified key genes and their functional biological information, the dynamic change in the DNB/leading network that may drive the critical transition and the survival analysis based on DNB scores that may help to identify 'dark' genes (nondifferential in terms of expression but differential in terms of DNB scores). TPD fits all current browsers, such as Chrome, Firefox, Edge, Opera, Safari and Internet Explorer. TPD is freely accessible at http://www.rpcomputationalbiology.cn/TPD.

Dynamic Bayesian Network Learning to Infer Sparse Models From Time Series Gene Expression Data.

One of the key challenges in systems biology is to derive gene regulatory networks (GRNs) from complex high-dimensional sparse data. Bayesian networks (BNs) and dynamic Bayesian networks (DBNs) have been widely applied to infer GRNs from gene expression data. GRNs are typically sparse but traditional approaches of BN structure learning to elucidate GRNs often produce many spurious (false positive) edges. We present two new BN scoring functions, which are extensions to the Bayesian Information Criterion (BIC) score, with additional penalty terms and use them in conjunction with DBN structure search methods to find a graph structure that maximises the proposed scores. Our BN scoring functions offer better solutions for inferring networks with fewer spurious edges compared to the BIC score. The proposed methods are evaluated extensively on auto regressive and DREAM4 benchmarks. We found that they significantly improve the precision of the learned graphs, relative to the BIC score. The proposed methods are also evaluated on three real time series gene expression datasets. The results demonstrate that our algorithms are able to learn sparse graphs from high-dimensional time series data. The implementation of these algorithms is open source and is available in form of an R package on GitHub at https://github.com/HamdaBinteAjmal/DBN4GRN, along with the documentation and tutorials.

Dynamic multivariate analysis for pollution assessment and river habitat conservation in the Vietnamese La Buong watershed.

Analysis of temporal patterns of high-dimensional time-series water quality data is essential for pollution management worldwide. This study has applied dynamic factor analysis (DFA) and cluster analysis (CA) to analyze time-series water quality data monitored at the five stations installed along the La Buong river in Southern Vietnam. Application of the DFA identified two types of temporal patterns, one of the run-off driven parameters (total suspended solid (TSS), turbidity, and iron) and the other of diffuse source pollution. The association of the variables like BOD5 and COD at most stations to the run-off-driven parameters revealed their sharing of drivers. On the contrary, separating variables like phosphate (PO43) at the three upstream stations from the run-off patterns suggested their local point-source origin. The DFA-derived factors were later used in the time-point CA to explore the seasonality of water quality parameters and their pollution intensities compared to regulatory levels. The result suggested intensification in wet season of Fe, TSS, BOD5, and COD concentrations at most sites, which are unobservable in run-off detached parameters like reactive nitrogen, phosphate (PO43-), and E. coli. These findings generated robust insights to support water quality management for river habitat conservation.

A novel method to detect the early warning signal of COVID-19 transmission

BackgroundInfectious illness outbreaks, particularly the corona-virus disease 2019 (COVID-19) pandemics in recent years, have wreaked havoc on human society, and the growing number of infected patients has put a strain on medical facilities. It’s necessary to forecast early warning signals of potential outbreaks of COVID-19, which would facilitate the health ministry to take some suitable control measures timely to prevent or slow the spread of COVID-19. However, since the intricacy of COVID-19 transmission, which connects biological and social systems, it is a difficult task to predict outbreaks of COVID-19 epidemics timely.ResultsIn this work, we developed a new model-free approach, called, the landscape network entropy based on Auto-Reservoir Neural Network (ARNN-LNE), for quantitative analysis of COVID-19 propagation, by mining dynamic information from regional networks and short-term high-dimensional time-series data. Through this approach, we successfully identified the early warning signals in six nations or areas based on historical data of COVID-19 infections.ConclusionBased on the newly published data on new COVID-19 disease, the ARNN-LNE method can give early warning signals for the outbreak of COVID-19. It’s worth noting that ARNN-LNE only relies on small samples data. Thus, it has great application potential for monitoring outbreaks of infectious diseases.

Functional non-parametric latent block model: A multivariate time series clustering approach for autonomous driving validation

Advanced driving-assistance systems validation remains one of the biggest challenges car manufacturers must tackle to provide safe driverless cars. The reliable validation of these systems requires to assess their reaction's quality and consistency to a broad spectrum of driving scenarios. In this context, large-scale simulation systems bypass the physical “on-tracks” limitations and produce important quantities of high-dimensional time series data. The challenge is to find valuable information in these multivariate unlabeled datasets that may contain noisy, sometimes correlated or non-informative variables. A new model-based tool is proposed for multivariate time series clustering based on a Bayesian co-clustering approach. The tool discriminates groups of correlated temporal variables, while modeling noise and providing probabilistic confidence interval for outlier detection. The proposed Functional Non-Parametric Latent Block Model (FunNPLBM) simultaneously creates a partition of observations and a partition of variables, using latent multivariate Gaussian block distributions. The model parameters follow a bi-dimensional Dirichlet Process as a prior for the block distribution parameters and for block proportions, and natively provides model selection. The method's capacities are illustrated with experiments and benchmarks on a simulated dataset and on an advanced driver-assistance system validation use-case.

Predicting high-dimensional time series data with spatial, temporal and global information

In the field of time series forecasting, deep learning and dynamics-based methods are two main research directions. The former focuses on the temporal information of the data while the latter emphasizes on the spatial information of the data, and rare methods combine the two information properly. In order to make better use of the information in the data, we propose the STSM (spatiotemporal skip-connection model) based on the dynamics framework, which contains a temporal module composed of CNN and a spatial module composed of fully connected layers, as well as a skip connection to the original input to fuse temporal, spatial, and global information in the data. To predict the future value of the target variable, STSM is required to learn a mapping from original attractors to delay attractors in an end-to-end framework. The results of ablation and contrast experiments on one simulated dataset and seven real-world datasets show that STSM not only performs better than a separate temporal or spatial module, but also predicts more accurately than other traditional methods. Besides, we verify the robustness of the model in different scenarios through several experiments.

Sparse representations of high dimensional neural data

Conventional Vector Autoregressive (VAR) modelling methods applied to high dimensional neural time series data result in noisy solutions that are dense or have a large number of spurious coefficients. This reduces the speed and accuracy of auxiliary computations downstream and inflates the time required to compute functional connectivity networks by a factor that is at least inversely proportional to the true network density. As these noisy solutions have distorted coefficients, thresholding them as per some criterion, statistical or otherwise, does not alleviate the problem. Thus obtaining a sparse representation of such data is important since it provides an efficient representation of the data and facilitates its further analysis. We propose a fast Sparse Vector Autoregressive Greedy Search (SVARGS) method that works well for high dimensional data, even when the number of time points is relatively low, by incorporating only statistically significant coefficients. In numerical experiments, our methods show high accuracy in recovering the true sparse model. The relative absence of spurious coefficients permits accurate, stable and fast evaluation of derived quantities such as power spectrum, coherence and Granger causality. Consequently, sparse functional connectivity networks can be computed, in a reasonable time, from data comprising tens of thousands of channels/voxels. This enables a much higher resolution analysis of functional connectivity patterns and community structures in such large networks than is possible using existing time series methods. We apply our method to EEG data where computed network measures and community structures are used to distinguish emotional states as well as to ADHD fMRI data where it is used to distinguish children with ADHD from typically developing children.

A hybrid deep learning approach for driver anomalous lane changing identification

Reliable knowledge of driving states is of great importance to ensure road safety. Anomaly detection in driving behavior means recognizing anomalous driving states as a direct result of either environmental or psychological factors. This paper provides an efficient anomaly recognition approach to identify anomalous lane-changing events in a personalized manner. The proposed framework includes three unsupervised algorithms. First, a Recurrent-Convolutional Autoencoder extracts the spatio-temporal characteristics from a high-dimensional naturalistic driving dataset. Second, in order to recognize anomalous lane-changing events of individual drivers, the extracted latent feature space is analyzed using Pauta criterion-based reconstruction loss analysis, as well as one-class Support Vector Machine. Last, t-Distributed Stochastic Neighbor Embedding is employed to visualize the latent space for better understanding and interpretability. Temporal anomalies of lane-changing events were analyzed by a personalized grey relational coefficient analysis, to represent robust similarities for individual drivers. Validation and calibration were performed with a natural driving study dataset collected from 50 drivers with 59,372 lane change events. The results showed heterogeneity in the pattern of abnormal lane changing behavior across the sample. At the same time, each driver exhibited heterogeneous anomalous behaviors in both temporal and spatial sequences. Without prior labels, the proposed model effectively captures personalized driving patterns and abnormal lane-changing events from high-dimensional time-series data. This unsupervised hybrid approach is a novel attempt to complete personalized anomalous lane-changing behaviors identification based on naturalistic driving data involving various traffic environments. Our approach enables the extraction of natural individual lane-changing behavior patterns and provides insights for the improvement of personalized driving behavior monitoring systems.

Multivariate autoregressive model estimation for high-dimensional intracranial electrophysiological data

Fundamental to elucidating the functional organization of the brain is the assessment of causal interactions between different brain regions. Multivariate autoregressive (MVAR) modeling techniques applied to multisite electrophysiological recordings are a promising avenue for identifying such causal links. They estimate the degree to which past activity in one or more brain regions is predictive of another region's present activity, while simultaneously accounting for the mediating effects of other regions. Including as many mediating variables as possible in the model has the benefit of drastically reducing the odds of detecting spurious causal connectivity. However, effective bounds on the number of MVAR model coefficients that can be estimated reliably from limited data make exploiting the potential of MVAR models challenging for even modest numbers of recording sites. Here, we utilize well-established dimensionality-reduction techniques to fit MVAR models to human intracranial data from ∼100 – 200 recording sites spanning dozens of anatomically and functionally distinct cortical regions. First, we show that high-dimensional MVAR models can be successfully estimated from long segments of data and yield plausible connectivity profiles. Next, we use these models to generate synthetic data with known ground-truth connectivity to explore the utility of applying principal component analysis and group least absolute shrinkage and selection operator (gLASSO) to reduce the number of parameters (connections) during model fitting to shorter data segments. We show that gLASSO is highly effective for recovering ground-truth connectivity in the limited data regime, capturing important features of connectivity for high-dimensional models with as little as 10 seconds of data. The methods presented here have broad applicability to the analysis of high-dimensional time series data in neuroscience, facilitating the elucidation of the neural basis of sensation, cognition, and arousal.