High-dimensional Time Series Data Research Articles

A Deep Learning-Based Approach for High-Dimensional Industrial Steam Consumption Prediction to Enhance Sustainability Management

The continuous increase in industrialized sustainable development and energy demand, particularly in the use of steam, highlights the critical importance of efficient energy forecasting for sustainability. While current deep learning models have proven effective, they often involve numerous hyperparameters that are challenging to control and optimize. To address these issues, this research presents an innovative deep learning model, automatically fine-tuned using an improved RIME optimization algorithm (IRIME), with the aim of enhancing accuracy in energy forecasting. Initially, the bidirectional gated recurrent unit (BiGRU) exhibited promising results in prediction tasks but encountered difficulties in handling the complexity of high-dimensional time-series data related to industrial steam. To overcome this limitation, a bidirectional temporal convolutional network (BiTCN) was introduced to more effectively capture long-term dependencies. Additionally, the integration of a multi-head self-attention (MSA) mechanism enabled the model to more accurately identify and predict key features within the data. The IRIME-BiTCN-BiGRU-MSA model achieved outstanding predictive performance, with an R2 of 0.87966, MAE of 0.25114, RMSE of 0.34127, and MAPE of 1.2178, outperforming several advanced forecasting methods. Although the model is computationally complex, its high precision and potential for automation offer a promising tool for high-precision forecasting of industrial steam emissions. This development supports broader objectives of enhancing energy efficiency and sustainability in industrial processes.

Temporal Autoregressive Matrix Factorization for High-Dimensional Time Series Prediction of OSS.

Open-source software (OSS) plays an increasingly significant role in modern software development tendency, so accurate prediction of the future development of OSS has become an essential topic. The behavioral data of different open-source software are closely related to their development prospects. However, most of these behavioral data are typical high-dimensional time series data streams with noise and missing values. Hence, accurate prediction on such cluttered data requires the model to be highly scalable, which is not a property of traditional time series prediction models. To this end, we propose a temporal autoregressive matrix factorization (TAMF) framework that supports data-driven temporal learning and prediction. Specifically, we first construct a trend and period autoregressive model to extract trend and period features from OSS behavioral data, and then combine the regression model with a graph-based matrix factorization (MF) to complete the missing values by exploiting the correlations among the time series data. Finally, use the trained regression model to make predictions on the target data. This scheme ensures that TAMF can be applied to different types of high-dimensional time series data and thus has high versatility. We selected ten real developer behavior data from GitHub for case analysis. The experimental results show that TAMF has good scalability and prediction accuracy.

Deep learning-based source term estimation of hydrogen leakages from a hydrogen fueled gas turbine

Hydrogen fueled gas turbine is an efficient and environmentally friendly energy conversion equipment. However, it is prone to gas leakages from corrosion-prone components and pipe connections. In order to address gas leakage problems, source term estimation (STE) can be applied to estimate the location and intensity of the leakage sources. Traditional STE methods are based on an optimization procedure using an atmospheric transport and dispersion model, which requires considerable computation efforts and cannot meet the need of real-time implementation. The complex flow field around the gas turbine, the possible existence of multiple leakage sources, and the high-dimensional time series data further increase the difficulty of the STE problem. To address these challenges, in the present study, a deep learning based STE approach is proposed. The basic idea is to use long short-term memory auto-encoder (LSTM-AE) network to extract the encoded features of multi-sensor data and then use a deep neural network (NN) to establish the relationship between the encoded features and the source term parameters of hydrogen leakages. The multi-sensor data are generated using computational fluid dynamics (CFD) analysis to simulate relevant hydrogen leakage scenarios of concern. The results show that compared with other machine learning algorithms, the proposed approach has an overall best performance in terms of the STE accuracy. Specifically, its hydrogen leakage source localization accuracy is 0.9798, and the leakage strength estimation R-squared (R2) can reach 0.9632. When the training data proportion is only 20–30%, the proposed approach can still perform well, indicating the ability of the model to effectively predict the location and strength of the leakage source even with relatively less training data.

A data-driven LSTMSCBLS model for soft sensor of industrial process

Abstract In the chemical industry, data-driven soft sensor modeling plays a crucial role in efficiently monitoring product quality and status. Industrial data in these applications typically exhibit significant temporal characteristics, meaning that current information is influenced by data from previous periods. Effectively extracting and utilizing these temporal features is essential for achieving accurate soft sensor modeling in complex chemical scenarios. To address this challenge, this study proposes a data-driven Broad Learning System (BLS) model, which combines Long Short-Term Memory (LSTM) networks with an adaptive algorithm known as the Stochastic Configuration Algorithm (SC), referred to as LSTMSCBLS. The model operates in two stages: temporal feature extraction and final prediction. In the temporal feature extraction stage, the integration of the LSTM network with a feature attention mechanism allows for efficient extraction of temporal features from high-dimensional time-series data. In the final prediction stage, the SC is integrated into the BLS, effectively mitigating issues related to node space redundancy and the determination of the number of nodes. The effectiveness and superiority of the proposed model are demonstrated through two industrial case studies involving a debutanizer column and a sulfur recovery unit.

Testing ARCH effect of high-dimensional time series data

Testing for the autoregressive conditional heteroscedasticity (ARCH) effect is an important problem in economic and financial time series. In this article, we consider the ARCH effect test for high-dimensional time series. Our test statistic is built based on the maximum element of Spearman’s rank correlation at different lags. The bootstrap methods are used to approximate the null distribution of the proposed test statistics. We also consider the goodness-of-fit test for ARCH models by applying the test to the residuals of fitted models. Simulation results show that the proposed test statistics enjoy good properties of size in finite samples. Compared to the existing methods, the new test possesses much better power. Moreover, the test is not sensitive to the underlying distribution of the time series. The proposed method is illustrated via two real data.

Construction of time series causal network based on partial rank correlation

Discovering causal relationships from time series data is a significant research area in data mining. To tackle the challenges of identifying and constructing causal networks in non-linear, high-dimensional time series data, we introduce the partial rank correlation coefficient and propose two new structure learning algorithms suitable for time-series causal network modeling. In this paper, we make three main contributions. Firstly, we demonstrate the suitability of the partial rank correlation as a criterion for independence testing. Secondly, by integrating the partial rank correlation with constraint-based causal discovery methods, we propose the TS-PRCS algorithm for temporal causal network discovery. Thirdly, we combine ideas from local learning, constraint-based methods, and score-based search to propose the TS-MCHC algorithm, which combines mixed constraints and hill-climbing search for time-series causal discovery. Finally, we validate the effectiveness of the TS-PRCS and TS-MCHC algorithms through experiments on both simulated and real-world data. Compared to existing methods, our proposed algorithms exhibit robust causal detection capabilities in diverse non-linear environments and demonstrate efficient performance in high-dimensional time series networks. The experimental results affirm the ability of our methods to effectively uncover causal relationships among time series data, presenting promising prospects for practical applications.

Research and Application of Power Grid Data Anomaly Tracing Method Based on Time Series Correlation

Abstract Power grid data is a “barometer” that reflects the operational trend of the power grid system, so conducting a traceability study on abnormal data generated by the power grid system is crucial to maintaining the stable operation of the power grid system and preventing further malfunctions. This article proposes a maximum time series correlation ring tracing algorithm based on time series correlation: firstly, for a large amount of time series data collected from measurement points in the power grid system, a correlation coefficient matrix is calculated, and then a time series correlation graph is constructed using graph theory knowledge. The Kruskal algorithm is used to search for the longest spanning tree in the time series correlation graph. Finally, based on the spanning tree, the breadth-first search (BFS) algorithm is further used to obtain the maximum time series correlation ring[1]. By using the time series correlation of each node within the ring and the physical topological relationship between nodes, the abnormal data can be traced back. Distinguish whether the generation of abnormal data is due to a single point failure or system failure. Most of the existing fault tracing techniques are based on machine learning algorithms, which are difficult to widely apply in high-dimensional time series data due to their high complexity.Through experiments on real power grid data, the results verify the effectiveness of this method in anomaly tracing of high-dimensional time series data. Through comparative experiments, this method is superior to machine learning model algorithms in terms of efficiency and stability. At the same time, this method does not require additional calculations of the statistical distribution of the samples to be tested, thus greatly saving computational costs.

Robust Multi-Dimensional Time Series Forecasting.

Large-scale and high-dimensional time series data are widely generated in modern applications such as intelligent transportation and environmental monitoring. However, such data contains much noise, outliers, and missing values due to interference during measurement or transmission. Directly forecasting such types of data (i.e., anomalous data) can be extremely challenging. The traditional method to deal with anomalies is to cut out the time series with anomalous value entries or replace the data. Both methods may lose important knowledge from the original data. In this paper, we propose a multidimensional time series forecasting framework that can better handle anomalous values: the robust temporal nonnegative matrix factorization forecasting model (RTNMFFM) for multi-dimensional time series. RTNMFFM integrates the autoregressive regularizer into nonnegative matrix factorization (NMF) with the application of the L2,1 norm in NMF. This approach improves robustness and alleviates overfitting compared to standard methods. In addition, to improve the accuracy of model forecasts on severely missing data, we propose a periodic smoothing penalty that keeps the sparse time slices as close as possible to the time slice with high confidence. Finally, we train the model using the alternating gradient descent algorithm. Numerous experiments demonstrate that RTNMFFM provides better robustness and better prediction accuracy.

Autoencoding for the "Good Dictionary" of eigenpairs of the Koopman operator

<abstract><p>Reduced order modelling relies on representing complex dynamical systems using simplified modes, which can be achieved through the Koopman operator(KO) analysis. However, computing Koopman eigenpairs for high-dimensional observable data can be inefficient. This paper proposes using deep autoencoders(AE), a type of deep learning technique, to perform nonlinear geometric transformations on raw data before computing Koopman eigenvectors. The encoded data produced by the deep AE is diffeomorphic to a manifold of the dynamical system and has a significantly lower dimension than the raw data. To handle high-dimensional time series data, Takens' time delay embedding is presented as a preprocessing technique. The paper concludes by presenting examples of these techniques in action.</p></abstract>

Deep learning for the prediction of clinical outcomes in internet-delivered CBT for depression and anxiety.

In treating depression and anxiety, just over half of all clients respond. Monitoring and obtaining early client feedback can allow for rapidly adapted treatment delivery and improve outcomes. This study seeks to develop a state-of-the-art deep-learning framework for predicting clinical outcomes in internet-delivered Cognitive Behavioural Therapy (iCBT) by leveraging large-scale, high-dimensional time-series data of client-reported mental health symptoms and platform interaction data. We use de-identified data from 45,876 clients on SilverCloud Health, a digital platform for the psychological treatment of depression and anxiety. We train deep recurrent neural network (RNN) models to predict whether a client will show reliable improvement by the end of treatment using clinical measures, interaction data with the iCBT program, or both. Outcomes are based on total improvement in symptoms of depression (Patient Health Questionnaire-9, PHQ-9) and anxiety (Generalized Anxiety Disorder-7, GAD-7), as reported within the iCBT program. Using internal and external datasets, we compare the proposed models against several benchmarks and rigorously evaluate them according to their predictive accuracy, sensitivity, specificity and AUROC over treatment. Our proposed RNN models consistently predict reliable improvement in PHQ-9 and GAD-7, using past clinical measures alone, with above 87% accuracy and 0.89 AUROC after three or more review periods, outperforming all benchmark models. Additional evaluations demonstrate the robustness of the achieved models across (i) different health services; (ii) geographic locations; (iii) iCBT programs, and (iv) client severity subgroups. Results demonstrate the robust performance of dynamic prediction models that can yield clinically helpful prognostic information ready for implementation within iCBT systems to support timely decision-making and treatment adjustments by iCBT clinical supporters towards improved client outcomes.

Disparities in resilience and recovery of ridesourcing usage during COVID-19

The COVID-19 pandemic has impacted ridesourcing services dramatically, but empirical research on disparities in the resilience and recovery of ridesourcing has been scarce. To address this literature gap, we used ridesourcing trip data in Chicago to create two time series: one for Census tract-level ridesourcing usage (including pickups and dropoffs) and the other for linkages between origin and destination (OD) pairs. We performed time-series clustering analyses that integrated manifold learning and Gaussian Mixture Modeling to optimize the number of clusters for high-dimensional time-series data. The tract-level usage can be grouped into three clusters, and the OD-pair linkages can be grouped into six clusters. We examined the spatial patterns of the tract-level usage clusters and the OD-pair linkage clusters. Furthermore, we estimated a multinomial logit regression model to examine the relationships between clusters and land use, built environment, and sociodemographic factors. Our results suggested that the share of residential land use had a positive association with high resilience and fast recovery of ridesourcing usage. Limited transportation accessibility and a lack of alternative transportation modes were also associated with high resilience and fast recovery of ridesourcing usage. Trips that linked dense employment centers were less likely to be made during the pandemic. Census tracts with a greater share of minorities or a higher poverty rate tended to generate more ridesourcing trips during the pandemic.

Reduced-Rank Envelope Vector Autoregressive Model

The standard vector autoregressive (VAR) models suffer from overparameterization which is a serious issue for high-dimensional time series data as it restricts the number of variables and lags that can be incorporated into the model. Several statistical methods, such as the reduced-rank model for multivariate (multiple) time series (Velu, Reinsel, and Wichern; Reinsel and Velu; Reinsel, Velu, and Chen) and the Envelope VAR model (Wang and Ding), provide solutions for achieving dimension reduction of the parameter space of the VAR model. However, these methods can be inefficient in extracting relevant information from complex data, as they fail to distinguish between relevant and irrelevant information, or they are inefficient in addressing the rank deficiency problem. We put together the idea of envelope models into the reduced-rank VAR model to simultaneously tackle these challenges, and propose a new parsimonious version of the classical VAR model called the reduced-rank envelope VAR (REVAR) model. Our proposed REVAR model incorporates the strengths of both reduced-rank VAR and envelope VAR models and leads to significant gains in efficiency and accuracy. The asymptotic properties of the proposed estimators are established under different error assumptions. Simulation studies and real data analysis are conducted to evaluate and illustrate the proposed method.

Sensing anomaly of photovoltaic systems with sequential conditional variational autoencoder

The market for urban distributed photovoltaics (DPV) is expected to take off in the next decade. However, these systems are often subject to complex urban contexts and sub-optimal conditions, requiring scalable and comprehensive solutions to detect their underperformances. In recent years, deep generative models (DGMs) have exhibited outstanding performance in the anomaly detection domain, dealing with generic high-dimensional time series data. Nevertheless, the existing applications of DGMs in the photovoltaic (PV) sector are still unable to account for environmental information, limiting their performance under various environmental conditions. This study proposes the Sequential Conditional Variational Autoencoder (SCVAE), which can cope with the sequential impacts of the environment on PV power generation. Using real-world data collected from 30 rooftop PV sites located across China, a data processing pipeline is developed to construct the training datasets which contain mostly normal samples for unsupervised SCVAE model training. This work also constructs a synthetic dataset with a wide variety of artificial anomalies in reference to the domain insights and engineering practice of DPV systems. After checking and refining by experts, the synthetic dataset can finally be used to validate the anomaly detection models. The results demonstrate that the SCVAE model outperforms existing state-of-the-art unsupervised anomaly detection models and can be effectively generalized to unseen PV sites. Moreover, the latent variables of SCVAE could be used to identify the type of DPV failure, thereby enabling more targeted diagnostics of anomaly mechanisms.

Pynapple, a toolbox for data analysis in neuroscience.

Datasets collected in neuroscientific studies are of ever-growing complexity, often combining high-dimensional time series data from multiple data acquisition modalities. Handling and manipulating these various data streams in an adequate programming environment is crucial to ensure reliable analysis, and to facilitate sharing of reproducible analysis pipelines. Here, we present Pynapple, the PYthon Neural Analysis Package, a lightweight python package designed to process a broad range of time-resolved data in systems neuroscience. The core feature of this package is a small number of versatile objects that support the manipulation of any data streams and task parameters. The package includes a set of methods to read common data formats and allows users to easily write their own. The resulting code is easy to read and write, avoids low-level data processing and other error-prone steps, and is open source. Libraries for higher-level analyses are developed within the Pynapple framework but are contained within a collaborative repository of specialized and continuously updated analysis routines. This provides flexibility while ensuring long-term stability of the core package. In conclusion, Pynapple provides a common framework for data analysis in neuroscience.

Deep Direct Discriminative Decoders for High-dimensional Time-series Data Analysis

The state-space models (SSMs) are widely utilized in the analysis of time-series data. SSMs rely on an explicit definition of the state and observation processes. Characterizing these processes is not always easy and becomes a modeling challenge when the dimension of observed data grows or the observed data distribution deviates from the normal distribution. Here, we propose a new formulation of SSM for high-dimensional observation processes with a heavy-tailed distribution. We call this solution the deep direct discriminative process (D4). The D4 brings deep neural networks’ expressiveness and scalability to the SSM formulation letting us build a novel solution that efficiently estimates the underlying state processes through high-dimensional observation signal.We demonstrate the D4 solutions in simulated and real data such as Lorenz attractors, Langevin dynamics, random walk dynamics, and rat hippocampus spiking neural data and show that the D4’s performance precedes traditional SSMs and RNNs. The D4 can be applied to a broader class of time-series data where the connection between high-dimensional observation and the underlying latent process is hard to characterize.

A dive into spectral inference networks: improved algorithms for self-supervised learning of continuous spectral representations

We propose a self-supervising learning framework for finding the dominant eigenfunction-eigenvalue pairs of linear and self-adjoint operators. We represent target eigenfunctions with coordinate-based neural networks and employ the Fourier positional encodings to enable the approximation of high-frequency modes. We formulate a self-supervised training objective for spectral learning and propose a novel regularization mechanism to ensure that the network finds the exact eigenfunctions instead of a space spanned by the eigenfunctions. Furthermore, we investigate the effect of weight normalization as a mechanism to alleviate the risk of recovering linear dependent modes, allowing us to accurately recover a large number of eigenpairs. The effectiveness of our methods is demonstrated across a collection of representative benchmarks including both local and non-local diffusion operators, as well as high-dimensional time-series data from a video sequence. Our results indicate that the present algorithm can outperform competing approaches in terms of both approximation accuracy and computational cost.

Tsunami waveform forecasting at cooling water intakes of nuclear reactors with deep learning model

The Fukushima nuclear disaster highlights the importance of accurate and fast predictions of tsunami hazard to critical coastal infrastructure to devise mitigation strategies in both long-term and real-time events. Recently, deep learning models allowed us to make accurate and rapid forecasts on high dimensional, non-linear, and non-stationary time series data such as that associated with tsunami waveforms. Thus, this study uses a one-dimensional convolutional neural network (CNN) model to predict waveforms at cooling water intakes of nuclear power plant at Uljin in South Korea. The site is particularly vulnerable to tsunamis originating from the west coast of Japan. Data for the CNN model are generated by numerical simulation of 1107 cases of tsunami propagation initiating from fault locations. The time series data for waveforms were predicted at 13 virtual gauges located in the nearshore region of the study area, 10 of which were classified as observation points and 3 gauges situated at the cooling water intakes were categorized as target locations. The performance assessment of the model's forecasts showed excellent results with rapid predictions. The study highlights two main points: (i) deep learning models can be based on sparse waveform in situ data (such as that recorded by deep-ocean assessment and reporting of tsunamis or any locally operating monitoring stations for ocean waves) or numerically simulated data at only a few points along the dominant wave propagation direction, and (ii) deep learning models are fully capable of accurate and fast predictions of complex geo-hazards that prompt rapid emergency response to coordinate mitigation efforts.

Learning State Transition Rules from High-Dimensional Time Series Data with Recurrent Temporal Gaussian-Bernoulli Restricted Boltzmann Machines

Understanding the dynamics of a system is crucial in various scientific and engineering domains. Machine learning techniques have been employed to learn state transition rules from observed time-series data. However, these data often contain sequences of noisy and ambiguous continuous variables, while we typically seek simplified dynamics rules that capture essential variables. In this work, we propose a method to extract a small number of essential hidden variables from high-dimensional time-series data and learn state transition rules between hidden variables. Our approach is based on the Restricted Boltzmann Machine (RBM), which models observable data in the visible layer and latent features in the hidden layer. However, real-world data, such as video and audio, consist of both discrete and continuous variables with temporal relationships. To address this, we introduce the Recurrent Temporal Gaussian-Bernoulli Restricted Boltzmann Machine (RTGB-RBM), which combines the Gaussian-Bernoulli Restricted Boltzmann Machine (GB-RBM) to handle continuous visible variables and the Recurrent Temporal Restricted Boltzmann Machine (RT-RBM) to capture time dependencies among discrete hidden variables. Additionally, we propose a rule-based method to extract essential information as hidden variables and represent state transition rules in an interpretable form. We evaluate our proposed method on the Bouncing Ball, Moving MNIST, and dSprite datasets. Experimental results demonstrate that our approach effectively learns the dynamics of these physical systems by extracting state transition rules between hidden variables. Moreover, our method can predict unobserved future states based on observed state transitions.

Zero-Inflated Time Series Clustering Via Ensemble Thick-Pen Transform

This study develops a new clustering method for high-dimensional zero-inflated time series data. The proposed method is based on thick-pen transform (TPT), in which the basic idea is to draw along the data with a pen of a given thickness. Since TPT is a multi-scale visualization technique, it provides some information on the temporal tendency of neighborhood values. We introduce a modified TPT, termed ‘ensemble TPT (e-TPT)’, to enhance the temporal resolution of zero-inflated time series data that is crucial for clustering them efficiently. Furthermore, this study defines a modified similarity measure for zero-inflated time series data considering e-TPT and proposes an efficient iterative clustering algorithm suitable for the proposed measure. Finally, the effectiveness of the proposed method is demonstrated by simulation experiments and two real datasets: step count data and newly confirmed COVID-19 case data.

Event Detection in Groundwater Sensor Networks using Artificial Intelligence Autho

In this paper, a method for detecting spatial and temporal anomalous events in groundwater sensor networks (high-dimensional time series data), such as system faults and attacks will be developed. Unlike recently developed deep learning frameworks for anomaly detection which do not consider the dependences between the variables and apply the existing relationships to predict the expected behavior of the sensors, the method in this paper extracts the relationships between the sensors spatially and temporally and learn to detect and simultaneously explain deviations from these relationships. This challenge is solved by using graph attention neural networks and structured learning. Attention neural networks can give useful interpretability in context of the anomalies detected and allows to identify their causes. To improve robustness the method considers aleatoric and parametric uncertainties by using ensemble specific value prediction and prediction intervals without assuming any data distribution. Furthermore, the model was connected to a fully connected classifier to classify typical groundwater network anomalies. The method was applied to a study area and it could be shown that the method could capture in 92% of the cases the complex correlations between the high dimensional variables, and enabled analysts to identify the causes of the anomalies.