Representation Of Time Series Research Articles

Identify Optimal Pedestrian Flow Forecasting Methods in Great Britain Retail Areas: A Comparative Study of Time Series Forecasting on a Footfall Dataset

The UK retail landscape has undergone significant changes over the past decade, driven by factors such as the rise of online shopping, economic downturns, and, more recently, the COVID-19 pandemic. Accurately measuring pedestrian flows in retail areas with high spatial and temporal resolution is essential for selecting the most appropriate forecasting model for different retail locations. However, several studies have adopted a one-size-fits-all approach, overlooking important local characteristics that are only occasionally captured by the best global model. In this work, using data generated by the SmartStreetSensor project, a large network of sensors installed across UK cities that collect Wi-Fi probe requests generated by mobile devices, we examine the optimal forecasting method to predict pedestrian footfall in various retail areas across Great Britain. After assessing six representative time series forecasting models, our results show that the LSTM model outperforms traditional methods in most areas. However, pedestrian counts at certain locations with specific spatial characteristics are better forecasted by other algorithms.

Research on time series analysis of data products in electric power marketing management information platform based on recurrent neural networks

Abstract The power marketing management information platform can promote the energy industry's decision-making ability for power marketing business and improve demand-side management. Due to the large number of power marketing users and heterogeneous data, traditional time series analysis methods are difficult to flexibly deal with this scenario, and deep learning methods based on feedforward neural networks have weak modeling capabilities for the time dimension of data, making it difficult to effectively use these data to support decision-making. To this end, this study proposes a new time series analysis model for the power marketing management information platform, which uses a recurrent neural network (RNN) enhanced by an attention mechanism. By integrating the long-term time dependency modeling capability of RNN and the adaptive extraction capability of the attention mechanism, the model can efficiently represent time series data. Experimental evaluation shows that our method greatly outperforms traditional models and achieves higher prediction accuracy on several key performance indicators.

Ratai: recurrent autoencoder with imputation units and temporal attention for multivariate time series imputation

Multivariate time series is ubiquitous in real-world applications, yet it often suffers from missing values that impede downstream analytical tasks. In this paper, we introduce the Long Short-Term Memory Network based Recurrent Autoencoder with Imputation Units and Temporal Attention Imputation Model (RATAI), tailored for multivariate time series. RATAI is designed to address certain limitations of traditional RNN-based imputation methods, which often focus on predictive modeling to estimate missing values, sometimes neglecting the contextual impact of observed data at and beyond the target time step. Drawing inspiration from Kalman smoothing, which effectively integrates past and future information to refine state estimations, RATAI aims to extract feature representations from time series data and use them to reconstruct a complete time series, thus overcoming the shortcomings of existing approaches. It employs a dual-stage imputation process: the encoder utilizes temporal information and attribute correlations to predict and impute missing values, and extract feature representation of imputed time series. Subsequently, the decoder reconstructs the series from the feature representation, and the reconstructed values are used as the final imputation values. Additionally, RATAI incorporates a temporal attention mechanism, allowing the decoder to focus on highly relevant inputs during reconstruction. This model can be trained directly using data that contains missing values, avoiding the misleading effects on model training that can arise from setting initial values for missing values. Our experiments demonstrate that RATAI outperforms benchmark models in multivariate time series imputation.

Time-Series Representation Feature Refinement with a Learnable Masking Augmentation Framework in Contrastive Learning.

In this study, we propose a novel framework for time-series representation learning that integrates a learnable masking-augmentation strategy into a contrastive learning framework. Time-series data pose challenges due to their temporal dependencies and feature-extraction complexities. To address these challenges, we introduce a masking-based reconstruction approach within a contrastive learning context, aiming to enhance the model's ability to learn discriminative temporal features. Our method leverages self-supervised learning to effectively capture both global and local patterns by strategically masking segments of the time-series data and reconstructing them, which aids in revealing nuanced temporal dependencies. We utilize learnable masking as a dynamic augmentation technique, which enables the model to optimize contextual relationships in the data and extract meaningful representations that are both context-aware and robust. Extensive experiments were conducted on multiple time-series datasets, including SleepEDF-78, 20, UCI-HAR, achieving improvements of 2%, 2.55%, and 3.89% each and similar performance on Epilepsy in accuracy over baseline methods. Our results show significant performance gains compared to existing methods, highlighting the potential of our framework to advance the field of time-series analysis by improving the quality of learned representations and enhancing downstream task performance.

Intraflow temporal correlation-based network traffic prediction

Traditional network traffic prediction methods often rely on complex models that extract features by spatiotemporal dependency analysis in network traffic matrices. However, these methods are limited by specific network topologies and lack generalization ability. Furthermore, it is difficult for traditional time series models to capture multiperiod features. In this study, we propose a network traffic prediction method to address the aforementioned issues, where we combine the flow-by-flow mechanism with an improved TimesNet model. The proposed method transforms the original network traffic matrix into origin-destination sequences using a flow-by-flow mechanism and then uses the improved TimesNet model to transform one-dimensional time series into multiperiod two-dimensional tensors. These transformations not only overcome the limitations of network topologies and the lack of one-dimensional time series representation ability but also focus the analysis on the temporal correlations within the flow. The CoTAttention module is introduced to extract the deep features of periodically changing network traffic, thus improving the accuracy of network traffic prediction. We conduct experiments on the Abilene and GÉANT datasets to verify the effectiveness of the proposed network traffic prediction model based on intraflow temporal correlations. Moreover, we compare its performance with other advanced methods. The experimental results show that the proposed method exhibits superior performance in network traffic prediction tasks, providing powerful guidance for network bandwidth allocation and resource optimization.

A novel tucker decomposition driven taxi travel demand forecasting algorithm

<p>This research aims to enhance taxi travel demand forecasting for sustainable urban traffic management and planning. We extend the Seasonal Autoregressive Integrated Moving Average model into a high-dimensional tensor form by treating the urban transport network as a Euclidean space and introducing Tucker decomposition. This novel approach, both theoretically significant and practically applicable, represents time series data as tensors to better capture multimodal structures and correlations, improving predictive accuracy. Tucker decomposition reduces computational complexity and memory requirements, making it ideal for large-scale urban traffic network prediction. Experiments on six real-world datasets show that the model’s MAE and RMSE are reduced by about 39.43% and 27.01% on average, respectively, compared to the baseline model. Notably, the model is computationally very efficient and takes only a relatively short time to train., suitable for real-time traffic management, congestion mitigation, and resource optimization. In summary, this work innovates time series analysis, providing an efficient and precise tool for urban traffic management and planning, contributing to sustainable urban transportation advancement.<strong></strong></p>

Self-Supervised Time Series Representation Learning via Cross Reconstruction Transformer.

Since labeled samples are typically scarce in real-world scenarios, self-supervised representation learning in time series is critical. Existing approaches mainly employ the contrastive learning framework, which automatically learns to understand similar and dissimilar data pairs. However, they are constrained by the request for cumbersome sampling policies and prior knowledge of constructing pairs. Also, few works have focused on effectively modeling temporal-spectral correlations to improve the capacity of representations. In this article, we propose the cross reconstruction transformer (CRT) to solve the aforementioned issues. CRT achieves time series representation learning through a cross-domain dropping-reconstruction task. Specifically, we obtain the frequency domain of the time series via the fast Fourier transform (FFT) and randomly drop certain patches in both time and frequency domains. Dropping is employed to maximally preserve the global context while masking leads to the distribution shift. Then a Transformer architecture is utilized to adequately discover the cross-domain correlations between temporal and spectral information through reconstructing data in both domains, which is called Dropped Temporal-Spectral Modeling. To discriminate the representations in global latent space, we propose instance discrimination constraint (IDC) to reduce the mutual information between different time series samples and sharpen the decision boundaries. Additionally, a specified curriculum learning (CL) strategy is employed to improve the robustness during the pretraining phase, which progressively increases the dropping ratio in the training process. We conduct extensive experiments to evaluate the effectiveness of the proposed method on multiple real-world datasets. Results show that CRT consistently achieves the best performance over existing methods by 2%-9%. The code is publicly available at https://github.com/BobZwr/Cross-Reconstruction-Transformer.

Bi-Branching Feature Interaction Representation Learning for Multivariate Time Series

Representational learning of time series plays a crucial role in various fields. However, existing time-series models do not perform well in representation learning. These models usually focus only on the relationship between variables at the same timestamp or only consider the change of individual variables in time, while failing to effectively integrate the two, which limits their ability to capture complex time dependencies and multivariate interactions. We propose a Bi-Branching Feature Interaction Representation Learning for Multivariate Time Series (Bi-FI) to address these issues. Specifically, we elaborated a frequency domain analysis branch to address the complex associations between variables that are difficult to visualize in the time domain. In addition, to eliminate the time lag effect, another branch employs the mechanism of variable tokenization for attention to learning intra-variable representations. Ultimately, we interactively fuse the features learned from the two branches to obtain a more comprehensive time series representation. Bi-FI performs well in three time series analysis tasks: long sequence prediction, classification, and anomaly detection. Our code and dataset will be available at https://github.com/wwy8/Bi_FI.

Rethinking self-supervised learning for time series forecasting: A temporal perspective

Self-supervised learning has garnered significant attention for its ability to learn meaningful representations. Recent advancements have introduced self-supervised methods for time series forecasting. However, these efforts have faced limitations due to two primary drawbacks. Firstly, these approaches often borrow techniques from vision and language domains without adequately addressing the unique temporal dependencies inherent in time series data. Secondly, time series often show that the distribution shifts over time, which makes accurate forecasting challenging. In response to these issues, we propose TempSSL, a self-supervised learning framework designed for time series forecasting. TempSSL divides the time series data into context (history data) and target (future data), employing two pre-training strategies: (1) Temporal Masked Modeling (TMM) designed to capture temporal dependencies by reconstructing future time series based on historical context; (2) Temporal Contrastive Learning (TCL) employs context and target as positive samples to enhance discriminative representations and mitigate distribution shifts within the time series. TempSSL’s innovation lies in two key aspects. Firstly, it underscores the importance of temporal dependencies for time series forecasting by designing specific pre-training tasks. Secondly, it effectively integrates contrastive learning and masked modeling, leveraging their respective strengths to develop time series representation with strong instance discriminability and local perceptibility. Extensive experiments across seven widely used benchmark datasets demonstrate that TempSSL consistently outperforms existing self-supervised and end-to-end forecasting methods, achieving improvements ranging from 1.92% ∼ 78.12%. Additionally, TempSSL’s practical effectiveness is further demonstrated through successful application in natural gas demand forecasting.

TVGeAN: Tensor Visibility Graph-Enhanced Attention Network for Versatile Multivariant Time Series Learning Tasks

This paper introduces Tensor Visibility Graph-enhanced Attention Networks (TVGeAN), a novel graph autoencoder model specifically designed for MTS learning tasks. The underlying approach of TVGeAN is to combine the power of complex networks in representing time series as graphs with the strengths of Graph Neural Networks (GNNs) in learning from graph data. TVGeAN consists of two new main components: TVG which extend the capabilities of visibility graph algorithms in representing MTSs by converting them into weighted temporal graphs where both the nodes and the edges are tensors. Each node in the TVG represents the MTS observations at a particular time, while the weights of the edges are defined based on the visibility angle algorithm. The second main component of the proposed model is GeAN, a novel graph attention mechanism developed to seamlessly integrate the temporal interactions represented in the nodes and edges of the graphs into the core learning process. GeAN achieves this by using the outer product to quantify the pairwise interactions of nodes and edges at a fine-grained level and a bilinear model to effectively distil the knowledge interwoven in these representations. From an architectural point of view, TVGeAN builds on the autoencoder approach complemented by sparse and variational learning units. The sparse learning unit is used to promote inductive learning in TVGeAN, and the variational learning unit is used to endow TVGeAN with generative capabilities. The performance of the TVGeAN model is extensively evaluated against four widely cited MTS benchmarks for both supervised and unsupervised learning tasks. The results of these evaluations show the high performance of TVGeAN for various MTS learning tasks. In particular, TVGeAN can achieve an average root mean square error of 6.8 for the C-MPASS dataset (i.e., regression learning tasks) and a precision close to one for the SMD, MSL, and SMAP datasets (i.e., anomaly detection learning tasks), which are better results than most published works.

TS-MAE: A masked autoencoder for time series representation learning

Self-supervised learning (SSL) has been widely researched in recent years. In Particular, generative self-supervised learning methods have achieved remarkable success in many AI domains, such as MAE in computer vision, well-known BERT, GPT in natural language processing, and GraphMAE in graph learning. However, in the context of time series analysis, not only is the work that follows this line limited but also the performance has not reached the potential as promised in other fields. To fill this gap, we propose a simple and elegant masked autoencoder for time series representation learning. Firstly, unlike most existing work which uses the Transformer as the backbone, we build our model based on neural ordinary differential equation which possesses excellent mathematical properties. Compared with the position encoding in Transformer, modeling the evolution patterns continuously could better extract the temporal dependency. Secondly, a timestamp-wise mask strategy is provided to cooperate with the autoencoder to avoid bias, and it also could reduce the cross-imputation between variables to learn more robust representations. Lastly, extensive experiments conducted on two classical tasks demonstrate the superiority of our model over the state-of-the-art ones.

Multi-label classification with imbalanced classes by fuzzy deep neural networks

Multi-label classification is an advantageous technique for managing uncertainty in classification problems where each data instance is associated with several labels simultaneously. Such situations are frequent in real-world scenarios, where decisions rely on imprecise or noisy data and adaptable classification methods are preferred. However, the problem of class imbalance represents a common characteristic of several multi-label datasets, in which the distribution of samples and their corresponding labels is non-uniform across the data space. In this paper, we propose a multi-label classification approach utilizing fuzzy logic in order to deal with the class imbalance problem. To eliminate the need for an expert to determine the logical rules of inference, deep neural networks are adopted, which have proven to be exceptionally effective for such problems. By combining both fuzzy inference systems and deep neural networks, the strengths and weaknesses of each approach can be mitigated. As a further development, a symbolic representation of time series is put in place to reduce data dimensionality and speed up the training procedure. This allows for more flexibility in model application, in particular with respect to time constraints arising from the causality of observed time series. Tests carried out on a multi-label classification dataset related to the current and voltage profiles of several household appliances show that the proposed model outperforms four baseline models for time series classification.

Power equipment defect prediction based on time series knowledge graph

With the increasing scale of power facilities, the operation monitoring of power equipment has become an important part of power system management. Defect prediction of power equipment is a key step in power system operation monitoring. In order to solve the problem of defect prediction of power equipment in large-scale power systems, this paper proposes a power equipment defect prediction model based on time series knowledge graph in combination with artificial intelligence technology. Multimodal information is fused through the attention mechanism, and then the time series representation of entities and relations is obtained by using the relationship-aware graph neural network and recurrent neural network. Finally, the defects of power equipment are predicted based on the time series representation. The method proposed in this paper can make full use of multimodal information and improve the accuracy of power equipment defect prediction. Experiments show that the performance of the model proposed in this paper is better than that of the baseline model.

Meta Koopman decomposition for time series forecasting under temporal distribution shifts

Time series forecasting facilitates various real-world applications and has attracted great research interests. In real-world scenarios, time series forecasting models confront a fundamental issue of temporal distribution shifts (TDS), i.e., the statistical properties of time series are evolving over time. In this paper, we utilize Koopman theory to address temporal distribution shifts. Koopman theory states any time series can be mapped into a Koopman space by proper measurement functions with their dynamics modeled by the infinite-dimensional linear Koopman operator. Therefore, time series under different distributions can be modeled by different Koopman operators. Considering the linearity of Koopman operators, the Koopman operators for representing time series under different distributions can be decomposed as a linear combination of a set of Koopman operators, which we termed as meta Koopman operators. We further theoretically show the infinite-dimensional Koopman operators can be approximated by finite matrix multiplications and the meta Koopman operators are equivalent to a set of matrices. Based on the analysis, we propose an auto-encoder framework for implementing the meta Koopman decomposition of time series, which is theoretically capable of handling TDS. Extensive experiments conducted on four real-world time series datasets demonstrate the superiority of the proposed model in tackling temporal distribution shifts.

Clustering-based compression for raster time series

Abstract A raster time series is a sequence of independent rasters arranged chronologically covering the same geographical area. These are commonly used to depict the temporal evolution of represented variables. The $T$-$k^{2}$-raster is a compact data structure that performs very well in practice for compact representations for raster time series. This structure classifies each raster as a snapshot or a log and encodes logs concerning their reference snapshots, which are the immediately preceding selected snapshots. An enhanced version of the $T$-$k^{2}$-raster, called Heuristic $T$-$k^{2}$-raster, incorporates a heuristic for automating the selection of snapshots. In this study, we investigate the optimality of the heuristic employed in Heuristic $T$-$k^{2}$-raster by comparing it with a dynamic programming (DP) approach. Our experimental evaluation demonstrates that Heuristic $T$-$k^{2}$-raster is a near-optimal solution, achieving compression performance almost identical to the DP method. These results indicate that variations of the structure that maintain the temporal order of the rasters are unlikely to significantly improve compression. Consequently, we explore an alternative approach based on clustering, where rasters are grouped according to their similarity, regardless of their temporal order. Our experimental evaluation reveals that this clustering-based strategy can enhance compression in scenarios characterized by cyclic behaviour.

The Time Series Classification of Discrete-Time Chaotic Systems Using Deep Learning Approaches

Discrete-time chaotic systems exhibit nonlinear and unpredictable dynamic behavior, making them very difficult to classify. They have dynamic properties such as the stability of equilibrium points, symmetric behaviors, and a transition to chaos. This study aims to classify the time series images of discrete-time chaotic systems by integrating deep learning methods and classification algorithms. The most important innovation of this study is the use of a unique dataset created using the time series of discrete-time chaotic systems. In this context, a large and unique dataset representing various dynamic behaviors was created for nine discrete-time chaotic systems using different initial conditions, control parameters, and iteration numbers. The dataset was based on existing chaotic system solutions in the literature, but the classification of the images representing the different dynamic structures of these systems was much more complex than ordinary image datasets due to their nonlinear and unpredictable nature. Although there are studies in the literature on the classification of continuous-time chaotic systems, no studies have been found on the classification of discrete-time chaotic systems. The obtained time series images were classified with deep learning models such as DenseNet121, VGG16, VGG19, InceptionV3, MobileNetV2, and Xception. In addition, these models were integrated with classification algorithms such as XGBOOST, k-NN, SVM, and RF, providing a methodological innovation. As the best result, a 95.76% accuracy rate was obtained with the DenseNet121 model and XGBOOST algorithm. This study takes the use of deep learning methods with the graphical representations of chaotic time series to an advanced level and provides a powerful tool for the classification of these systems. In this respect, classifying the dynamic structures of chaotic systems offers an important innovation in adapting deep learning models to complex datasets. The findings are thought to provide new perspectives for future research and further advance deep learning and chaotic system studies.

Evaluating and Comparing Measures of Aperiodic Neural Activity.

Neuro-electrophysiological recordings contain prominent aperiodic activity - meaning irregular activity, with no characteristic frequency - which has variously been referred to as 1/f (or 1/f-like activity), fractal, or 'scale-free' activity. Previous work has established that aperiodic features of neural activity is dynamic and variable, relating (between subjects) to healthy aging and to clinical diagnoses, and also (within subjects) tracking conscious states and behavioral performance. There are, however, a wide variety of conceptual frameworks and associated methods for the analyses and interpretation of aperiodic activity - for example, time domain measures such as the autocorrelation, fractal measures, and/or various complexity and entropy measures, as well as measures of the aperiodic exponent in the frequency domain. There is a lack of clear understanding of how these different measures relate to each other and to what extent they reflect the same or different properties of the data, which makes it difficult to synthesize results across approaches and complicates our overall understanding of the properties, biological significance, and demographic, clinical, and behavioral correlates of aperiodic neural activity. To address this problem, in this project we systematically survey the different approaches for measuring aperiodic neural activity, starting with an automated literature analysis to curate a collection of the most common methods. We then evaluate and compare these methods, using statistically representative time series simulations. In doing so, we establish consistent relationships between the measures, showing that much of what they capture reflects shared variance - though with some notable idiosyncrasies. Broadly, frequency domain methods are more specific to aperiodic features of the data, whereas time domain measures are more impacted by oscillatory activity. We extend this analysis by applying the measures to a series of empirical EEG and iEEG datasets, replicating the simulation results. We conclude by summarizing the relationships between the multiple methods, emphasizing opportunities for reexamining previous findings and for future work.

Skip-patching spatial–temporal discrepancy-based anomaly detection on multivariate time series

Anomaly detection in the Industrial Internet of Things (IIoT) is a challenging task that relies heavily on the efficient learning of multivariate time series representations. We introduce Skip-patching and Spatial–Temporal discrepancy mechanisms to improve the efficiency of detecting anomalies. Traditional feature extraction is hindered by redundant information in limited datasets. The situation is that feature generation from stable operational processes results in low-quality representations. To address this challenge, we propose the Skip-Patching mechanism. This approach involves selectively extracting features from partial data patches, prompting the model to learn more meaningful knowledge through self-supervised learning. It also effectively doubles the training sample size by creating independent sub-groups of patches. Despite the complex spatial and temporal relationships in IIoT systems, existing methods mainly extracted features from a single domain, either temporal or spatial (sensor-wise), or simply cascaded two features, i.e., one after one, which limited anomaly detection capabilities. To address this, we introduce the Spatial–Temporal Association Discrepancy component, which leverages discrepancies between spatial and temporal features to enhance latent representation learning. Our Skip-Patching Spatial–Temporal Anomaly Detection (SSAD) framework combines these two components to provide a more diverse and comprehensive learning process. Tested across four multivariate time series anomaly detection benchmarks, SSAD demonstrates superior performance, confirming the efficacy of combining Skip-patching and Spatial–Temporal features to enhance anomaly detection in IIoT systems.

Temporal scaling theory for bursty time series with clusters of arbitrarily many events.

Long-term temporal correlations in time series in a form of an event sequence have been characterized using an autocorrelation function that often shows a power-law decaying behavior. Such scaling behavior has been mainly accounted for by the heavy-tailed distribution of interevent times, i.e., the time interval between two consecutive events. Yet, little is known about how correlations between consecutive interevent times systematically affect the decaying behavior of the autocorrelation function. Empirical distributions of the burst size, which is the number of events in a cluster of events occurring in a short time window, often show heavy tails, implying that arbitrarily many consecutive interevent times may be correlated with each other. In the present study, we propose a model for generating a time series with arbitrary functional forms of interevent time and burst size distributions. Then, we analytically derive the autocorrelation function for the model time series. In particular, by assuming that the interevent time and burst size are power-law distributed, we derive scaling relations between power-law exponents of the autocorrelation function decay, interevent time distribution, and burst size distribution. These analytical results are confirmed by numerical simulations. Our approach helps to rigorously and analytically understand the effects of correlations between arbitrarily many consecutive interevent times on the decaying behavior of the autocorrelation function.

TimeCSL: Unsupervised Contrastive Learning of General Shapelets for Explorable Time Series Analysis

Unsupervised (a.k.a. Self-supervised) representation learning (URL) has emerged as a new paradigm for time series analysis, because it has the ability to learn generalizable time series representation beneficial for many downstream tasks without using labels that are usually difficult to obtain. Considering that existing approaches have limitations in the design of the representation encoder and the learning objective, we have proposed Contrastive Shapelet Learning (CSL), the first URL method that learns the general-purpose shapelet-based representation through unsupervised contrastive learning, and shown its superior performance in several analysis tasks, such as time series classification, clustering, and anomaly detection. In this paper, we develop TimeCSL, an end-to-end system that makes full use of the general and interpretable shapelets learned by CSL to achieve explorable time series analysis in a unified pipeline. We introduce the system components and demonstrate how users interact with TimeCSL to solve different analysis tasks in the unified pipeline, and gain insight into their time series by exploring the learned shapelets and representation.