Representation Of Time Series Research Articles

Learning Informative Representation for Fairness-Aware Multivariate Time-Series Forecasting: A Group-Based Perspective

Learning Informative Representation for Fairness-Aware Multivariate Time-Series Forecasting: A Group-Based Perspective

Multifractal Properties of Human Chromosome Sequences

The intricacy and fractal properties of human DNA sequences are examined in this work. The core of this study is to discern whether complete DNA sequences present distinct complexity and fractal attributes compared with sequences containing exclusively exon regions. In this regard, the entire base pair sequences of DNA are extracted from the NCBI (National Center for Biotechnology Information) database. In order to create a time series representation for the base pair sequence {G,C,T,A}, we use the Chaos Game Representation (CGR) approach and a mapping rule f, which enables us to apply the metric known as the Complexity–Entropy Plane (CEP) and multifractal detrended fluctuation analysis (MF-DFA). To carry out our investigation, we divided human DNA into two groups: the first is composed of the 24 chromosomes, which comprises all the base pairs that form the DNA sequence, and another group that also includes the 24 chromosomes, but the DNA sequences rely only on the exons’ presence. The results show that both sets provide fractal patterns in their structure, as obtained by the CGR approach. Complete DNA sequences show a sharper visual fractal pattern than sequences composed only of exons. Moreover, the sequences occupy distinct areas of the complexity–entropy plane, and the complete DNA sequences lead to greater statistical complexity and lower entropy than the exon sequences. Also, we observed that different fractal parameters between chromosomes indicate diversity in genomic sequences. All these results occur in different scales for all chromosomes.

Addressing the Non-Stationarity and Complexity of Time Series Data for Long-Term Forecasts

Real-life time series datasets exhibit complications that hinder the study of time series forecasting (TSF). These datasets inherently exhibit non-stationarity as their distributions vary over time. Furthermore, the intricate inter- and intra-series relationships among data points pose challenges for modeling. Many existing TSF models overlook one or both of these issues, resulting in inaccurate forecasts. This study proposes a novel TSF model designed to address the challenges posed by real-life data, delivering accurate forecasts in both multivariate and univariate settings. First, we propose methods termed “weak-stationarizing” and “non-stationarity restoring” to mitigate distributional shift. These methods enable the removal and restoration of non-stationary components from individual data points as needed. Second, we utilize the spectral decomposition of weak-stationary time series to extract informative features for forecasting. To learn features from the spectral decomposition of weak-stationary time series, we exploit a mixer architecture to find inter- and intra-series dependencies from the unraveled representation of the overall time series. To ensure the efficacy of our model, we conduct comparative evaluations against state-of-the-art models using six real-world datasets spanning diverse fields. Across each dataset, our model consistently outperforms or yields comparable results to existing models.

Two-dimensional LSTM soft sensor using noisy process data

Establishing a reliable data-based soft sensor still faces a series of challenges, particularly the presence of outliers and different kinds of noise which are non-negligible in process data. To address these challenges, a correntropy-based two-dimensional long short-term memory (TLSTM) model is developed to handle noisy process data. First, the multidimensional time series samples are reconstructed into numerous two-dimensional input matrices in the feature and time directions. Then, the convolution and pooling operations are used to extract useful information in the process variables related to the quality variable. Meanwhile, a gating mechanism is employed to learn the internal representation of time series. Finally, a correntropy-based strategy is utilized to assign relatively small weights to outliers automatically, enabling reliable prediction. Two cases illustrate the reliability and advantages of TLSTM in effectively extracting quality-related features for prediction.

Biometric identity recognition based on contrastive positive-unlabeled learning

With the development of neural networks, identity recognition techniques utilizing human physiological signals have found applications in healthcare systems. However, many current physiological signal datasets have to spend a lot of resources in the pre-labeling phase, which leads to longer training times. To reduce the cost of labeling, we propose a self-supervised time-series representation learning framework based on contrastive positive-unlabeled learning (TS − CPUL), to learn the representations of temporal neighborhood sequences. To obtain the representations, the unit root test divides the unlabeled time series into neighborhood samples and unlabeled samples. Secondly, we use neighborhood sample representations and unlabeled sample representations to do self-supervised contrastive learning and positive-unlabeled learning with a negative sample selector. Both approaches can deepen the model’s understanding of the sample representation. Lastly, to enhance the model’s capability in discriminating between ambiguous sample pairs, we design the distance-polarized matrix. The proposed TS-CPUL is evaluated through experiments on three physiological signal datasets. The results demonstrate its effectiveness in understanding time-series representations and performing well in classification. For the identity recognition task, we achieve accuracy comparable to the supervised model by fine-tuning the model with only 10% of the labeled data.

Multi-task self-supervised time-series representation learning

Time-series representation learning is crucial for extracting meaningful representations from time-series data with temporal dynamics and sparse labels. Contrastive learning, a powerful technique for exploiting the inherent data patterns, has been applied to explore the diverse consistencies in time-series data, achieved through careful selection of contrastive pairs and design of appropriate loss. Encouraging such consistency is essential for acquiring comprehensive representations of time-series data. In this paper, we propose a new framework for time-series representation learning that combines the advantages of contextual, temporal, and transformation consistencies. This framework enables the network to learn general representations suitable for different tasks and domains. First, positive and negative pairs are generated to establish a multi-task learning setup. Then, contrastive losses are formulated to explore contextual, temporal, and transformation consistencies, which are jointly optimized to learn general time-series representations. In addition, we investigate an uncertainty weighting approach to enhance the effectiveness of multi-task learning. To evaluate the performance of our framework, we conduct experiments on three downstream tasks: time-series classification, forecasting, and anomaly detection. The experimental results demonstrate the superior performance of our framework compared to benchmark models across different tasks. Furthermore, our framework shows efficiency in cross-domain transfer learning scenarios.

HFN: Heterogeneous feature network for multivariate time series anomaly detection

As the key step of anomaly detection for multivariate time-series (MTS) data, learning the relations among different variables has been explored by many approaches. However, most existing approaches overlook the heterogeneity among variables, that is, different types of variables (continuous numerical variables, discrete categorical variables or hybrid variables) may have different edge distributions. In this paper, we propose a novel semi-supervised anomaly detection framework based on a heterogeneous feature network (HFN) for MTS. Specifically, we first combine the embedding similarity subgraph generated by sensor embedding and the feature value similarity subgraph generated by sensor values to construct a time-series heterogeneous graph, which fully utilizes the rich heterogeneous mutual information among variables. Then, a prediction model containing nodes and channel attentions is jointly optimized to obtain better time-series representations. This approach fuses the state-of-the-art technologies of heterogeneous graph structure learning (HGSL) and representation learning. Experimental results on four sensor datasets from real-world applications demonstrate that our approach achieves more accurate anomaly detection compared to baseline methods, laying a foundation for the rapid positioning of anomalies.

Learning deep news sentiment representations for macro-finance

AbstractThis paper introduces custom neural network techniques to the problem of latent economic factor extraction for voluminous news analytics data. In the context of macro-financial news, we derive low-dimensional representations of time series that arise in textual sentiment analyses spanning various topics. We explore three applications for compressed news sentiment data: nowcasting GDP growth, explaining asset class returns in a panel data analysis, and time series momentum investment. Our empirical study shows that nonlinear data representations based on supervised autoencoder architectures compare favorably to alternatives across all applications. In specific, we demonstrate that augmenting autoencoders with supervision tasks based on common asset class returns and market characteristics disciplines the dimension reduction and naturally supports the transparency of resulting representations. Taken together, our findings position supervised autoencoders as attractive competitor models alongside PCA and PLS approaches.

TimesURL: Self-Supervised Contrastive Learning for Universal Time Series Representation Learning

Learning universal time series representations applicable to various types of downstream tasks is challenging but valuable in real applications. Recently, researchers have attempted to leverage the success of self-supervised contrastive learning (SSCL) in Computer Vision(CV) and Natural Language Processing(NLP) to tackle time series representation. Nevertheless, due to the special temporal characteristics, relying solely on empirical guidance from other domains may be ineffective for time series and difficult to adapt to multiple downstream tasks. To this end, we review three parts involved in SSCL including 1) designing augmentation methods for positive pairs, 2) constructing (hard) negative pairs, and 3) designing SSCL loss. For 1) and 2), we find that unsuitable positive and negative pair construction may introduce inappropriate inductive biases, which neither preserve temporal properties nor provide sufficient discriminative features. For 3), just exploring segment- or instance-level semantics information is not enough for learning universal representation. To remedy the above issues, we propose a novel self-supervised framework named TimesURL. Specifically, we first introduce a frequency-temporal-based augmentation to keep the temporal property unchanged. And then, we construct double Universums as a special kind of hard negative to guide better contrastive learning. Additionally, we introduce time reconstruction as a joint optimization objective with contrastive learning to capture both segment-level and instance-level information. As a result, TimesURL can learn high-quality universal representations and achieve state-of-the-art performance in 6 different downstream tasks, including short- and long-term forecasting, imputation, classification, anomaly detection and transfer learning.

Energy Efficient Streaming Time Series Classification with Attentive Power Iteration

Efficiently processing time series data streams in real-time on resource-constrained devices offers significant advantages in terms of enhanced computational energy efficiency and reduced time-related risks. We introduce an innovative streaming time series classification network that utilizes attentive power iteration, enabling real-time processing on resource-constrained devices. Our model continuously updates a compact representation of the entire time series, enhancing classification accuracy while conserving energy and processing time. Notably, it excels in streaming scenarios without requiring complete time series access, enabling swift decisions. Experimental results show that our approach excels in classification accuracy and energy efficiency, with over 70% less consumption and threefold faster task completion than benchmarks. This work advances real-time responsiveness, energy conservation, and operational effectiveness for constrained devices, contributing to optimizing various applications.

JoLT: Jointly Learned Representations of Language and Time-Series for Clinical Time-Series Interpretation (Student Abstract)

Time-series and text data are prevalent in healthcare and frequently co-exist, yet they are typically modeled in isolation. Even studies that jointly model time-series and text, do so by converting time-series to images or graphs. We hypothesize that explicitly modeling time-series jointly with text can improve tasks such as summarization and question answering for time-series data, which have received little attention so far. To address this gap, we introduce JoLT to jointly learn desired representations from pre-trained time-series and text models. JoLT utilizes a Querying Transformer (Q-Former) to align the time-series and text representations. Our experiments on a large real-world electrocardiography dataset for medical time-series summarization show that JoLT outperforms state-of-the-art image captioning approaches.

Hidden Follower Detection: How Is the Gaze-Spacing Pattern Embodied in Frequency Domain?

Spatiotemporal social behavior analysis is a technique that studies the social behavior patterns of objects and estimates their risks based on their trajectories. In social public scenarios such as train stations, hidden following behavior has become one of the most challenging issues due to its probability of evolving into violent events, which is more than 25%. In recent years, research on hidden following detection (HFD) has focused on differences in time series between hidden followers and normal pedestrians under two temporal characteristics: gaze and spatial distance. However, the time-domain representation for time series is irreversible and usually causes the loss of critical information. In this paper, we deeply study the expression efficiency of time/frequency domain features of time series, by exploring the recovery mechanism of features to source time series, we establish a fidelity estimation method for feature expression and a selection model for frequency-domain features based on the signal-to-distortion ratio (SDR). Experimental results demonstrate the feature fidelity of time series and HFD performance are positively correlated, and the fidelity of frequency-domain features and HFD performance are significantly better than the time-domain features. On both real and simulated datasets, the accuracy of the proposed method is increased by 3%, and the gaze-only module is improved by 10%. Related research has explored new methods for optimal feature selection based on fidelity, new patterns for efficient feature expression of hidden following behavior, and the mechanism of multimodal collaborative identification.

Time Series Representation for Visualization in Apache IoTDB

When analyzing time series, often interactively, the analysts frequently demand to visualize instantly large-scale data stored in databases. M4 visualization selects the first, last, bottom and top data points in each pixel column to ensure pixel-perfectness of the two-color line chart visualization. While M4 already shows its preciseness of encasing time series in different scales into a fixed size of pixels, how to efficiently support M4 representation in a time series native database is still absent. It is worth noting that, to enable fast writes, the commodity time series database systems, such as Apache IoTDB or InfluxDB, employ LSM-Tree based storage. That is, a time series is segmented and stored in a number of chunks, with possibly out-of-order arrivals, i.e., disordered on timestamps. To implement M4, a natural idea is to merge online the chunks as a whole series, with costly merge sort on timestamps, and then perform M4 representation as in relational databases. In this study, we propose a novel chunk merge free approach called M4-LSM to accelerate M4 representation and visualization. In particular, we utilize the metadata of chunks to prune and avoid the costly merging of any chunk. Moreover, intra-chunk indexing and pruning are enabled for efficiently accessing the representation points, referring to the special properties of time series. Remarkably, the time series database native operator M4-LSM has been implemented in Apache IoTDB, an open-source time series database, and deployed in companies across various industries. In the experiments over real-world datasets, the proposed M4-LSM operator demonstrates high efficiency without sacrificing preciseness.

Multi-label classification with imbalanced classes by fuzzy deep neural networks

Multi-label classification is an advantageous technique for managing uncertainty in classification problems where each data instance is associated with several labels simultaneously. Such situations are frequent in real-world scenarios, where decisions rely on imprecise or noisy data and adaptable classification methods are preferred. However, the problem of class imbalance represents a common characteristic of several multi-label datasets, in which the distribution of samples and their corresponding labels is non-uniform across the data space. In this paper, we propose a multi-label classification approach utilizing fuzzy logic in order to deal with the class imbalance problem. To eliminate the need for an expert to determine the logical rules of inference, deep neural networks are adopted, which have proven to be exceptionally effective for such problems. By combining both fuzzy inference systems and deep neural networks, the strengths and weaknesses of each approach can be mitigated. As a further development, a symbolic representation of time series is put in place to reduce data dimensionality and speed up the training procedure. This allows for more flexibility in model application, in particular with respect to time constraints arising from the causality of observed time series. Tests carried out on a multi-label classification dataset related to the current and voltage profiles of several household appliances show that the proposed model outperforms four baseline models for time series classification.

Well googled is half done: Multimodal forecasting of new fashion product sales with image‐based google trends

AbstractNew fashion product sales forecasting is a challenging problem that involves many business dynamics and cannot be solved by classical forecasting approaches. In this paper, we investigate the effectiveness of systematically probing exogenous knowledge in the form of Google Trends time series and combining it with multi‐modal information related to a brand‐new fashion item, in order to effectively forecast its sales despite the lack of past data. In particular, we propose a neural network‐based approach, where an encoder learns a representation of the exogenous time series, while the decoder forecasts the sales based on the Google Trends encoding and the available visual and metadata information. Our model works in a non‐autoregressive manner, avoiding the compounding effect of large first‐step errors. As a second contribution, we present VISUELLE, a publicly available dataset for the task of new fashion product sales forecasting, containing multimodal information for 5,577 real, new products sold between 2016 and 2019 from Nunalie, an Italian fast‐fashion company. The dataset is equipped with images of products, metadata, related sales, and associated Google Trends. We use VISUELLE to compare our approach against state‐of‐the‐art alternatives and several baselines, showing that our neural network‐based approach is the most accurate in terms of both percentage and absolute error. It is worth noting that the addition of exogenous knowledge boosts the forecasting accuracy by 1.5% in terms of Weighted Absolute Percentage Error (WAPE), revealing the importance of exploiting informative external information. The code and dataset are both available online (at https://github.com/HumaticsLAB/GTM-Transformer).

TS-Fastformer: Fast Transformer for Time-series Forecasting

Many real-world applications require precise and fast time-series forecasting. Recent trends in time-series forecasting models are shifting from LSTM-based models to Transformer-based models. However, the Transformer-based model has a limited ability to represent sequential relationships in time-series data. In addition, the transformer-based model suffers from slow training and inference speed due to the bottleneck incurred by a deep encoder and step-by-step decoder inference. To address these problems, we propose a time-series forecasting optimized Transformer model, called TS-Fastformer. TS-Fastformer introduces three new optimizations: First, we propose a Sub Window Tokenizer for compressing input in a simple manner. The Sub Window Tokenizer reduces the length of input sequences to mitigate the complexity of self-attention and enables both single and multi-sequence learning. Second, we propose Time-series Pre-trained Encoder to extract effective representations through pre-training. This optimization enables TS-Fastformer to capture both seasonal and trend representations as well as to mitigate bottlenecks of conventional transformer models. Third, we propose the Past Attention Decoder to forecast target by incorporating past long short-term dependency patterns. Furthermore, Past Attention Decoder achieves high performance improvement by removing a trend distribution that changes over a long period. We evaluate the efficiency of our model with extensive experiments using seven real-world datasets and compare our model to six representative time-series forecasting approaches. The results show that the proposed TS-Fastformer reduces MSE by 10.1% compared to state-of-the-art model and demonstrates 21.6% faster training time compared to the existing fastest transformer, respectively.

Universal representation learning for multivariate time series using the instance-level and cluster-level supervised contrastive learning

AbstractThe multivariate time series classification (MTSC) task aims to predict a class label for a given time series. Recently, modern deep learning-based approaches have achieved promising performance over traditional methods for MTSC tasks. The success of these approaches relies on access to the massive amount of labeled data (i.e., annotating or assigning tags to each sample that shows its corresponding category). However, obtaining a massive amount of labeled data is usually very time-consuming and expensive in many real-world applications such as medicine, because it requires domain experts’ knowledge to annotate data. Insufficient labeled data prevents these models from learning discriminative features, resulting in poor margins that reduce generalization performance. To address this challenge, we propose a novel approach: supervised contrastive learning for time series classification (SupCon-TSC). This approach improves the classification performance by learning the discriminative low-dimensional representations of multivariate time series, and its end-to-end structure allows for interpretable outcomes. It is based on supervised contrastive (SupCon) loss to learn the inherent structure of multivariate time series. First, two separate augmentation families, including strong and weak augmentation methods, are utilized to generate augmented data for the source and target networks, respectively. Second, we propose the instance-level, and cluster-level SupCon learning approaches to capture contextual information to learn the discriminative and universal representation for multivariate time series datasets. In the instance-level SupCon learning approach, for each given anchor instance that comes from the source network, the low-variance output encodings from the target network are sampled as positive and negative instances based on their labels. However, the cluster-level approach is performed between each instance and cluster centers among batches, as opposed to the instance-level approach. The cluster-level SupCon loss attempts to maximize the similarities between each instance and cluster centers among batches. We tested this novel approach on two small cardiopulmonary exercise testing (CPET) datasets and the real-world UEA Multivariate time series archive. The results of the SupCon-TSC model on CPET datasets indicate its capability to learn more discriminative features than existing approaches in situations where the size of the dataset is small. Moreover, the results on the UEA archive show that training a classifier on top of the universal representation features learned by our proposed method outperforms the state-of-the-art approaches.

Identifying baseflow source areas using remotely sensed and ground‐based hydrologic data

AbstractUnderstanding how rainfall and snowmelt influence baseflow, the groundwater‐fed component of streamflow, is essential for sound water resources management. Current approaches to understand the spatial couplings between these processes and baseflow are limited. The most commonly used methods include geochemical tracers and hydrologic models. A key limitation of the first is cost, while the second is limited by the need for simplifying assumptions. This study developed a data‐driven approach which leverages satellite Earth observation data and ground‐based data to assess the degree to which baseflow is influenced by upstream rainfall and snowmelt in California's Sierra Nevada. The procedure involved: (1) separation of baseflow from streamflow time series using a low‐pass filtering technique, (2) quantification of aquifer drainage timescales through baseflow recession analysis, (3) application of time series and information theory methods to identify the areas which have the greatest impacts on baseflow through both rainfall and snowmelt, and (4) characterization of the elevation zones which have a prevailing influence on baseflow. Results suggest that areas which have the strongest impact on baseflow through rainfall and snowmelt are not necessarily the areas which experience the highest annual rates of snowmelt or rainfall; snowmelt occurring in the 3000–3700 m elevation range was found to be the most important driver of baseflow.

An Application of Fuzzy Symbolic Time-Series for Energy Demand Forecasting

In this paper, we present a new fuzzy symbolization technique for energy load forecasting with neural networks, FPLS-Sym. Symbolization techniques transform a numerical time series into a smaller string of symbols, providing a high-level representation of time series by combining segmentation, aggregation and discretization. The dimensional reduction obtained with symbolization can speed up substantially the time required to train neural networks, however, it can also lead to considerable information losses that could lead to a less accurate forecast. FPLS-Sym introduces the use of fuzzy logic in the discretization process, maintaining more information about each segment of the neural network at the expense of requiring more space in memory. Extensive experimentation was made to evaluate FPLS-Sym with various neural-network-based models, including different neural network architectures and activation functions. The evaluation was done with energy demand data from Spain taken from 2009 to 2019. Results show that FPLS-Sym provides better quality metrics than other symbolization techniques and outperforms the use of the standard numerical time series representation in both quality metrics and training time.

Statistical characterization of erosion and sediment transport mechanics in shallow tidal environments – Part 2: Suspended sediment dynamics

Abstract. A proper understanding of sediment resuspension and transport processes is key to the morphodynamics of shallow tidal environments. However, a complete spatial and temporal coverage of suspended sediment concentration (SSC) to describe these processes is hardly available, preventing the effective representation of depositional dynamics in long-term modelling approaches. Through aiming to couple erosion and deposition dynamics in a unique synthetic theoretical framework, we here investigate SSC dynamics, following a similar approach to that adopted for erosion (D'Alpaos et al., 2024). The analysis with the peak-over-threshold theory of SSC time series computed using a fully coupled, bi-dimensional model allows us to identify interarrival times, intensities, and durations of over-threshold events and test the hypothesis of modelling SSC dynamics as a Poisson process. The effects of morphological modifications on spatial and temporal SSC patterns are investigated in the Venice Lagoon, for which several historical configurations in the last 4 centuries are available. Our results show that, similar to erosion events, SSC can be modelled as a marked Poisson process in the intertidal flats for all the analysed morphological lagoon configurations because exponentially distributed random variables describe over-threshold events well. Although erosion and resuspension are intimately intertwined, erosion alone does not suffice to describe also SSC because of the non-local dynamics due to advection and dispersion processes. The statistical characterization of SSC events completes the framework introduced for erosion mechanics, and together, they represent a promising tool to generate synthetic, yet realistic, time series of shear stress and SSC for the long-term modelling of tidal environments.