Multivariate Time Series Classification Research Articles

Time and frequency-domain feature fusion network for multivariate time series classification

Multivariate time series classification is a significant research topic in the realm of data mining, which encompasses a wide array of practical applications in domains such as healthcare, energy systems, and traffic. The complex temporal and spatial dependencies inherent in multivariate time series pose challenges to classification tasks. Previous studies have usually focused on the time-domain information of multivariate time series. However, achieving accurate classification using only the time-domain information may be difficult. To address this challenge, a time and frequency-domain feature fusion network (TF-Net) for multivariate time series classification is proposed in this paper. Our model contains two modules, the time-domain module and the frequency-domain module. The time-domain module is used to capture the time-domain features of multivariate time series. It is constructed using CNNs and GCNs, enabling the capture of both temporal and spatial dependencies within the time-domain. The frequency-domain module is used to capture the frequency-domain features of multivariate time series data. In this module, we treat the frequency-domain features as images and innovatively transform the multivariate time series classification task into an image classification task. Our method is able to classify multivariate time series from both the time-domain and frequency-domain, which provides a new perspective for multivariate time series analysis. We conduct extensive experiments on the UAE archive, and the experimental results show that our model achieves the best performance compared to the ten state-of-the-art methods.

SEGAL time series classification — Stable explanations using a generative model and an adaptive weighting method for LIME

Local Interpretability Model-agnostic Explanations (LIME) is a well-known post-hoc technique for explaining black-box models. While very useful, recent research highlights challenges around the explanations generated. In particular, there is a potential lack of stability, where the explanations provided vary over repeated runs of the algorithm, casting doubt on their reliability. This paper investigates the stability of LIME when applied to multivariate time series classification. We demonstrate that the traditional methods for generating neighbours used in LIME carry a high risk of creating ‘fake’ neighbours, which are out-of-distribution in respect to the trained model and far away from the input to be explained. This risk is particularly pronounced for time series data because of their substantial temporal dependencies. We discuss how these out-of-distribution neighbours contribute to unstable explanations. Furthermore, LIME weights neighbours based on user-defined hyperparameters which are problem-dependent and hard to tune. We show how unsuitable hyperparameters can impact the stability of explanations. We propose a two-fold approach to address these issues. First, a generative model is employed to approximate the distribution of the training data set, from which within-distribution samples and thus meaningful neighbours can be created for LIME. Second, an adaptive weighting method is designed in which the hyperparameters are easier to tune than those of the traditional method. Experiments on real-world data sets demonstrate the effectiveness of the proposed method in providing more stable explanations using the LIME framework. In addition, in-depth discussions are provided on the reasons behind these results.

MTS2Graph: Interpretable multivariate time series classification with temporal evolving graphs

Conventional time series classification approaches based on bags of patterns or shapelets face significant challenges in dealing with a vast amount of feature candidates from high-dimensional multivariate data. In contrast, deep neural networks can learn low-dimensional features efficiently, and in particular, convolutional neural networks have shown promising results in classifying multivariate time series data. A key factor in the success of deep neural networks is this astonishing expressive power. However, this power comes at the cost of complex, black-boxed models, conflicting with the goals of building reliable and human-understandable models. In this work11The code can be found at https://github.com/raneeny/MTS2Graph., we introduce a new interpretable framework for multivariate time series data that by extracting and clustering the input quantifies the contribution of time-varying input variables and each signal’s role to the classification. We construct a graph that captures the temporal relationship between the extracted patterns for each layer and propose an effective merging strategy to aggregate those graphs into one. Finally, a graph embedding algorithm generates new representations of the created interpretable time-series features. Our extensive experiments indicate the benefit of our time-aware graph-based representation in multivariate time series classification while enriching them with more interpretability.

Graph-Aware Contrasting for Multivariate Time-Series Classification

Contrastive learning, as a self-supervised learning paradigm, becomes popular for Multivariate Time-Series (MTS) classification. It ensures the consistency across different views of unlabeled samples and then learns effective representations for these samples. Existing contrastive learning methods mainly focus on achieving temporal consistency with temporal augmentation and contrasting techniques, aiming to preserve temporal patterns against perturbations for MTS data. However, they overlook spatial consistency that requires the stability of individual sensors and their correlations. As MTS data typically originate from multiple sensors, ensuring spatial consistency becomes essential for the overall performance of contrastive learning on MTS data. Thus, we propose Graph-Aware Contrasting for spatial consistency across MTS data. Specifically, we propose graph augmentations including node and edge augmentations to preserve the stability of sensors and their correlations, followed by graph contrasting with both node- and graph-level contrasting to extract robust sensor- and global-level features. We further introduce multi-window temporal contrasting to ensure temporal consistency in the data for each sensor. Extensive experiments demonstrate that our proposed method achieves state-of-the-art performance on various MTS classification tasks. The code is available at https://github.com/Frank-Wang-oss/TS-GAC.

DSDCLNet: Dual-stream encoder and dual-level contrastive learning network for supervised multivariate time series classification

In recent years, deep learning approaches have shown remarkable advancements in Multivariate Time Series Classification (MTSC) tasks. However, the existing approaches primarily focus on capturing the long-term correlations of time series or identifying local key sequence fragments, inevitably neglecting the synergistic properties between global and local components. Additionally, most representation learning methods for MTSC rely on self-supervised learning, which limits their ability to fully exploit label information. Hence, this paper proposes a novel approach termed Dual-Stream Encoder and Dual-Level Contrastive Learning Network (DSDCLNet), which integrates a Dual-Stream Encoder (DSE) and Dual-level Contrastive Learning (DCL). First, to extract multiscale local-global features from multivariate time series data, we employ a DSE architecture comprising an Attention-Gated Recurrent Unit (AGRU) and a Dual-layer Multiscale Convolutional Neural Network (DMSCNN). Specifically, DMSCNN consists of a series of multi-scale convolutional layers and a max pooling layer. Second, to maximize the utilization of label information, a new loss function is designed, which combines classification loss, instance-level contrastive loss, and temporal-level contrastive loss. Finally, experiments are conducted on the UEA datasets and the results demonstrate that DSDCLNet achieves the highest average accuracy of 0.77, outperforming traditional approaches, deep learning approaches, and self-supervised approaches on 30, 23, and 27 datasets, respectively.

Monitoring Flow-Forming Processes Using Design of Experiments and a Machine Learning Approach Based on Randomized-Supervised Time Series Forest and Recursive Feature Elimination.

The machines of WF Maschinenbau process metal blanks into various workpieces using so-called flow-forming processes. The quality of these workpieces depends largely on the quality of the blanks and the condition of the machine. This creates an urgent need for automated monitoring of the forming processes and the condition of the machine. Since the complexity of the flow-forming processes makes physical modeling impossible, the present work deals with data-driven modeling using machine learning algorithms. The main contributions of this work lie in showcasing the feasibility of utilizing machine learning and sensor data to monitor flow-forming processes, along with developing a practical approach for this purpose. The approach includes an experimental design capable of providing the necessary data, as well as a procedure for preprocessing the data and extracting features that capture the information needed by the machine learning models to detect defects in the blank and the machine. To make efficient use of the small number of experiments available, the experimental design is generated using Design of Experiments methods. They consist of two parts. In the first part, a pre-selection of influencing variables relevant to the forming process is performed. In the second part of the design, the selected variables are investigated in more detail. The preprocessing procedure consists of feature engineering, feature extraction and feature selection. In the feature engineering step, the data set is augmented with time series variables that are meaningful in the domain. For feature extraction, an algorithm was developed based on the mechanisms of the r-STSF, a state-of-the-art algorithm for time series classification, extending them for multivariate time series and metric target variables. This feature extraction algorithm itself can be seen as an additional contribution of this work, because it is not tied to the application domain of monitoring flow-forming processes, but can be used as a feature extraction algorithm for multivariate time series classification in general. For feature selection, a Recursive Feature Elimination is employed. With the resulting features, random forests are trained to detect several quality features of the blank and defects of the machine. The trained models achieve good prediction accuracy for most of the target variables. This shows that the application of machine learning is a promising approach for the monitoring of flow-forming processes, which requires further investigation for confirmation.

Breast cancer classification through multivariate radiomic time series analysis in DCE-MRI sequences

Breast cancer is the most prevalent disease that poses a significant threat to women’s health. Despite the Dynamic Contrast-Enhanced MRI (DCE-MRI) has been widely used for breast cancer classification, its diagnostic performance is still suboptimal. In this work, the Radiomic workflow was implemented to classify the whole DCE-MRI sequence based on the distinction in contrast agent uptake between benign and malignant lesions. The radiomic features extracted from each of the seven time instants within the DCE-MRI sequence were fed into a multi-instant features selection strategy to select the discriminative features for time series classification. Several time series classification algorithms including Rocket, MultiRocket, K-Nearest Neighbor, Time Series Forest, and Supervised Time Series Forest were compared. Firstly, a univariate classification was performed to find the five most informative radiomic series, and then, a multivariate time series classification was implemented via a voting mechanism. The Multivariate Rocket model was the most accurate (Accuracy = 0.852, AUC-ROC = 0.852, Specificity = 0.823, Sensitivity = 0.882). The intelligible radiomic features enabled model findings explanations and clinical validation. In particular, the Energy and TotalEnergy were among the most important features, and the most descriptive for the change in signal intensity, which is the main effect of the contrast agent.

Design of an Integrated Arrhythmia Detection Model using Connectivity Features and Multivariate Time Series Classification

Cardiac arrhythmia, characterized by irregular heart rhythms, represents a widespread concern within the realm of cardiology. It encompasses a range of rhythm irregularities, with some being benign and others carrying substantial health risks. Therefore, the timely detection of arrhythmia holds considerable importance. Existing methods to detect arrhythmia mainly utilize either the traditional machine learning classifiers like SVM, and random forest or the recent deep learning-based models like CNN, LSTM, and RNN for the classification while few other methods use the classical signal processing-based transforms to extract the discriminating features. This paper proposes a novel integrated approach to classify the ECG signals for arrhythmia detection. Unlike existing methods, it considers the multivariate time series nature of the input along with the interrelationships among different ECG leads. The approach utilizes multivariate time series features extracted using ROCKET (RandOM Convolutional KErnal Transform) and introduces new connectivity-based features such as correlation and coherence for improved ECG signal classification. The state-of-the-art classification performance of the proposed integrated model on the PTB-XL PhysioNet dataset attested to the efficacy of the same.

Feature Selection-Based Multiview Concentration for Multivariate Time Series Classification and Its Application

Feature Selection-Based Multiview Concentration for Multivariate Time Series Classification and Its Application

Universal representation learning for multivariate time series using the instance-level and cluster-level supervised contrastive learning

AbstractThe multivariate time series classification (MTSC) task aims to predict a class label for a given time series. Recently, modern deep learning-based approaches have achieved promising performance over traditional methods for MTSC tasks. The success of these approaches relies on access to the massive amount of labeled data (i.e., annotating or assigning tags to each sample that shows its corresponding category). However, obtaining a massive amount of labeled data is usually very time-consuming and expensive in many real-world applications such as medicine, because it requires domain experts’ knowledge to annotate data. Insufficient labeled data prevents these models from learning discriminative features, resulting in poor margins that reduce generalization performance. To address this challenge, we propose a novel approach: supervised contrastive learning for time series classification (SupCon-TSC). This approach improves the classification performance by learning the discriminative low-dimensional representations of multivariate time series, and its end-to-end structure allows for interpretable outcomes. It is based on supervised contrastive (SupCon) loss to learn the inherent structure of multivariate time series. First, two separate augmentation families, including strong and weak augmentation methods, are utilized to generate augmented data for the source and target networks, respectively. Second, we propose the instance-level, and cluster-level SupCon learning approaches to capture contextual information to learn the discriminative and universal representation for multivariate time series datasets. In the instance-level SupCon learning approach, for each given anchor instance that comes from the source network, the low-variance output encodings from the target network are sampled as positive and negative instances based on their labels. However, the cluster-level approach is performed between each instance and cluster centers among batches, as opposed to the instance-level approach. The cluster-level SupCon loss attempts to maximize the similarities between each instance and cluster centers among batches. We tested this novel approach on two small cardiopulmonary exercise testing (CPET) datasets and the real-world UEA Multivariate time series archive. The results of the SupCon-TSC model on CPET datasets indicate its capability to learn more discriminative features than existing approaches in situations where the size of the dataset is small. Moreover, the results on the UEA archive show that training a classifier on top of the universal representation features learned by our proposed method outperforms the state-of-the-art approaches.

Multivariate time series classification based on fusion features

In various areas of real life, Multivariate Time Series Classification (MTSC) is widely used. It has been the focus of attention of many researchers, and a number of MTSC methods have been proposed in recent years. These methods tend to focus on the features in only a single domain. However, they have ignored the correlation and complementarity between the features in a space of multiple domains. In this paper, a novel MTSC method based on fusion features (MTSC_FF) is presented to address this problem. Firstly, MTSC_FF extracts the frequency domain features using an attention layer based on continuous wavelet transform. In parallel, MTSC_FF extracts long-range dependency features from the time domain, using a sparse self-attention layer. Simultaneously, MTSC_FF obtains spatial correlations between multivariate time series dimensions via Kendall coefficient. Then, a graph neural network is used to fuse all features. Finally, the fusion features are used to predict the classification labels by means of the fully connected layer. The experimental results obtained on the UEA datasets show that the proposed method can achieve high accuracy. In addition, the proposed method can visualize the classification-dependent features, which is an improvement in the interpretability of the results.

Design of an Integrated Myocardial Infarction Detection Model Using ECG Connectivity Features and Multivariate Time Series Classification

Design of an Integrated Myocardial Infarction Detection Model Using ECG Connectivity Features and Multivariate Time Series Classification

Ensemble Predictors: Possibilistic Combination of Conformal Predictors for Multivariate Time Series Classification.

In this article we propose a conceptual framework to study ensembles of conformal predictors (CP), that we call Ensemble Predictors (EP). Our approach is inspired by the application of imprecise probabilities in information fusion. Based on the proposed framework, we study, for the first time in the literature, the theoretical properties of CP ensembles in a general setting, by focusing on simple and commonly used possibilistic combination rules. We also illustrate the applicability of the proposed methods in the setting of multivariate time-series classification, showing that these methods provide better performance (in terms of both robustness, conservativeness, accuracy and running time) than both standard classification algorithms and other combination rules proposed in the literature, on a large set of benchmarks from the UCR time series archive.

Time series classification with their image representation

The study is concerned with the problem of classification of multivariate time series using convolutional neural networks (CNNs). As CNNs regard inputs in the form of images, an original image-like format of temporal data is proposed. Along this line, several design alternatives are studied by forming images with the two corresponding coordinates built by the original temporal data and their differences and second differences. An overall design process is presented with a focus on investigating time series-image transformations. Experimental studies involving publicly available data sets are reported, along with a slew of comparative analyses.

Persistent Homology-Based Classification of Chaotic Multi-variate Time Series: Application to Electroencephalograms

In this work, we present a combination of dimension reduction techniques and persistent homology for detection of epileptic events in electroencephalograms for a special kind of epilepsy called petit-mal epilepsy. Persistent homology, one of the main methods in topological data analysis, extracts information about the structures appearing in a given data set. Since during an epileptic seizure of the above type the electrical brain activity is more synchronized, we take the resulting structure in the EEG signal as classification feature that is analyzed topologically by means of persistent homology. As preprocessing step, the dimension of the data is reduced by two alternative techniques, principal component analysis and dynamical component analysis, and their performance is compared. Our results show that in comparison to principal component analysis, dynamical component analysis captures the dynamics of the system when projected onto a low-dimensional subspace. Furthermore, the results prove that persistent homology is well-suited for the detection of petit-mal epileptic seizures by means of their inherent structure.

Dyformer: A dynamic transformer-based architecture for multivariate time series classification

Multivariate time series classification is a crucial task with applications in broad areas such as finance, medicine, and engineering. Transformer is promising for time series classification, but as a generic approach, they have limited capability to effectively capture the distinctive characteristics inherent in time series data and adapt to diverse architectural requirements. This paper proposes a novel dynamic transformer-based architecture called Dyformer to address the above limitations of traditional transformers in multivariate time series classification. Dyformer incorporates hierarchical pooling to decompose time series into subsequences with different frequency components. Then, it employs Dyformer modules to achieve adaptive learning strategies for different frequency components based on a dynamic architecture. Furthermore, we introduce feature-map-wise attention mechanisms to capture multi-scale temporal dependencies and a joint loss function to facilitate model training. To evaluate the performance of Dyformer, we conducted extensive experiments using 30 benchmark datasets. The results unequivocally demonstrate that our model consistently outperforms a multitude of state-of-the-art methods and baseline approaches. Our model also copes well with limited training samples when pre-trained.

Multivariate time-series classification using memory and attention for long and short-term dependence$$^{\star }$$

Multivariate time-series classification using memory and attention for long and short-term dependence$$^{\star }$$

Neural fingerprinting on MEG time series using MiniRocket.

Neural fingerprinting is the identification of individuals in a cohort based on neuroimaging recordings of brain activity. In magneto- and electroencephalography (M/EEG), it is common practice to use second-order statistical measures, such as correlation or connectivity matrices, when neural fingerprinting is performed. These measures or features typically require coupling between signal channels and often ignore the individual temporal dynamics. In this study, we show that, following recent advances in multivariate time series classification, such as the development of the RandOm Convolutional KErnel Transformation (ROCKET) classifier, it is possible to perform classification directly on short time segments from MEG resting-state recordings with remarkably high classification accuracies. In a cohort of 124 subjects, it was possible to assign windows of time series of 1 s in duration to the correct subject with above 99% accuracy. The achieved accuracies are vastly superior to those of previous methods while simultaneously requiring considerably shorter time segments.

Network intrusion detection based on the temporal convolutional model

Recurrent networks have been adopted as default architecture in approaches performing sequence modelling of network intrusion detection problems. However, models based on Recurrent Neural Network (RNN) architecture have some limitations regarding computation complexity and detection performance due to information loss. Recent studies suggested that certain convolutional-based architectures that can natively process temporal data outperform recurrent networks in time series modelling, such as speech synthesis and machine translation. Inspired by the success of temporal-based convolutional architectures in other domains, this paper designs and implements four temporal-based convolutional models in network intrusion detection and studies their classification performance. The models are based on MINImally RandOm Convolutional KErnel Transform (MiniRocket), eXplainable Convolutional Neural Network for Multivariate Time Series Classification (XCM), One-Dimensional Convolution Neural Network (OS-CNN), and Time Series Transformer (TST) architecture. We evaluate our models using the effectiveness and efficiency metrics based on CICDDoS2019 and CSE-CIC-IDS2018 datasets. Every model has achieved a high evaluation performance in the range between 98.07% and 99.99% in most considered metrics using the test dataset. MiniRocket and OS-CNN obtained the highest evaluation results regarding the effectiveness metrics. The high evaluation results suggest that they can improve the detection effectiveness of approaches formulating network intrusion detection as a time series task.

Multivariate Time-Series Classification of Critical Events from Industrial Drying Hopper Operations: A Deep Learning Approach

In recent years, the advancement of Industry 4.0 and smart manufacturing has made a large amount of industrial process data attainable with the use of sensors installed on machines. This paper proposes an experimental predictive maintenance framework for an industrial drying hopper so that it can detect any unusual event in the hopper, which reduces the risk of erroneous fault diagnosis in the manufacturing shop floor. The experimental framework uses Deep Learning (DL) algorithms to classify Multivariate Time-Series (MTS) data into two categories—failure or unusual events and regular events—thus formulating the problem as a binary classification. The raw data extracted from the sensors contained missing values, suffered from imbalancedness, and were not labeled. Therefore, necessary preprocessing is performed to make them usable for DL algorithms and the dataset is self-labeled after defining the two categories precisely. To tackle the imbalanced data issue, data balancing techniques like ensemble learning with undersampling and Synthetic Minority Oversampling Technique (SMOTE) are used. Moreover, along with DL algorithms like Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM), Machine Learning (ML) algorithms like Support Vector Machine (SVM) and K-nearest neighbor (KNN) have also been used to perform a comparative analysis on the results obtained from these algorithms. The result shows that CNN is arguably the best algorithm for classifying this dataset into two categories and outperforms other traditional approaches as well as deep learning algorithms.