Learning For Time Series Classification Research Articles

Series2vec: similarity-based self-supervised representation learning for time series classification

We argue that time series analysis is fundamentally different in nature to either vision or natural language processing with respect to the forms of meaningful self-supervised learning tasks that can be defined. Motivated by this insight, we introduce a novel approach called Series2Vec for self-supervised representation learning. Unlike the state-of-the-art methods in time series which rely on hand-crafted data augmentation, Series2Vec is trained by predicting the similarity between two series in both temporal and spectral domains through a self-supervised task. By leveraging the similarity prediction task, which has inherent meaning for a wide range of time series analysis tasks, Series2Vec eliminates the need for hand-crafted data augmentation. To further enforce the network to learn similar representations for similar time series, we propose a novel approach that applies order-invariant attention to each representation within the batch during training. Our evaluation of Series2Vec on nine large real-world datasets, along with the UCR/UEA archive, shows enhanced performance compared to current state-of-the-art self-supervised techniques for time series. Additionally, our extensive experiments show that Series2Vec performs comparably with fully supervised training and offers high efficiency in datasets with limited-labeled data. Finally, we show that the fusion of Series2Vec with other representation learning models leads to enhanced performance for time series classification. Code and models are open-source at https://github.com/Navidfoumani/Series2Vec

Deep Learning for Time Series Classification and Extrinsic Regression: A Current Survey

Time Series Classification and Extrinsic Regression are important and challenging machine learning tasks. Deep learning has revolutionized natural language processing and computer vision and holds great promise in other fields such as time series analysis where the relevant features must often be abstracted from the raw data but are not known a priori. This article surveys the current state of the art in the fast-moving field of deep learning for time series classification and extrinsic regression. We review different network architectures and training methods used for these tasks and discuss the challenges and opportunities when applying deep learning to time series data. We also summarize two critical applications of time series classification and extrinsic regression, human activity recognition and satellite earth observation.

Meta-Transfer Metric Learning for Time Series Classification in 6G-Supported Intelligent Transportation Systems

Deep learning-based time series classification in 6G-supported Intelligent Transportation Systems (ITS) helps transport decision-making. Deep learning classifier training necessitates a large amount of labeled data for feature extraction. Labeling time series data in 6G-supported ITS is tough. Meta-learning can be used to train deep classifiers with limited data. However, in meta-learning, the tasks are frequently modeled by a low-complexity base learner. It is unable to use more complicate and powerful structures. The meta-learning-pretrained classifier can only perform new classification problems with the same number of classes. Most pre-training strategies do not prioritize enhancing the pre-training phase’s convergence rate and lowering the computational cost. Most research work aims to improve classification performance by increasing the complexity of the classification model. However, this raises computing costs. In this paper, we propose a one-dimensional Multi-Scale Dilated Convolution Neural Network time series classifier (MSDCNN). MSDCNN combines multi-scale CNN and dilated CNN. It can extract multi-scale characteristics from time series and reduce the complexity of the classifier. Furthermore, we propose a pre-training strategy, called Meta-transfer metric Learning using Scale function (MLS). MLS allows the classifier to gain experience from different tasks with various numbers of classes. Experiments show that MLS reduces pre-training computation costs during the pre-training phase. The pre-trained classifier, without using any fine tuning techniques, achieves the highest accuracy by comparing with the state-of-the-art methods. Finally, we present a case study of applying MSDCNN and MLS to detect road accidents in 6G-supported transportation systems.

Universal representation learning for multivariate time series using the instance-level and cluster-level supervised contrastive learning

AbstractThe multivariate time series classification (MTSC) task aims to predict a class label for a given time series. Recently, modern deep learning-based approaches have achieved promising performance over traditional methods for MTSC tasks. The success of these approaches relies on access to the massive amount of labeled data (i.e., annotating or assigning tags to each sample that shows its corresponding category). However, obtaining a massive amount of labeled data is usually very time-consuming and expensive in many real-world applications such as medicine, because it requires domain experts’ knowledge to annotate data. Insufficient labeled data prevents these models from learning discriminative features, resulting in poor margins that reduce generalization performance. To address this challenge, we propose a novel approach: supervised contrastive learning for time series classification (SupCon-TSC). This approach improves the classification performance by learning the discriminative low-dimensional representations of multivariate time series, and its end-to-end structure allows for interpretable outcomes. It is based on supervised contrastive (SupCon) loss to learn the inherent structure of multivariate time series. First, two separate augmentation families, including strong and weak augmentation methods, are utilized to generate augmented data for the source and target networks, respectively. Second, we propose the instance-level, and cluster-level SupCon learning approaches to capture contextual information to learn the discriminative and universal representation for multivariate time series datasets. In the instance-level SupCon learning approach, for each given anchor instance that comes from the source network, the low-variance output encodings from the target network are sampled as positive and negative instances based on their labels. However, the cluster-level approach is performed between each instance and cluster centers among batches, as opposed to the instance-level approach. The cluster-level SupCon loss attempts to maximize the similarities between each instance and cluster centers among batches. We tested this novel approach on two small cardiopulmonary exercise testing (CPET) datasets and the real-world UEA Multivariate time series archive. The results of the SupCon-TSC model on CPET datasets indicate its capability to learn more discriminative features than existing approaches in situations where the size of the dataset is small. Moreover, the results on the UEA archive show that training a classifier on top of the universal representation features learned by our proposed method outperforms the state-of-the-art approaches.

Energy-efficient Online Continual Learning for Time Series Classification in Nanorobot-based Smart Health.

Nanorobots have been used in smart health to collect time series data such as electrocardiograms and electroencephalograms. Real-time classification of dynamic time series signals in nanorobots is a challenging task. Nanorobots in the nanoscale range require a classification algorithm with low computational complexity. First, the classification algorithm should be able to dynamically analyze time series signals and update itself to process the concept drifts (CD). Second, the classification algorithm should have the ability to handle catastrophic forgetting (CF) and classify historical data. Most importantly, the classification algorithm should be energy-efficient to use less computing power and memory to classify signals in real-time on a smart nanorobot. To solve these challenges, we design an algorithm that can Prevent Concept Drift in Online continual Learning for time series classification (PCDOL). The prototype suppression item in PCDOL can reduce the impact caused by CD. It also solves the CF problem through the replay feature. The computation per second and the memory consumed by PCDOL are only 3.572M and 1KB, respectively. The experimental results show that PCDOL is better than several state-of-the-art methods for dealing with CD and CF in energy-efficient nanorobots.

Semi-Supervised Contrastive Learning for Time Series Classification in Healthcare

Semi-Supervised Contrastive Learning for Time Series Classification in Healthcare

PIP: Pictorial Interpretable Prototype Learning for Time Series Classification.

Time series classifiers are not only challenging to design, but they are also notoriously difficult to deploy for critical applications because end users may not understand or trust black-box models. Despite new efforts, explanations generated by other interpretable time series models are complicated for non-engineers to understand. The goal of PIP is to provide time series explanations that are tailored toward specific end users. To address the challenge, this paper introduces PIP, a novel deep learning architecture that jointly learns classification models and meaningful visual class prototypes. PIP allows users to train the model on their choice of class illustrations. Thus, PIP can create a user-friendly explanation by leaning on end-users definitions. We hypothesize that a pictorial description is an effective way to communicate a learned concept to non-expert users. Based on an end-user experiment with participants from multiple backgrounds, PIP offers an improved combination of accuracy and interpretability over baseline methods for time series classification.

Deep Multiple Metric Learning for Time Series Classification

Effective distance metric plays an important role in time series classification. Metric learning, which aims to learn a data-adaptive distance metric to measure the distance among samples, has achieved promising results on time series classification. However, most existing approaches focus on learning a single linear metric, which is unsuitable for nonlinear relationships and heterogeneous datasets with locality information. Besides, the hard samples in the training set account for only a small part, which may fail to characterize the global geometry of the metric embedding space. In this paper, we propose a novel deep multiple metric learning (DMML) method for time series classification. DMML contains a convolutional network component to extract nonlinear features of time series. For exploiting locality information, the last feature layer of the convolutional network is divided into several nonoverlapping groups and a separate metric learner is built on each group to get multiple metrics. In order to reduce the correlations among learners and facilitate robust metric learning, we design an adversarial negative generator to synthesize different hard negative complements for different metric learners. Moreover, an auxiliary loss is introduced to increase the robustness of DMML for the magnitude of distance. Extensive experiments on UCR datasets demonstrate the effectiveness of DMML for time series classification.

Temporal representation learning for time series classification

Recent years have witnessed the exponential growth of time series data as the popularity of sensing devices and development of IoT techniques; time series classification has been considered as one of the most challenging studies in time series data mining, attracting great interest over the last two decades. According to the empirical evidences, temporal representation learning-based time series classification has more superiority of accuracy, efficiency and interpretability as compared to hundreds of existing time series classification methods. However, due to the high time complexity of feature process, the performance of these methods has been severely restricted. In this paper, we first presented an efficient shapelet transformation method to improve the overall efficiency of time series classification, and then, we further developed a novel enhanced recurrent neural network model for deep representation learning to further improve the classification accuracy. Experimental results on typical real-world datasets have justified the superiority of our models over several shallow and deep representation learning competitors.

Deep learning for time series classification: a review

Time Series Classification (TSC) is an important and challenging problem in data mining. With the increase of time series data availability, hundreds of TSC algorithms have been proposed. Among these methods, only a few have considered Deep Neural Networks (DNNs) to perform this task. This is surprising as deep learning has seen very successful applications in the last years. DNNs have indeed revolutionized the field of computer vision especially with the advent of novel deeper architectures such as Residual and Convolutional Neural Networks. Apart from images, sequential data such as text and audio can also be processed with DNNs to reach state-of-the-art performance for document classification and speech recognition. In this article, we study the current state-of-the-art performance of deep learning algorithms for TSC by presenting an empirical study of the most recent DNN architectures for TSC. We give an overview of the most successful deep learning applications in various time series domains under a unified taxonomy of DNNs for TSC. We also provide an open source deep learning framework to the TSC community where we implemented each of the compared approaches and evaluated them on a univariate TSC benchmark (the UCR/UEA archive) and 12 multivariate time series datasets. By training 8,730 deep learning models on 97 time series datasets, we propose the most exhaustive study of DNNs for TSC to date.