Time Series Classification Methods Research Articles

Quant: a minimalist interval method for time series classification

AbstractWe show that it is possible to achieve the same accuracy, on average, as the most accurate existing interval methods for time series classification on a standard set of benchmark datasets using a single type of feature (quantiles), fixed intervals, and an ‘off the shelf’ classifier. This distillation of interval-based approaches represents a fast and accurate method for time series classification, achieving state-of-the-art accuracy on the expanded set of 142 datasets in the UCR archive with a total compute time (training and inference) of less than 15 min using a single CPU core.

A Fuzzy-Probabilistic Representation Learning Method for Time Series Classification

A Fuzzy-Probabilistic Representation Learning Method for Time Series Classification

A small-sample time-series signal augmentation and analysis method for quantitative assessment of bradykinesia in Parkinson's disease

Patients with Parkinson's disease (PD) usually have varying degrees of bradykinesia, and the current clinical assessment is mainly based on the Movement Disorder Society Unified PD Rating Scale, which can hardly meet the needs of objectivity and accuracy. Therefore, this paper proposed a small-sample time series classification method (DTW-TapNet) based on dynamic time warping (DTW) data augmentation and attentional prototype network. Firstly, for the problem of small sample sizes of clinical data, a DTW-based data merge method is used to achieve data augmentation. Then, the time series are dimensionally reorganized using random grouping, and convolutional operations are performed to learn features from multivariate time series. Further, attention mechanism and prototype learning are introduced to optimize the distance of the class prototype to which each time series belongs to train a low-dimensional feature representation of the time series, thus reducing the dependency on data volume. Clinical experiments were conducted to collect motion capture data of upper and lower limb movements from 36 patients with PD and eight healthy controls. For the upper limb movement data, the proposed method improved the classification accuracy, weighted precision, and kappa coefficient by 8.89%-15.56%, 9.22%-16.37%, and 0.13-0.23, respectively, compared with support vector machines, long short-term memory, and convolutional prototype network. For the lower limb movement data, the proposed method improved the classification accuracy, weighted precision, and kappa coefficient by 8.16%-20.41%, 10.01%-23.73%, and 0.12-0.28, respectively. The experiments and results show that the proposed method can objectively and accurately assess upper and lower limb bradykinesia in PD.

Healthcare facilities management: A novel data-driven model for predictive maintenance of computed tomography equipment

BackgroundThe breakdown of healthcare facilities is a huge challenge for hospitals. Medical images obtained by Computed Tomography (CT) provide information about the patients' physical conditions and play a critical role in diagnosis of disease. To deliver high-quality medical images on time, it is essential to minimize the occurrence frequencies of anomalies and failures of the equipment. MethodsWe extracted the real-time CT equipment status time series data such as oil temperature, of three equipment, between May 19, 2020, and May 19, 2021. Tube arcing is treated as the classification label. We propose a dictionary-based data-driven model SAX-HCBOP, where the two methods, Histogram-based Information Gain Binning (HIGB) and Coefficient improved Bag of Pattern (CoBOP), are implemented to transform the data into the bag-of-words paradigm. We compare our model to the existing predictive maintenance models based on statistical and time series classification algorithms. ResultsThe results show that the Accuracy, Recall, Precision and F1-score of the proposed model achieve 0.904, 0.747, 0.417, 0.535, respectively. The oil temperature is identified as the most important feature. The proposed model is superior to other models in predicting CT equipment anomalies. In addition, experiments on the public dataset also demonstrate the effectiveness of the proposed model. ConclusionsThe two proposed methods can improve the performance of the dictionary-based time series classification methods in predictive maintenance. In addition, based on the proposed real-time anomaly prediction system, the model assists hospitals in making accurate healthcare facilities maintenance decisions.

A constrastive semi-supervised deep learning framework for land cover classification of satellite time series with limited labels

In this work, we present a new semi-supervised learning framework to cope with satellite image time series (SITS) classification in a data paucity scenario, considering extreme low levels of supervision. The proposed methodology, referred as S3ITS (Semi-Supervised Satellite Image Time Series classification method), is based on temporal convolutional neural networks and it takes advantage of both labelled and unlabelled information. S3ITS enforces the data to be projected in a discriminative manifold via contrastive learning, in order to produce a data representation where samples belonging to the same category are closer than the ones belonging to different ones. Pseudo-labelling is employed on unlabelled samples to take the most out of the available unlabelled information. Experiments on two study sites described by SITS of Sentinel-2 images highlight the quality of the proposed method with respect to common SITS-based classification methods and recent machine learning approaches especially tailored for the semi-supervised classification of multi-variate time series data.

Optimization of data pre-processing methods for time-series classification of electroencephalography data

ABSTRACT The performance of time-series classification of electroencephalographic data varies strongly across experimental paradigms and study participants. Reasons are task-dependent differences in neuronal processing and seemingly random variations between subjects, amongst others. The effect of data pre-processing techniques to ameliorate these challenges is relatively little studied. Here, the influence of spatial filter optimization methods and non-linear data transformation on time-series classification performance is analyzed by the example of high-frequency somatosensory evoked responses. This is a model paradigm for the analysis of high-frequency electroencephalography data at a very low signal-to-noise ratio, which emphasizes the differences of the explored methods. For the utilized data, it was found that the individual signal-to-noise ratio explained up to 74% of the performance differences between subjects. While data pre-processing was shown to increase average time-series classification performance, it could not fully compensate the signal-to-noise ratio differences between the subjects. This study proposes an algorithm to prototype and benchmark pre-processing pipelines for a paradigm and data set at hand. Extreme learning machines, Random Forest, and Logistic Regression can be used quickly to compare a set of potentially suitable pipelines. For subsequent classification, however, machine learning models were shown to provide better accuracy.

Time series classification with random temporal features

Time series classification exists in widespread domains such as EEG/ECG classification, device anomaly detection, and speaker authentication. Although many methods have been proposed, efficient selection of intuitive temporal features to accurately classify time series remains challenging. Therefore, this paper presents TSC-RTF, a new time series classification method using random temporal features. First, to ensure the intuitiveness of the features, TSC-RTF selects subsequences containing important data points as candidates for intuitive temporal features. Then, TSC-RTF uses random sampling to reduce the number of candidates significantly. Next, TSC-RTF selects the final temporal features using a random forest to ensure the validity of the final temporal features. Finally, a deep learning classifier is trained by TSC-RTF to achieve high accuracy. The experimental results show that the proposed method can compete with the state-of-the-art methods.

WEASEL 2.0: a random dilated dictionary transform for fast, accurate and memory constrained time series classification

A time series is a sequence of sequentially ordered real values in time. Time series classification (TSC) is the task of assigning a time series to one of a set of predefined classes, usually based on a model learned from examples. Dictionary-based methods for TSC rely on counting the frequency of certain patterns in time series and are important components of the currently most accurate TSC ensembles. One of the early dictionary-based methods was WEASEL, which at its time achieved SotA results while also being very fast. However, it is outperformed both in terms of speed and accuracy by other methods. Furthermore, its design leads to an unpredictably large memory footprint, making it inapplicable for many applications. In this paper, we present WEASEL 2.0, a complete overhaul of WEASEL based on two recent advancements in TSC: Dilation and ensembling of randomized hyper-parameter settings. These two techniques allow WEASEL 2.0 to work with a fixed-size memory footprint while at the same time improving accuracy. Compared to 15 other SotA methods on the UCR benchmark set, WEASEL 2.0 is significantly more accurate than other dictionary methods and not significantly worse than the currently best methods. Actually, it achieves the highest median accuracy over all data sets, and it performs best in 5 out of 12 problem classes. We thus believe that WEASEL 2.0 is a viable alternative for current TSC and also a potentially interesting input for future ensembles.

Attack Graph Model for Cyber-Physical Power Systems Using Hybrid Deep Learning

Electrical power grids are vulnerable to cyber attacks, as seen in Ukraine in 2015 and 2016. However, existing attack detection methods are limited. Most of them are based on power system measurement anomalies that occur when an attack is successfully executed at the later stages of the cyber kill chain. In contrast, the attacks on the Ukrainian power grid show the importance of system-wide, early-stage attack detection through communication-based anomalies. Therefore, in this paper, we propose a novel method for online cyber attack situational awareness that enhances the power grid resilience. It supports power system operators in the identification and localization of active attack locations in Operational Technology (OT) networks in near real-time. The proposed method employs a hybrid deep learning model of Graph Convolutional Long Short-Term Memory (GC-LSTM) and a deep convolutional network for time series classification-based anomaly detection. It is implemented as a combination of software defined networking, anomaly detection in communication throughput, and a novel attack graph model. Results indicate that the proposed method can identify active attack locations, e.g., within substations, control center, and wide area network, with an accuracy above 96%. Hence, it outperforms existing state-of-the-art deep learning-based time series classification methods.

A time series classification method combining graph embedding and the bag-of-patterns algorithm

A time series classification method combining graph embedding and the bag-of-patterns algorithm

Financial time series classification method based on low‐frequency approximate representation

AbstractAiming at the mode mixing problems of high frequency information caused by fluctuation agglomeration and pointed peak thick tail of financial time series, a time series classification method based on low frequency approximate representation is proposed. The steps are as follows. Firstly, complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN) was used to decompose the time series into a series of modal components and residual terms, and the permutation entropy of the modal components and residual terms was calculated. And the permutation entropy was clustered into two categories by Fisher's optimal segmentation method. Then, according to the clustering results of permutation entropy, the corresponding modal components and residual terms were integrated into high frequency information and low frequency information, so as to realize the adaptive extraction of low frequency information. Secondly, distance matrices were obtained by Euclidean distance (ED) or dynamic time warping (DTW) for low frequency information, and then the nearest neighbor 1‐NN algorithm was used to classify time series.

CALIMERA: A new early time series classification method

Early time series classification is a variant of the time series classification task, in which a label must be assigned to the incoming time series as quickly as possible without necessarily screening through the whole sequence. It needs to be realized on the algorithmic level by fusing a decision-making method that detects the right moment to stop and a classifier that assigns a class label. The contribution addressed in this paper is twofold. Firstly, we present a new method for finding the best moment to perform an action (terminate/continue). Secondly, we propose a new learning scheme using classifier calibration to estimate classification accuracy. The new approach, called CALIMERA, is formalized as a cost minimization problem. Using two benchmark methodologies for early time series classification, we have shown that the proposed model achieves better results than the current state-of-the-art. Two most serious competitors of CALIMERA are ECONOMY and TEASER. The empirical comparison showed that the new method achieved a higher accuracy than TEASER for 35 out of 45 datasets and it outperformed ECONOMY in 20 out of 34 datasets.

Place-Centered Bus Accessibility Time Series Classification with Floating Car Data: An Actual Isochrone and Dynamic Time Warping Distance-Based k-Medoids Method

Classifying a time series is a fundamental task in temporal analysis. This provides valuable insights into the temporal characteristics of data. Although it has been applied to traffic flow and individual-centered accessibility analysis, it has yet to be applied to place-centered accessibility research. In this study, we have proposed an actual isochrone and dynamic time-wrapping distance-based k-medoids method and tested its applicability to a bus accessibility analysis. Using bus floating car data, our method calculated the actual isochrone area as an accessibility measurement and constructs an accessibility time series for each hexagonal geographical unit within the area of interest. We then calculated the dynamic time warp distance between the accessibility time series of pairwise geographical units and used these distances for k-medoid clustering. The optimized class number k was selected by considering the elbow method, silhouette score, and human examination. Our case study in Hefei, China demonstrates the feasibility of our method for accessibility time series classification. We also discovered that the resulting classes follow clear spatial patterns, indicating that different time series classes may be correlated with their spatial location. To our knowledge, this is the first time that such a classification method has been applied to place-centered accessibility time series analysis. Our data-driven method can inform place-centered accessibility in an era in which large quantities of spatiotemporal data like floating car data are available.

Oil-Net: A Learning-Based Framework for Working Conditions Diagnosis of Oil Well Through Dynamometer Cards Identification

Working conditions identification of oil well relies on high-cost and slow-speed artificial inspection in past decades. Working conditions diagnosis through dynamometer cards identification based on machine learning has significant economic value and environmental meaning for oil production. In this paper, we collect dynamometer cards from the famous Chinese Shengli Oil Field as research data. We firstly investigate properties of dynamometers under different working conditions and create a dynamometer card dataset. Further we introduce calibration/visualization preprocess methods, and propose the Oil-Net 1D/2D identification models from the time-series and computer vision views respectively. Experiment results indicate that comparing with other machine learning and time series classification methods, Oil-Net 1D/2D improve identification accuracy significantly. This study provides guidance for the design and implementation of learning-based oil well working conditions intelligent diagnosis system.

Choose Wisely: An Extensive Evaluation of Model Selection for Anomaly Detection in Time Series

Anomaly detection is a fundamental task for time-series analytics with important implications for the downstream performance of many applications. Despite increasing academic interest and the large number of methods proposed in the literature, recent benchmark and evaluation studies demonstrated that no overall best anomaly detection methods exist when applied to very heterogeneous time series datasets. Therefore, the only scalable and viable solution to solve anomaly detection over very different time series collected from diverse domains is to propose a model selection method that will select, based on time series characteristics, the best anomaly detection method to run. Existing AutoML solutions are, unfortunately, not directly applicable to time series anomaly detection, and no evaluation of time series-based approaches for model selection exists. Towards that direction, this paper studies the performance of time series classification methods used as model selection for anomaly detection. Overall, we compare 17 different classifiers over 1800 time series, and we propose the first extensive experimental evaluation of time series classification as model selection for anomaly detection. Our results demonstrate that model selection methods outperform every single anomaly detection method while being in the same order of magnitude regarding execution time. This evaluation is the first step to demonstrate the accuracy and efficiency of time series classification algorithms for anomaly detection, and represents a strong baseline that can then be used to guide the model selection step in general AutoML pipelines.

Abnormal discharge detection using adaptive neuro-fuzzy inference method with probability density-based feature and modified subtractive clustering

Abnormal discharge (AD) is a discharge mode in electroencephalogram (EEG) with sharp outlines. Lack of related open-access datasets and insufficient annotation hamper the development of AD detection, while cognitive neuroscience, clinical research and pilots’ neural screening demand stable and reliable AD detection methods. An adaptive neuro-fuzzy inference method for AD detection is proposed in this work. First, a probability-density-based (PD) method is proposed to extract lognormal amplitude features from EEG envelopes. Second, the subtractive clustering method (SCM) is modified so that clustering radii can adapt to cluster shapes for each dimension instead of using identical radii for each cluster. The outputs of modified SCM (mSCM), coordinates of cluster centers and adjusting rates for radius components are used to automatically initialize Gaussian membership functions of adaptive-network-based fuzzy inference system (ANFIS). Finally, we conducted multiple experiments to validate mSCM-based ANFIS, comparing it with traditional machine learning classifiers (support vector machines, multi-layer perceptrons, decision trees, and random forests) and the state-of-the-art deep learning-based time-series classification methods InceptionTime and Minirocket, using synthetic data, small-size datasets and a private EEG dataset. Results show that combining PD features with features proposed in previous studies, such as smoothed nonlinear energy operator features and discrete wavelet transform features, achieved higher accuracy (95.13 ± 0.86%) and recall (89.17 ± 3.06%) than other feature combinations. mSCM created more suitable cluster boundaries than SCM on small-size datasets, and its clustering results demonstrated potential in helping interpretation of classification rules built in the ANFIS network. Results of comparison experiments showed that mSCM-based ANFIS produced competitive results in accuracy and recall for AD detection, with lower computational cost, compared to the top 2 results obtained by InceptionTime and Minrocket.

DEEP LEARNING FOR SIMULTANEOUS IMPUTATION AND CLASSIFICATION OF TIME SERIES INCOMPLETE DATA

Classification with time series data has many practical applications in finance, medicine, manufacturing, energy, transportation, and agriculture. However, time series data is often plagued by missing values, which not only decreases the accuracy but also increases the complexity of classification models. A common approach to address classification with missing data is to use imputation methods to estimate the missing values before constructing classification models. Recurrent neural network models have been shown to be effective in estimating missing values for time series data. On the other hand, convolutional neural network models have demonstrated their effectiveness in building classification models for time series data. However, there has been no research that specifically addresses how to combine these two types of models to simultaneously address the problem of time series classification with incomplete data. Therefore, in this paper, a model combining recurrent neural networks and convolutional neural networks is proposed to build a model that can simultaneously estimate missing values and classify time series data. The experimental results demonstrate that the proposed model performs better than the existing methods for time series classification with incomplete data.

Improved Recurrence Plots Compression Distance by Learning Parameter for Video Compression Quality.

As the Internet-of-Things is deployed widely, many time-series data are generated everyday. Thus, classifying time-series automatically has become important. Compression-based pattern recognition has attracted attention, because it can analyze various data universally with few model parameters. RPCD (Recurrent Plots Compression Distance) is known as a compression-based time-series classification method. First, RPCD transforms time-series data into an image called "Recurrent Plots (RP)". Then, the distance between two time-series data is determined as the dissimilarity between their RPs. Here, the dissimilarity between two images is computed from the file size, when an MPEG-1 encoder compresses the video, which serializes the two images in order. In this paper, by analyzing the RPCD, we give an important insight that the quality parameter for the MPEG-1 encoding that controls the resolution of compressed videos influences the classification performance very much. We also show that the optimal parameter value depends extremely on the dataset to be classified: Interestingly, the optimal value for one dataset can make the RPCD fall behind a naive random classifier for another dataset. Supported by these insights, we propose an improved version of RPCD named qRPCD, which searches the optimal parameter value by means of cross-validation. Experimentally, qRPCD works superiorly to the original RPCD by about 4% in terms of classification accuracy.

What about the Latent Space? The Need for Latent Feature Saliency Detection in Deep Time Series Classification

Saliency methods are designed to provide explainability for deep image processing models by assigning feature-wise importance scores and thus detecting informative regions in the input images. Recently, these methods have been widely adapted to the time series domain, aiming to identify important temporal regions in a time series. This paper extends our former work on identifying the systematic failure of such methods in the time series domain to produce relevant results when informative patterns are based on underlying latent information rather than temporal regions. First, we both visually and quantitatively assess the quality of explanations provided by multiple state-of-the-art saliency methods, including Integrated Gradients, Deep-Lift, Kernel SHAP, and Lime using univariate simulated time series data with temporal or latent patterns. In addition, to emphasize the severity of the latent feature saliency detection problem, we also run experiments on a real-world predictive maintenance dataset with known latent patterns. We identify Integrated Gradients, Deep-Lift, and the input-cell attention mechanism as potential candidates for refinement to yield latent saliency scores. Finally, we provide recommendations on using saliency methods for time series classification and suggest a guideline for developing latent saliency methods for time series.

Local morphological patterns for time series classification

The key problem of time series classification is the similarity measure between time series. In recent years, efficient and accurate similarity measurement methods of time series have attracted extensive attention from researchers. According to the different similarity measure strategies, the existing time series classification methods can be roughly divided into shape-based (original value) methods and structure-based (symbol transformation) methods. Shape-based methods usually use Euclidean distance (ED), dynamic time warping (DTW), or other methods to measure the global similarity between sequences. The disadvantage of these methods is that their measurement process does not necessarily achieve local sensible matchings of time series, which leads to a decrease in their accuracy and interpretability. To better capture the local information of the sequence, the structure-based methods discretize or symbolize the local value of the time sequence, which leads to the loss of the original information of the sequence. To address these problems, this paper proposes a novel similarity measurement method named dynamic time warping based on the local morphological pattern (MPDTW), which first decomposes the local subsequences of time series using discrete wavelet transforms for extracting the local structure information. Then, the decomposed subsequence will be encoded by the morphological pattern. Finally, the ED between points and their local structure difference based on morphological pattern will be weighted and applied to the DTW algorithm to measure the similarity between sequences. Experiments have been carried out on the classification tasks of the UCR datasets and the results show that our method outperforms the existing baselines.