UCR Time Series Research Articles

A new multi-process collaborative architecture for time series classification

Time series classification (TSC) is the problem of categorizing time series data by using machine learning techniques. Its applications vary from cybersecurity and health care to remote sensing and human activity recognition. In this paper, we propose a novel multi-process collaborative architecture for TSC. The propositioned method amalgamates multi-head convolutional neural networks and capsule mechanism. In addition to the discovery of the temporal relationship within time series data, our approach derives better feature extraction with different scaled capsule routings and enhances representation learning. Unlike the original CapsNet, our proposed approach does not need to reconstruct to increase the accuracy of the model. We examine our proposed method through a set of experiments running on the domain-agnostic TSC benchmark datasets from the UCR Time Series Archive. The results show that, compared to a number of recently developed and currently used algorithms, we achieve 36 best accuracies out of 128 datasets. The accuracy analysis of the proposed approach demonstrates its significance in TSC by offering very high classification confidence with the potential of making inroads into plentiful future applications.

SRPM\u2013CNN: a combined model based on slide relative position matrix and CNN for time series classification

Research on the time series classification is gaining an increased attention in the machine learning and data mining areas due to the existence of the time series data almost everywhere, especially in our daily work and life. Recent studies have shown that the convolutional neural networks (CNN) can extract good features from the images and texts, but it often encounters the problem of low accuracy, when it is directly employed to solve the problem of time series classification. In this pursuit, the present study envisaged a novel combined model based on the slide relative position matrix and CNN for time series. The proposed model first adopted the slide relative position for converting the time series data into 2D images during preprocessing, and then employed CNN to classify these images. This made the best of the temporal sequence characteristic of time series data, thereby utilizing the advantages of CNN in image recognition. Finally, 14 UCR time series datasets were chosen to evaluate the performance of the proposed model, whose results indicate that the accuracy of the proposed model was higher than others.

Current Time Series Anomaly Detection Benchmarks are Flawed and are Creating the Illusion of Progress

Time series anomaly detection has been a perennially important topic in data science, with papers dating back to the 1950s. However, in recent years there has been an explosion of interest in this topic, much of it driven by the success of deep learning in other domains and for other time series tasks. Most of these papers test on one or more of a handful of popular benchmark datasets, created by Yahoo, Numenta, NASA, etc. In this work we make a surprising claim. The majority of the individual exemplars in these datasets suffer from one or more of four flaws. Because of these four flaws, we believe that many published comparisons of anomaly detection algorithms may be unreliable, and more importantly, much of the apparent progress in recent years may be illusionary. In addition to demonstrating these claims, with this paper we introduce the UCR Time Series Anomaly Archive. We believe that this resource will perform a similar role as the UCR Time Series Classification Archive, by providing the community with a benchmark that allows meaningful comparisons between approaches and a meaningful gauge of overall progress.

Towards Smart Data Selection From Time Series Using Statistical Methods

Transmitting and storing large volumes of dynamic / time series data collected by modern sensors can represent a significant technological challenge. A possibility to mitigate this challenge is to effectively select a subset of significant data points in order to reduce data volumes without sacrificing the quality of the results of the subsequent analysis. This paper proposes a method for adaptively identifying optimal data point selection algorithms for sensor time series on a window-by-window basis. Thus, this contribution focuses on quantifying the effect of the application of data selection algorithms to time series windows. The proposed approach is first used on multiple synthetically generated time series obtained by concatenating multiple sources one after the other, and then validated in the entire UCR time series public data archive.

Efficient Time Series Clustering by Minimizing Dynamic Time Warping Utilization

Dynamic Time Warping (DTW) is a widely used distance measurement in time series clustering. DTW distance is invariant to time series phase perturbations but has a quadratic complexity. An effective acceleration method must reduce the DTW utilization ratio during time series clustering; for example, TADPole uses both upper and lower bounds to prune off a large ratio of expensive DTW calculations. To further reduce the DTW utilization ratio, we find that the linear-complexity L1-norm distance (Manhattan distance) is effective enough when the time series only comprise small phase perturbations. Therefore, we propose a novel time series clustering by Minimizing Dynamic Time Warping Utilization (MiniDTW) algorithm to accelerate time series clustering. In MiniDTW, the dataset is first greedily summarized into seed clusters, which comprise time series of small phase perturbations, by L1-norm distance. Then, we develop a new Sparse Symmetric Non-negative Matrix Factorization (SSNMF) algorithm, which factorizes the DTW distance matrix of seed cluster centers, to merge the seed clusters into the final clusters. The experiments on UCR time series datasets demonstrate that MiniDTW, pruning 98.52% of the DTW utilization, is better than the counterpart method, TADPole, which only prunes 75.56% of the DTW utilization; and thus MiniDTW is 10 times faster than TADPole.

USING AUTOENCODERS FOR DETECTING ECG ANOMALIES

Massive amounts of electronic health records, including data on vital signs and electrocardiograms (ECGs), are now accessible due to the big data revolution. These signals are now more easily obtained and are frequently captured as a time series of observations. There is a particular need to provide innovative methods that enable efficient monitoring of these signals and prompt anomaly detection given the proliferation of smart devices with ECG capabilities. However, anomaly identification is still a very difficult task because the majority of created data is not yet categorized.
 Deep generative models have been used for unsupervised representation learning to develop expressive feature representations of sequences, which can improve the accuracy and ease of use of downstream tasks like anomaly detection. We suggest utilizing an autoencoder to learn representations of ECG sequences in an unsupervised manner. Then, we apply several detection algorithms to identify anomalies based on the learnt representations. We evaluated our method using the UCR time series classification archive's ECG5000 electrocardiogram dataset. Our findings demonstrate that the suggested strategy outperforms previous supervised and unsupervised techniques in identifying anomalies and learning expressive representations of ECG sequences.

A fast and accurate similarity measure for long time series classification based on local extrema and dynamic time warping

The problem of similarity measures is a major area of interest within the field of time series classification (TSC). With the ubiquitous of long time series and the increasing demand for analyzing them on limited resource devices, there is a crucial need for efficient and accurate measures to deal with such kind of data. In fact, there are a plethora of good time series similarity measures in the literature. However, most existing methods achieve good performance for short time series, but their effectiveness decreases quickly as time series are longer. In this paper, we develop a new parameter-free measure for the specific purpose of quickly and accurately assessing the similarity between two given long time series. The proposed “Local Extrema Dynamic Time Warping” (LE-DTW) consists of two steps. The first is a time series representation technique that starts by reducing the dimensionality of a given time series using its local extrema. Next, it physically separates the minima and maxima points for more intuitiveness and consistency of the so-obtained time series representation. The second step consists in adapting the Dynamic Time Warping (DTW) measure so as to evaluate the score of similarity between the generated representations. We test the performance of LE-DTW on a wide range of real-world problems from the UCR time series archive for TSC. Experimental results indicate that for short time series, the proposed method achieves reasonable classification accuracy as compared to DTW. However, for long time series, LE-DTW performs much better. Indeed, it outperforms DTW while providing competitive results against popular distance-based classifiers. Moreover, in terms of efficiency, LE-DTW is orders of magnitude faster than DTW.

Metaheuristics on time series clustering problem: theoretical and empirical evaluation

Considering the literature and the importance of using of the metaheuristic techniques in time series data mining tasks, especially time series clustering (TsC), it seems lack of a comparative study of such techniques in terms of efficiency for TsC problem. Hence, we try to offer the possibility of theoretical and empirical evaluation of metaheuristic techniques in TsC problem. In fact, we follow two main goals by performing this theoretical and empirical evaluation. These goals include: at the first, we would like to prove the effective role of metaheuristic techniques to enhance the efficiency of TsC algorithms, eliminate some challenges of such algorithms (e.g., sensitivity to initialization of some primary parameters), and indicates the popularity of their use in TsC problems during the last years because of their characteristics. Second, we would like to offer the possibility of a comparison between some metaheuristic techniques to analyze the percentage of their effectiveness for enhancing the accuracy of clustering results in TsC problems. The comparative analysis of results of the empirical evaluation for ten standard time series data sets collected from the UCR time series data sets repository show three keys conclusions: (1) generally, metaheuristic techniques have provided reasonable results and significant improvement in terms of efficiency in TsC problem in all experiments due to the main characteristic of metaheuristics that is finding an approximate solution more quickly. But, it is concluded that fuzzy metaheuristic techniques based on population solution class (e.g., FATPSO and FPSO) to provide better results versus single solution class in TsC algorithms. (2) It is deducted that consuming maximum computational time presented by fuzzy metaheuristic techniques in comparison to other optimization techniques in the performed experiments. (3) The paper has proposed that a hybrid of fuzzy metaheuristics (e.g., FATPSO) and the base TsC algorithms can be considered as a more adequate choice to present better results in terms of accuracy in different application areas.

Aero Engine Gas-Path Fault Diagnose Based on Multimodal Deep Neural Networks

Aeroengine, served by gas turbine, is a highly sophisticated system. It is a hard task to analyze the location and cause of gas-path faults by computational-fluid-dynamics software or thermodynamic functions. Thus, artificial intelligence technologies rather than traditional thermodynamics methods are widely used to tackle this problem. Among them, methods based on neural networks, such as CNN and BPNN, cannot only obtain high classification accuracy but also favorably adapt to aeroengine data of various specifications. CNN has superior ability to extract and learn the attributes hiding in properties, whereas BPNN can keep eyesight on fitting the real distribution of original sample data. Inspired by them, this paper proposes a multimodal method that integrates the classification ability of these two excellent models, so that complementary information can be identified to improve the accuracy of diagnosis results. Experiments on several UCR time series datasets and aeroengine fault datasets show that the proposed model has more promising and robust performance compared to the typical and the state-of-the-art methods.

LTSpAUC: Learning Time-Series Shapelets for Partial AUC Maximization

Shapelets are discriminative segments used to classify time-series instances. Shapelet methods that jointly learn both classifiers and shapelets have been studied in recent years because such methods provide both interpretable results and superior accuracy. The partial area under the receiver operating characteristic curve (pAUC) for a low range of false-positive rates (FPR) is an important performance measure for practical cases in industries such as medicine, manufacturing, and maintenance. In this article, we propose a method that jointly learns both shapelets and a classifier for pAUC optimization in any FPR range, including the full AUC. In addition, we propose the following two extensions for shapelet methods: (1) reducing algorithmic complexity in time-series length to linear time and (2) explicitly determining the classes that shapelets tend to match. Comparing with state-of-the-art learning-based shapelet methods, we demonstrated the superiority of pAUC on UCR time-series data sets and its effectiveness in industrial case studies from medicine, manufacturing, and maintenance.

Adversarial Joint-Learning Recurrent Neural Network for Incomplete Time Series Classification.

Incomplete time series classification (ITSC) is an important issue in time series analysis since temporal data often has missing values in practical applications. However, integrating imputation (replacing missing data) and classification within a model often rapidly amplifies the error from imputed values. Reducing this error propagation from imputation to classification remains a challenge. To this end, we propose an adversarial joint-learning recurrent neural network (AJ-RNN) for ITSC, an end-to-end model trained in an adversarial and joint learning manner. We train the system to categorize the time series as well as impute missing values. To alleviate the error introduced by each imputation value, we use an adversarial network to encourage the network to impute realistic missing values by distinguishing real and imputed values. Hence, AJ-RNN can directly perform classification with missing values and greatly reduce the error propagation from imputation to classification, boosting the accuracy. Extensive experiments on 68 synthetic datasets and 4 real-world datasets from the expanded UCR time series archive demonstrate that AJ-RNN achieves state-of-the-art performance. Furthermore, we show that our model can effectively alleviate the accumulating error problem through qualitative and quantitative analysis based on the trajectory of the dynamical system learned by the RNN. We also provide an analysis of the model behavior to verify the effectiveness of our approach.

Continual Learning Using Bayesian Neural Networks.

Continual learning models allow them to learn and adapt to new changes and tasks over time. However, in continual and sequential learning scenarios, in which the models are trained using different data with various distributions, neural networks (NNs) tend to forget the previously learned knowledge. This phenomenon is often referred to as catastrophic forgetting. The catastrophic forgetting is an inevitable problem in continual learning models for dynamic environments. To address this issue, we propose a method, called continual Bayesian learning networks (CBLNs), which enables the networks to allocate additional resources to adapt to new tasks without forgetting the previously learned tasks. Using a Bayesian NN, CBLN maintains a mixture of Gaussian posterior distributions that are associated with different tasks. The proposed method tries to optimize the number of resources that are needed to learn each task and avoids an exponential increase in the number of resources that are involved in learning multiple tasks. The proposed method does not need to access the past training data and can choose suitable weights to classify the data points during the test time automatically based on an uncertainty criterion. We have evaluated the method on the MNIST and UCR time-series data sets. The evaluation results show that the method can address the catastrophic forgetting problem at a promising rate compared to the state-of-the-art models.

Application of Discrete Wavelet Transform in Shapelet-Based Classification

Recently, several shapelet-based methods have been proposed for time series classification, which are accomplished by identifying the most discriminating subsequence. However, for time series datasets in some application domains, pattern recognition on the original time series cannot always obtain ideal results. To address this issue, we propose an ensemble algorithm by combining time frequency analysis and shape similarity recognition of time series. Discrete wavelet transform is used to decompose the time series into different components, and the shapelet features are identified for each component. According to the different correlations between each component and the original time series, an ensemble classifier is built by weighted majority voting, and the Monte Carlo method is used to search for optimal weight vector. The comparative experiments and sensitivity analysis are conducted on 25 datasets from UCR Time Series Classification Archive, which is an important open dataset resource in time series mining. The results show the proposed method has a better performance in terms of accuracy and stability than the compared classifiers.

Weighted kNN and constrained elastic distances for time-series classification

Time-series classification has been addressed by a plethora of machine-learning techniques, including neural networks, support vector machines, Bayesian approaches, and others. It is an accepted fact, however, that the plain vanilla 1-nearest neighbor (1NN) classifier, combined with an elastic distance measure such as Dynamic Time Warping (DTW), is competitive and often superior to more complex classification methods, including the majority-voting k-nearest neighbor (kNN) classifier. With this paper we continue our investigation of the kNN classifier on time-series data and the impact of various classic distance-based vote weighting schemes by considering constrained versions of four common elastic distance measures: DTW, Longest Common Subsequence (LCS), Edit Distance with Real Penalty (ERP), and Edit Distance on Real sequence (EDR). By performing experiments on the entire UCR Time Series Classification Archive we show that weighted kNN is able to consistently outperform 1NN. Furthermore, we provide recommendations for the choices of the constraint width parameter r, neighborhood size k, and weighting scheme, for each mentioned elastic distance measure.

Adaptively constrained dynamic time warping for time series classification and clustering

Time series classification and clustering are important for data mining research, which is conducive to recognizing movement patterns, finding customary routes, and detecting abnormal trajectories in transport (e.g. road and maritime) traffic. The dynamic time warping (DTW) algorithm is a classical distance measurement method for time series analysis. However, the over-stretching and over-compression problems are typical drawbacks of using DTW to measure distances. To address these drawbacks, an adaptive constrained DTW (ACDTW) algorithm is developed to calculate the distances between trajectories more accurately by introducing new adaptive penalty functions. Two different penalties are proposed to effectively and automatically adapt to the situations in which multiple points in one time series correspond to a single point in another time series. The novel ACDTW algorithm can adaptively adjust the correspondence between two trajectories and obtain greater accuracy between different trajectories. Numerous experiments on classification and clustering are undertaken using the UCR time series archive and real vessel trajectories. The classification results demonstrate that the ACDTW algorithm performs better than four state-of-the-art algorithms on the UCR time series archive. Furthermore, the clustering results reveal that the ACDTW algorithm has the best performance among three existing algorithms in modeling maritime traffic vessel trajectory.

Multi-Scale Shapelets Discovery for Time-Series Classification

Shapelets are subsequences of time-series that represent local patterns and can improve the accuracy and the interpretability of time-series classification. The major task of time-series classification using shapelets is to discover high quality shapelets. However, this is challenging since local patterns may have various scales/lengths rather than a unified scale. In this paper, we resolve this problem by discovering shapelets with multiple scales. We propose a novel Multi-Scale Shapelet Discovery (MSSD) algorithm to discover expressive multi-scale shapelets by extending initial single-scale shapelets (i.e., shapelets with a unified scale). MSSD adopts a bi-directional extension process and is robust to extend single-shapelets obtained by different methods. A supervised shapelet quality measurement is further developed to qualify the extension of shapelets. Comprehensive experiments conducted on 25 UCR time-series datasets show that multi-scale shapelets discovered by MSSD improve classification accuracy by around 10% (in average), compared with single-scale shapelets discovered by counterpart methods.

Information Granulation-Based Fuzzy Clustering of Time Series.

In this article, we propose a two-stage time-series clustering approach to cluster time series with different shapes. The first step is to represent the time series by a suite of information granules following the principle of justifiable granularity to perform dimensionality reduction, while the second step is to realize the fuzzy clustering of the time series in the transformed representation space (viz., the space of information granules). In the dimensionality reduction process, the numerical data are granulated using a collection of information granules forming a new sequence that can well describe the original time series. Then, when clustering the time series, dynamic time warping (DTW) is employed to measure the similarity between time series and DTW barycenter averaging (DBA) is generalized to weighted DBA to be involved in the fuzzy C -means (FCMs) algorithm. Finally, the experiments are conducted on the datasets coming from UCR time-series database and Chinese stocks to demonstrate the effectiveness and advantages of the proposed fuzzy clustering approach.

Deep Neural Networks for the Classification of Pure and Impure Strawberry Purees.

In this paper, a comparative study of the effectiveness of deep neural networks (DNNs) in the classification of pure and impure purees is conducted. Three different types of deep neural networks (DNNs)—the Gated Recurrent Unit (GRU), the Long Short Term Memory (LSTM), and the temporal convolutional network (TCN)—are employed for the detection of adulteration of strawberry purees. The Strawberry dataset, a time series spectroscopy dataset from the UCR time series classification repository, is utilized to evaluate the performance of different DNNs. Experimental results demonstrate that the TCN is able to obtain a higher classification accuracy than the GRU and LSTM. Moreover, the TCN achieves a new state-of-the-art classification accuracy on the Strawberry dataset. These results indicates the great potential of using the TCN for the detection of adulteration of fruit purees in the future.

SSDTW: Shape segment dynamic time warping

In order to increase the yield of a process, it is essential to establish a process control based on manufacturing data. Process management systems mainly consist of statistical process control (SPC), fault detection and classification (FDC), and advanced process control (APC), and are modeled using time series data. However, large amounts of time series data and various distributions are collected in the process; hence, preprocessing measures, such as length adjustment, are essential for modeling. Dynamic time warping (DTW) has been widely used as an algorithm that can measure the similarity between two different time series data and adjust their length. However, owing to the complex structure and time lag of processing time series data, there are limitations in applying the traditional DTW. Therefore, to solve this problem, we propose the shape segment dynamic time warping (SSDTW) algorithm that improves DTW in consideration of the structure information of time series data. By using the maximum overlap discrete wavelet transform (MODWT), the proposed method reflects the peripheral information of the time series data and divides the time series data interval to achieve a reasonable local alignment path. SSDTW attains more accurate alignment paths than DTW, derivative dynamic time warping (DDTW), and shapeDTW. Experiments conducted using semiconductor signal data and UCR time series data sets show that the proposed method is more effective than DTW, DDTW, and shapeDTW.

Time-Series Clustering Based on the Characterization of Segment Typologies.

Time-series clustering is the process of grouping time series with respect to their similarity or characteristics. Previous approaches usually combine a specific distance measure for time series and a standard clustering method. However, these approaches do not take the similarity of the different subsequences of each time series into account, which can be used to better compare the time-series objects of the dataset. In this article, we propose a novel technique of time-series clustering consisting of two clustering stages. In a first step, a least-squares polynomial segmentation procedure is applied to each time series, which is based on a growing window technique that returns different-length segments. Then, all of the segments are projected into the same dimensional space, based on the coefficients of the model that approximates the segment and a set of statistical features. After mapping, a first hierarchical clustering phase is applied to all mapped segments, returning groups of segments for each time series. These clusters are used to represent all time series in the same dimensional space, after defining another specific mapping process. In a second and final clustering stage, all the time-series objects are grouped. We consider internal clustering quality to automatically adjust the main parameter of the algorithm, which is an error threshold for the segmentation. The results obtained on 84 datasets from the UCR Time Series Classification Archive have been compared against three state-of-the-art methods, showing that the performance of this methodology is very promising, especially on larger datasets.