Time Series Classification Methods Research Articles

효과적인 시계열 데이터 분류를 위한 동적시간왜곡 기반의 시계열 길이 변환

Time series data produced from various sensors are mostly different from each other in length due to various environment of collection. Most time series classification methods, however, assume that the lengths of time series data are the same. There is no alternative, moreover, for variant-length time series classification except DTW and DTW-related methods and few researches on transformation of time series length can be found in direct ways. In this paper, we propose a length transformation method for effective variant-length time series classification. Proposed method is a similarity-preserving transformation and the restoration is possible. To evaluate the proposed method, experiments are conducted to compare the classification performance by applying well-known methods to the time series data before and after the transformation. The classification using the dataset transformed by the proposed method shows better performance on almost all measurements.

Clustering and classification of time series using topological data analysis with applications to finance

In this paper, we propose new methods for time series classification and clustering. These methods are based on techniques of Topological Data Analysis (TDA) such as persistent homology and time delay embedding for analysing time-series data. We present a new clustering method SOM-TDA and a new classification method RF-TDA based on TDA. Using SOM-TDA we examine the topological similarities and dissimilarities of some well-known time-series models used in finance. We also use the RF-TDA to examine if the topological features can be used to distinguish between time series models using simulated data. The performance of RF-TDA on the classification task is compared against three other classification methods. We also consider an application of RF-TDA to financial time series classification using real-life price data of stocks belonging to different sectors. RF-TDA is seen to perform quite well in the two experiments based on real-life stock-price data. This implies that the topological features of the time series of stock prices in the different sectors are not identical and have distinctive features that can be discerned through the use of TDA. We also briefly consider multi-class classification using RF-TDA.

Temporal representation learning for time series classification

Recent years have witnessed the exponential growth of time series data as the popularity of sensing devices and development of IoT techniques; time series classification has been considered as one of the most challenging studies in time series data mining, attracting great interest over the last two decades. According to the empirical evidences, temporal representation learning-based time series classification has more superiority of accuracy, efficiency and interpretability as compared to hundreds of existing time series classification methods. However, due to the high time complexity of feature process, the performance of these methods has been severely restricted. In this paper, we first presented an efficient shapelet transformation method to improve the overall efficiency of time series classification, and then, we further developed a novel enhanced recurrent neural network model for deep representation learning to further improve the classification accuracy. Experimental results on typical real-world datasets have justified the superiority of our models over several shallow and deep representation learning competitors.

ROCKET: exceptionally fast and accurate time series classification using random convolutional kernels

Most methods for time series classification that attain state-of-the-art accuracy have high computational complexity, requiring significant training time even for smaller datasets, and are intractable for larger datasets. Additionally, many existing methods focus on a single type of feature such as shape or frequency. Building on the recent success of convolutional neural networks for time series classification, we show that simple linear classifiers using random convolutional kernels achieve state-of-the-art accuracy with a fraction of the computational expense of existing methods. Using this method, it is possible to train and test a classifier on all 85 ‘bake off’ datasets in the UCR archive in $$<\,2\,\hbox {h}$$ , and it is possible to train a classifier on a large dataset of more than one million time series in approximately 1 h.

Development of Indicator of Data Sufficiency for Feature-based Early Time Series Classification with Applications of Bearing Fault Diagnosis

Diagnosis of bearing faults is crucial in various industries. Time series classification (TSC) assigns each time series to one of a set of pre-defined classes, such as normal and fault, and has been regarded as an appropriate approach for bearing fault diagnosis. Considering late and inaccurate fault diagnosis may have a significant impact on maintenance costs, it is important to classify bearing signals as early and accurately as possible. TSC, however, has a major limitation, which is that a time series cannot be classified until the entire series is collected, implying that a fault cannot be diagnosed using TSC in advance. Therefore, it is important to classify a partially collected time series for early time series classification (ESTC), which is a TSC that considers both accuracy and earliness. Feature-based TSCs can handle this, but the problem is to determine whether a partially collected time series is enough for a decision that is still unsolved. Motivated by this, we propose an indicator of data sufficiency to determine whether a feature-based fault detection classifier can start classifying partially collected signals in order to diagnose bearing faults as early and accurately as possible. The indicator is trained based on the cosine similarity between signals that were collected fully and partially as input to the classifier. In addition, a parameter setting method for efficiently training the indicator is also proposed. The results of experiments using four benchmark datasets verified that the proposed indicator increased both accuracy and earliness compared with the previous time series classification method and general time series classification.

A Metric Learning-Based Univariate Time Series Classification Method

High-dimensional time series classification is a serious problem. A similarity measure based on distance is one of the methods for time series classification. This paper proposes a metric learning-based univariate time series classification method (ML-UTSC), which uses a Mahalanobis matrix on metric learning to calculate the local distance between multivariate time series and combines Dynamic Time Warping(DTW) and the nearest neighbor classification to achieve the final classification. In this method, the features of the univariate time series are presented as multivariate time series data with a mean value, variance, and slope. Next, a three-dimensional Mahalanobis matrix is obtained based on metric learning in the data. The time series is divided into segments of equal intervals to enable the Mahalanobis matrix to more accurately describe the features of the time series data. Compared with the most effective measurement method, the related experimental results show that our proposed algorithm has a lower classification error rate in most of the test datasets.

Multiple time-series convolutional neural network for fault detection and diagnosis and empirical study in semiconductor manufacturing

The development of information technology and process technology have been enhanced the rapid changes in high-tech products and smart manufacturing, specifications become more sophisticated. Large amount of sensors are installed to record equipment condition during the manufacturing process. In particular, the characteristics of sensor data are temporal. Most the existing approaches for time series classification are not applicable to adaptively extract the effective feature from a large number of sensor data, accurately detect the fault, and provide the assignable cause for fault diagnosis. This study aims to propose a multiple time-series convolutional neural network (MTS-CNN) model for fault detection and diagnosis in semiconductor manufacturing. This study incorporates data augmentation with sliding window to generate amounts of subsequences and thus to enhance the diversity and avoid over-fitting. The key features of equipment sensor can be learned automatically through stacked convolution-pooling layers. The importance of each sensor is also identified through the diagnostic layer in the proposed MTS-CNN. An empirical study from a wafer fabrication was conducted to validate the proposed MTS-CNN and compare the performance among the other multivariate time series classification methods. The experimental results demonstrate that the MTS-CNN can accurately detect the fault wafers with high accuracy, recall and precision, and outperforms than other existing multivariate time series classification methods. Through the output value of the diagnostic layer in MTS-CNN, we can identify the relationship between each fault and different sensors and provider valuable information to associate the excursion for fault diagnosis.

Regularized shapelet learning for scalable time series classification

Time series shapelets are subsequences that best split time series data into classes. Therefore, shapelet discovery has attracted considerable interest in the time series classification community. However, almost all state-of-the-art shapelet-based time series classification methods have an inevitably high computational cost. To overcome this drawback, we present a regularized shapelet learning framework in which the fused lasso regularizer is used to maintain the time order of the shapelets and different loss functions can be employed to improve the speed and accuracy of the time series classification. The proposed framework converts the traditional brute force shapelet searching process into a regularized machine learning problem. The most prominent advantage of this conversion is that the speed of the shapelet learning process and the discrimination of the learned shapelets are both theoretically guaranteed. As such, both the speed and accuracy of the shapelet-based time series classification are improved in this paper. Comparison experiments on several datasets show that our framework effectively reduces the training time of time series classification while improving the classification accuracy.

An interpretable gait recognition method based on time series features

Gait recognition is a type of biometric recognition that can be used as an identification tool in various applications. Deep learning-based methods have recently exhibited promising accuracy in gait recognition tasks; however, in addition to an accurate prediction, these methods are required to explain the recognition results. The black-box nature of deep neural networks makes it very difficult to interpret the basis for their identification. The published studies on the interpretability of gait recognition are also in a blank state. Moreover, deep neural networks require a large amount of data to learn the model parameters and an effective generalization on unseen data is difficult when the problem size is small. Thus, this paper presents a gait recognition method combining accuracy and interpretability. The gait feature is represented as a multi-dimensional time series and a Shapelet-based time series classification method is used for gait recognition. A Shapelet is the most discriminative subsequence in time series that makes the proposed method provide interpretability and accuracy simultaneously. We conducted experiments on the CASIA-B dataset and compared the proposed method with several state-of-the-arts deep learning methods. Experiments show that the proposed method can provide an accuracy close to that of deep neural networks on small-scale data sets. At the same time, the decision-making reason of the model can be explained in detail. Concretely, our method can reveal discriminative gait features and frame numbers for specific subjects.

CNN-VSR: A Deep Learning Architecture with Validation-Based Stopping Rule for Time Series Classication

ABSTRACTDeep learning methods for univariate time series classification (TSC) are recently gaining attention. Especially, convolutional neural network (CNN) is utilized to solve the problem of predicting class labels of time series obtained through various important applications, such as engineering, biomedical, and finance. In this work, a novel CNN model is proposed with validation-based stopping rule (VSR) named as CNN-VSR, for univariate TSC using 2-D convolution operation, inspired by image processing properties. For this, first, we develop a novel 2-D transformation approach to convert 1-D time series of any length to 2-D matrix automatically without any manual preprocessing. The transformed time series will be given as an input to the proposed architecture. Further, the implicit and explicit regularization is applied, as time series signal is highly chaotic and prone to over-fitting with learning. Specifically, we define a VSR, which provides a set of parameters associated with a low validation set loss. Moreover, we also conduct a comparative empirical performance evaluation of the proposed CNN-VSR with the best available methods for individual benchmark datasets whose information are provided in a repository maintained by UCR and UEA. Our results reveal that proposed CNN-VSR advances the baseline methods by achieving higher performance accuracy. In addition, we demonstrate that the stopping rule considerably contributes to the classifying performance of the proposed CNN-VSR architecture. Furthermore, we also discuss the optimal model selection and study the effects of different factors on the performance of the proposed CNN-VSR.

Data Augmentation with Suboptimal Warping for Time-Series Classification.

In this paper, a novel data augmentation method for time-series classification is proposed. In the introduced method, a new time-series is obtained in warped space between suboptimally aligned input examples of different lengths. Specifically, the alignment is carried out constraining the warping path and reducing its flexibility. It is shown that the resultant synthetic time-series can form new class boundaries and enrich the training dataset. In this work, the comparative evaluation of the proposed augmentation method against related techniques on representative multivariate time-series datasets is presented. The performance of methods is examined using the nearest neighbor classifier with the dynamic time warping (NN-DTW), LogDet divergence-based metric learning with triplet constraints (LDMLT), and the recently introduced time-series cluster kernel (NN-TCK). The impact of the augmentation on the classification performance is investigated, taking into account entire datasets and cases with a small number of training examples. The extensive evaluation reveals that the introduced method outperforms related augmentation algorithms in terms of the obtained classification accuracy.

The MathIOmica Toolbox: General Analysis Utilities for Dynamic Omics Datasets.

MathIOmica is a package for bioinformatics, written in the Wolfram language, that provides multiple utilities to facilitate the analysis of longitudinal data generated from omics experiments, including transcriptomics, proteomics, and metabolomics data, as well as any generalized time series. MathIOmica uses Mathematica's notebook interface, wherein users can import longitudinal datasets, carry out quality control and normalization, generate time series, and classifytemporal trends. MathIOmica provides spectral methods based on periodograms and autocorrelations for automatically detecting classes of temporal behavior and allowing the user to visualize collective temporal behavior, and also assess biologicalsignificance through Gene Ontology and pathway enrichment analyses. MathIOmica's time-series classification methods address common issues including missing data and uneven sampling in measurements. As such, the software is ideally suited for the analysis of experimental data from individualized profiling of subjects, can facilitate analysis of data from the emerging field of individualized health monitoring, and can detect temporal trends that may be associated with adverse health events. In this article, we import a transcriptomics (RNA-sequencing) dataset collected over multiple timepoints and generate time series for each transcript represented in the data. We classify the time series to identify classes of significant temporal trends (using autocorrelations). We assess statistical significance cutoffs in the classification by generating null distributions using randomly resampled time series. We then visualize the significant trends in heatmaps and assess biological significance using enrichment analyses. Finally, we visualize pathway results for statistically significant pathways of interest. © 2019 by John Wiley & Sons, Inc. Basic Protocol: Time series analysis of transcriptomics expression dataset.

Distinct Feature Extraction for Video-Based Gait Phase Classification

Recent advances in image acquisition and analysis have resulted in disruptive innovation in physical rehabilitation systems facilitating cost-effective, portable, video-based gait assessment. While these inexpensive motion capture systems, suitable for home rehabilitation, do not generally provide accurate kinematics measurements on their own, image processing algorithms ensure gait analysis that is accurate enough for rehabilitation programs. This paper proposes high-accuracy classification of gait phases and muscle actions, using readings from low-cost motion capture systems. First, 12 gait parameters, drawn from the medical literature, are defined to characterize gait patterns. These proposed parameters are then used as input to our proposed multi-channel time-series classification and gait phase reconstruction methods. Proposed methods fully utilize temporal information of gait parameters, thus improving the final classification accuracy. The validation, conducted using 126 experiments, with 6 healthy volunteers and 9 stroke survivors with manually-labelled gait phases, achieves state-of-art classification accuracy of gait phase with lower computational complexity compared to previous solutions.1

A new method for time series classification using multi-dimensional phase space and a statistical control chart

Since large amounts of data were collected over time in many different areas, the classification of these data according to their similarities was an important problem. The methods used to classify time series are a combination of classifiers in different domains such as time, autocorrelation, frequency spectrum, and phase space. The weakest point of these methods is that they require high computational burden and the obtained features lead to misclassifications. When the phase space of the time series is modeled by the Gaussian mixture model, different conditions can be easily classified. However, this technique fails when the phase spaces of time series representing different conditions are similar. In this study, a new method for time series classification using multi-dimensional phase space is proposed using quality control charts that constructed from the phase space of a time series. It aims obtaining a new feature signal from the phase space, providing a faster method for classification of time series, and effectively detecting minor changes in time series. The method is consisted of six stages such as time series inputs, selecting an appropriate time delay and embedding dimension for each time series, construction of phase space, obtaining new time series from phase space using T2 control chart, alignment of time series with dynamic time warping, and classification with the nearest neighbor. The constructed time series is guaranteed to be a complete representation of a system where the phase space parameters are properly chosen. With the proposed new representation, the time series that belongs to different classes and whose phase spaces are similar can be easily distinguished. The k-nearest neighbor classifier is implemented for time series classification, and the datasets from two different domains are used for validation, including motor current signals and nine benchmark datasets from the UCR time series repository. The results show that the proposed method enhances the time series classification performance with new time series representation across these diverse domains.

Time series classification via divergence measures between probability density functions

In this work, we describe a new method for time series classification (TSC) that consists of modeling time series as probability density functions (PDFs) and applies the divergence (the Integrated Squared Error) between two PDFs as a similarity measure for classification via k-Nearest Neighbours (kNN). The proposed method starts by projecting the original time series data into the reconstructed phase space (RPS) via time delay embedding. From these data points in RPS, the corresponding underlying PDF is estimated by Kernel Density Estimation (KDE). Then, a similarity matrix is built by using the Integrated Squared Error (ISE) as a distance measure between two PDFs, from which kNN algorithms can be eventually applied for classification. Two experiments were conducted in order to evaluate this proposal. The first one investigated the impact of the time delay embedding parameters on classification accuracy. We concluded that the embedding dimension is the most influential parameter, for which the results have shown to be highly sensitive. The second experiment provides a comparative analysis of the proposed method against the main state-of-the-art methods for TSC on several well-known benchmark datasets. The results were quite encouraging, and our proposal was able to outperform the compared TSC methods in the majority of the datasets.

XG-SF: An XGBoost Classifier Based on Shapelet Features for Time Series Classification

Time series classification (TSC) has attracted significant interest over the past decade. A lot of TSC methods have been proposed. Among these TSC methods, shapelet based methods are promising for they are interpretable, more accurate, and faster than other methods. For this, a lot of acceleration strategies are proposed. However, the accuracies of speedup methods are not ideal. To address these problems, an XGBoost classifier based on shapelet features (XG-SF) is proposed in this work. In XG-SF, an XGBoost classifier based on shapelet features is used to improve classification accuracy. Our experimental results demonstrate that XG-SF is faster than the state-of-the-art classifiers and the classification accuracy rate is also improved to a certain extent.

Time Series Classification using Fuzzy Cognitive Maps

This paper presents a time-series classification method based on fuzzy cognitive maps. We advocate that fuzzy cognitive maps provide a sound representation of time series, and we can construct a classification mechanism based on them. The classifier has to distinguish maps constructed for time series belonging to different classes. The proposed classification procedure evaluates similarity of fuzzy cognitive maps, and it is done by comparing weight matrices based on the same set of concepts. A weight matrix describes relationships between concepts in a map. Concepts represent the underlying data, because they are extracted via a data-driven clustering procedure. Each data point of a time series is related to each concept, and we evaluate the strength of relationships with a membership function. This paper investigates performance of the proposed approach on a suite of real-world datasets. We compare classification accuracy of our method with 37 state-of-the-art time-series classification methods. Experiments show that the proposed method is performing well. In many cases, it is better than its competitors.

A novel non-parametric method for time series classification based on [formula omitted]-Nearest Neighbors and Dynamic Time Warping Barycenter Averaging

Time series classification is one of the most important issues in time series data mining. This problem has attracted more and more attention of researchers in recent years. Among proposed methods in literature, 1-Nearest Neighbor (1-NN), its variants and improvements have been widely considered as hard to be beaten on classification of time series. In this paper, we propose a novel non-parametric method to classify time series. The proposed method, namely Weighted Local Dynamic Time Warping Barycenter Averaging k-Nearest Neighbors (WLDBAk-NN), is an improvement of Local Mean-based k-Nearest Neighbors (LMk-NN) algorithm. It improves LMk-NN in that it replaces the local mean vectors by local Dynamic Time Warping Barycenter (DBA) vectors calculated using our method, namely Weighted DBA (WDBA). By experiments, we show that (i) WLDBAk-NN outperforms the Weighted Local Mean-based k-Nearest Neighbors (WLMk-NN) algorithm, and (ii) both WLMk-NN and WLDBAk-NN outperform 1-NN, LMk-NN, k-Nearest Centroid Neighbors (k-NCN), and LMk-NCN in 85 time series datasets of UCR Time Series Classification Archive. The experimental results also show that new local mean vectors used in WLMk-NN and WLDBAk-NN significantly contribute to the improvement of the performance of time series classification.

A review on distance based time series classification

Time series classification is an increasing research topic due to the vast amount of time series data that is being created over a wide variety of fields. The particularity of the data makes it a challenging task and different approaches have been taken, including the distance based approach. 1-NN has been a widely used method within distance based time series classification due to its simplicity but still good performance. However, its supremacy may be attributed to being able to use specific distances for time series within the classification process and not to the classifier itself. With the aim of exploiting these distances within more complex classifiers, new approaches have arisen in the past few years that are competitive or which outperform the 1-NN based approaches. In some cases, these new methods use the distance measure to transform the series into feature vectors, bridging the gap between time series and traditional classifiers. In other cases, the distances are employed to obtain a time series kernel and enable the use of kernel methods for time series classification. One of the main challenges is that a kernel function must be positive semi-definite, a matter that is also addressed within this review. The presented review includes a taxonomy of all those methods that aim to classify time series using a distance based approach, as well as a discussion of the strengths and weaknesses of each method.

Spatio-temporal analysis of phenology in Yangtze River Delta based on MODIS NDVI time series from 2001 to 2015

Phenology has become a good indicator for illustrating the long-term changes in the natural resources of the Yangtze River Delta. However, two issues can be observed from previous studies. On the one hand, existing time-series classification methods mainly using a single classifier, the discrimination power, can become deteriorated due to fluctuations characterizing the time series. On the other hand, previous work on the Yangtze River Delta was limited in the spatial domain (usually to 16 cities) and in the temporal domain (usually 2000–2010). To address these issues, this study attempts to analyze the spatiotemporal variation in phenology in the Yangtze River Delta (with 26 cities, enlarged by the state council in June 2016), facilitated by classifying the land cover types and extracting the phenological metrics based on Moderate Resolution Imaging Spectrometer (MODIS) Normalized Difference Vegetation Index (NDVI) time series collected from 2001 to 2015. First, ensemble learning (EL)-based classifiers are used for land cover classification, where the training samples (a total of 201,597) derived from visual interpretation based on GlobelLand30 are further screened using vertex component analysis (VCA), resulting in 600 samples for training and the remainder for validating. Then, eleven phenological metrics are extracted by TIMESAT (a package name) based on the time series, where a seasonal-trend decomposition procedure based on loess (STL-decomposition) is used to remove spikes and a Savitzky-Golay filter is used for filtering. Finally, the spatio-temporal phenology variation is analyzed by considering the classification maps and the phenological metrics. The experimental results indicate that: 1) random forest (RF) obtains the most accurate classification map (with an overall accuracy higher than 96%); 2) different land cover types illustrate the various seasonalities; 3) the Yangtze River Delta has two obvious regions, i.e., the north and the south parts, resulting from different rainfall, temperature, and ecosystem conditions; 4) the phenology variation over time is not significant in the study area; 5) the correlation between gross spring greenness (GSG) and gross primary productivity (GPP) is very high, indicating the potential use of GSG for assessing the carbon flux.