Time Series Classification Algorithms Research Articles

Automated Interictal Epileptiform Discharge Detection from Scalp EEG Using Scalable Time-series Classification Approaches.

Deep learning for automated interictal epileptiform discharge (IED) detection has been topical with many published papers in recent years. All existing works viewed EEG signals as time-series and developed specific models for IED classification; however, general time-series classification (TSC) methods were not considered. Moreover, none of these methods were evaluated on any public datasets, making direct comparisons challenging. This paper explored two state-of-the-art convolutional-based TSC algorithms, InceptionTime and Minirocket, on IED detection. We fine-tuned and cross-evaluated them on a public (Temple University Events - TUEV) and two private datasets and provided ready metrics for benchmarking future work. We observed that the optimal parameters correlated with the clinical duration of an IED and achieved the best area under precision-recall curve (AUPRC) of 0.98 and F1 of 0.80 on the private datasets, respectively. The AUPRC and F1 on the TUEV dataset were 0.99 and 0.97, respectively. While algorithms trained on the private sets maintained their performance when tested on the TUEV data, those trained on TUEV could not generalize well to the private data. These results emerge from differences in the class distributions across datasets and indicate a need for public datasets with a better diversity of IED waveforms, background activities and artifacts to facilitate standardization and benchmarking of algorithms.

Distinguishing normal, neuropathic and myopathic EMG with an automated machine learning approach

ObjectiveDistinguishing normal, neuropathic and myopathic electromyography (EMG) traces can be challenging. We aimed to create an automated time series classification algorithm. MethodsEMGs of healthy controls (HC, n = 25), patients with amyotrophic lateral sclerosis (ALS, n = 20) and inclusion body myositis (IBM, n = 20), were retrospectively selected based on longitudinal clinical follow-up data (ALS and HC) or muscle biopsy (IBM). A machine learning pipeline was applied based on 5-second EMG fragments of each muscle. Diagnostic yield expressed as area under the curve (AUC) of a receiver-operator characteristics curve, accuracy, sensitivity, and specificity were determined per muscle (muscle-level) and per patient (patient-level). ResultsDiagnostic yield of the classification ALS vs. HC was: AUC 0.834 ± 0.014 at muscle-level and 0.856 ± 0.009 at patient-level. For the classification HC vs. IBM, AUC was 0.744 ± 0.043 at muscle-level and 0.735 ± 0.029 at patient-level. For the classification ALS vs. IBM, AUC was 0.569 ± 0.024 at muscle-level and 0.689 ± 0.035 at patient-level. ConclusionsAn automated time series classification algorithm can distinguish EMGs from healthy individuals from those of patients with ALS with a high diagnostic yield. Using longer EMG fragments with different levels of muscle activation may improve performance. SignificanceIn the future, machine learning algorithms may help improve the diagnostic accuracy of EMG examinations.

Toward Continuous Breath Monitoring on a Mobile Phone Using a Frugal Conducting Cloth-Based Smart Mask.

A frugal humidity sensor that can detect changes in the humidity of exhaled breath of individuals has been fabricated. The sensor comprises a humidity-sensitive conducting polymer that is in situ formed on a cloth that acts as a substrate. Interdigitated silver electrodes were screen-printed on the modified cloth, and conducting threads connected the electrodes to the measurement circuit. The sensor's response to changing humidity was measured as a voltage drop across the sensor using a microcontroller. The sensor was capable of discerning between fast, normal, and slow breathing based on the response time. A response time of ∼1.3 s was observed for fast breathing. An Android-based mobile application was designed to collect sensor data via Bluetooth for analysis. A time series classification algorithm was implemented to analyze patterns in breathing. The sensor was later stitched onto a face mask, transforming it into a smart mask that can monitor changes in the breathing pattern at work, play, and sleep.

Classification from Sky: A Robust Remote Sensing Time Series Image Classification Using Spatial Encoder and Multi-Fast Channel Attention

The unparalleled availability of Satellite Image Time Series (SITS) for crop phenology classification unravels agricultural parcel observation and monitoring with applications of both economic and ecological importance. Moreover, the need for distinct classification of agricultural parcels into individual crop types falls on state-of-the-art deep learning models for this extrinsic task. However, most existing approaches implemented are complex and ineffective attention incorporated models, which in turn lack the resilience to recognize useful bands in achieving greater accuracy. We propose a Multi-Fast Channel Attention module for deep CNNs based on a Spatial Encoder (SE-MFCA) that requires a few parameters while enhancing the performance-complexity trade-off dilemma. Hence, we leverage on spatial encoder module to extract the images as disorderly sets of pixels to enhance the coarse spatial resolution features. We empirically show that appropriate parameter sharing in the cross channel interaction can preserve performance while significantly reducing model complexity. The proposed multi-channel attention module can efficiently be implemented via an encoder-decoder network to prevent the loss of detailed spatial information. Again, we parallelly distributed the input channel into multiple heads in our network to recover the specialized input features, which will concatenate with the residual to form a rich single feature representation. The extensive experiment has shown that our model SE-MFCA is efficient and effective compared with the previous state-of-the-art time series classification algorithm on a publicly available dataset of Sentinel-2 images for agricultural parcels. Performance-wise SE-MFCA achieves the highest overall accuracy of 94.50% and the highest mean intersection over union score of 51.92%, besides the least trainable params of 131 K and fewer floating point operations of 0.16 M.

Fog computing application of cyber-physical models of IoT devices with symbolic approximation algorithms

Smart manufacturing systems based on cloud computing deal with large amounts of data for various IoT devices, resulting in several challenges, including high latency and high bandwidth usage. Since fog computing physically close to IoT devices can alleviate these issues, much attention has recently been focused on this area. Fans are nearly ubiquitous in manufacturing sites for cooling and ventilation purposes. Thereby, we built a fan system with an accelerometer installed and monitored the operating state of the fan. We analyzed time-series data transmitted from the accelerometer. We applied machine learning under streaming data analytics at the fog computing level to create a fan’s cyber-physical model (CPM). This work employed the symbolic approximation algorithm to approximate the time series data as symbols of arbitrary length. We compared the performance of CPMs made with five time-series classification (TSC) algorithms to monitor the state of the fan for anomalies in real time. The CPM made with the BOSS VS algorithm, a symbol approximation algorithm, accurately determined the current state of the fan within a fog computing environment, achieving approximately 98% accuracy at a 95% confidence level. Furthermore, we conducted a posthoc analysis, running statistical rigor tests on experimental data and simulation results. The workflow proposed in this work would be expected to be utilized for various IoT devices in smart manufacturing systems.

Forest based on Interval Transformation (FIT): A time series classifier with adaptive features

Time series classification (TSC) is an important task in time series data mining and has attracted a lot of research attention. Most TSC algorithms aim to achieve high classification accuracy while reducing the computational complexity. Currently, Time Series Combination of Heterogeneous and Integrated Embedding Forest (TS-CHIEF) is considered to be one of the state-of-the-art TSC algorithms. However, compared with fast algorithms such as Time Series Forest (TSF), TS-CHIEF still has high computation cost. On the premise that the TSF algorithm is fast, we propose a new TSC algorithm, Forest based on Interval Transformation (called FIT), which takes into account both accuracy and efficiency. FIT uses cross-validation to select appropriate transformation series and corresponding interval features, and adaptively converts the interval features of each series in the process of formal training. Subsequently, the transformed feature set is combined with the random forest training FIT model. We evaluate the performance of FIT on 85 UCR time series classification datasets. The experimental results demonstrate that FIT can achieve better accuracy while maintaining high efficiency compared with the state-of-the-art methods.

Time Series Classification with Shapelet and Canonical Features

Shapelet-based time series classification methods are widely adopted models for time series classification tasks. However, the high computational cost greatly limits the practicability of the Shapelet-based methods. What is more, traditional Shapelet can only describe the overall shape characteristics of subsequences under the Euclidean distance metric, so it is vulnerable to noise. Other than Shapelet, there are other types of discriminative information contained in the subsequences. To deal with the aforementioned problems, an accurate and efficient time series classification algorithm, named Shapelet with Canonical Time Series Features, is proposed in this paper. The proposed algorithm is based on the following three key strategies: (1) randomly selecting Shapelet and limiting the scope of Shapelet to improve efficiency; (2) embedding multiple canonical time series features in Shapelet to improve the adaptability of the algorithm to different classification problems and make up for the accuracy loss caused by the random selection of Shapelet; and (3) building a random forest classifier based on the new feature representations to ensure the generalization ability of the algorithm. Experimental results on 112 UCR time series datasets show that the proposed algorithm is more accurate than the STC algorithm which is based on Shapelet exact search and the Shapelet transform technique, as well as many other types of state-of-the-art time series classification algorithms. Moreover, extensive experimental comparisons verify the significant advantages of the proposed algorithm in terms of efficiency.

MultiRocket: multiple pooling operators and transformations for fast and effective time series classification

We propose MultiRocket, a fast time series classification (TSC) algorithm that achieves state-of-the-art accuracy with a tiny fraction of the time and without the complex ensembling structure of many state-of-the-art methods. MultiRocket improves on MiniRocket, one of the fastest TSC algorithms to date, by adding multiple pooling operators and transformations to improve the diversity of the features generated. In addition to processing the raw input series, MultiRocket also applies first order differences to transform the original series. Convolutions are applied to both representations, and four pooling operators are applied to the convolution outputs. When benchmarked using the University of California Riverside TSC benchmark datasets, MultiRocket is significantly more accurate than MiniRocket, and competitive with the best ranked current method in terms of accuracy, HIVE-COTE 2.0, while being orders of magnitude faster.

User behaviour models to forecast electricity consumption of residential customers based on smart metering data

This paper presents a novel approach to forecast day-ahead electricity consumption for residential households where highly irregular human behaviour plays a significant role. The methodology requires data from fiscal smart meters, which makes it applicable to real scenarios where personal data gathering is not feasible. These data are rarely complete; therefore, a robust combination of machine-learning techniques is used to handle missing data and outliers. The novelty of this method relies on identifying and predicting user electricity consumption behaviour as a procedure to improve the forecasting of the overall electricity consumption of each individual customer. The methodology uses Gaussian mixture clustering to identify behaviour clusters and an eXtreme Gradient Boosting classification (XGBoost) model to predict the day-ahead behaviour pattern. This predicted user behaviour cluster is fed into an Artificial Neural Network (ANN) to enable an improved capturing of the highly unpredictable user conduct for the forecast of electricity consumption. A novel metric, namely the Euclidean Distance-based Accuracy (EDA), is finally proposed to enable a more thorough evaluation of time series classification algorithms. The whole development is tested over 500 residential users placed in a southeastern region of Spain. The results showed that, when the novel approach was used, the MAPEd and NRMSEd were reduced by 7% and 9% respectively, increasing to a 20% and 17% respective reduction for the best cases according to EDA. This methodology sets the basis for massive user-centred analyses, very profitable to any electricity company.

Predictive maintenance on sensorized stamping presses by time series segmentation, anomaly detection, and classification algorithms

Sheet metal forming tools, like stamping presses, play an ubiquitous role in the manufacture of several products. With increasing requirements of quality and efficiency, ensuring maximum uptime of these tools is fundamental to marketplace competitiveness. Using anomaly detection and predictive maintenance techniques, it is possible to develop lower risk and more intelligent approaches to maintenance scheduling, however, industrial implementations of these methods remain scarce due to the difficulties of obtaining acceptable results in real-world scenarios, making applications of such techniques in stamping processes seldom found. In this work, we propose a combination of two distinct approaches: (a) time segmentation together with feature dimension reduction and anomaly detection; and (b) machine learning classification algorithms, for effective downtime prediction. The approach (a)+(b) allows for an improvement rate up to 22.971% of the macro F1-score, when compared to sole approach (b). A ROC AUC index of 96% is attained by using Randomized Decision Trees, being the best classifier of twelve tested. An use case with a decentralized predictive maintenance architecture for the downtime forecasting of a stamping press, which is a critical machine in the manufacturing facilities of Bosch Thermo Technology, is discussed.

Data-Driven Fault Diagnosis in a Complex Hydraulic System based on Early Classification

In this paper, an early time-series classification (ETSC) algorithm is applied to support fault diagnosis in a complex hydraulic system (HS) with several interconnected components. The proposed technique aims at early classifying the state of the system while keeping the loss of classification inaccuracy at the minimum level. In contrast to baseline models that detect the eventual faults at the end of each working cycle, the ETSC model can diagnose any fault type of the HS components before observing the entire working cycle. Indeed, the early classification model successfully achieves a trade-off between the accuracy and the earliness criterion. Experimental results on a realistic HS dataset from the related literature show that the ETSC method can effectively identify different fault types with a higher accuracy and earlier compared to baseline methodologies.

Variable-Length Multivariate Time Series Classification Using ROCKET: A Case Study of Incident Detection

Multivariate time series classification is a machine learning problem that can be applied to automate a wide range of real-world data analysis tasks. ROCKET proved to be an outstanding algorithm capable to classify time series accurately and quickly. The textbook variant of the multivariate time series classification problem assumes that time series to be classified are all of the same length, while in real-world applications this assumption not necessarily holds. The literature of this domain does not pay enough attention to data processing pipelines for variable-length time series. Thus, in this paper, we present a thorough analysis of three preprocessing pipelines that handle variable-length time series that need to be classified with a method that requires the data to be of equal length. These three methods are truncation, padding, and forecasting of missing value. Experiments conducted on benchmark datasets, showed that the recommended procedure involves padding. Forecasting ensures similar classification accuracy, but comes at a much higher computational cost. Truncation is not a viable option. Furthermore, in the paper, we present a novel domain of application of multivariate time series classification algorithms, that is incident detection in cash transactions. This area poses substantive challenges for automated model training procedures since the data is not only variable-length, but also heavily imbalanced. In the study, we list various incident types and present trained classifiers capable to aid human auditors in their daily work.

MultiETSC: automated machine learning for early time series classification

Early time series classification (EarlyTSC) involves the prediction of a class label based on partial observation of a given time series. Most EarlyTSC algorithms consider the trade-off between accuracy and earliness as two competing objectives, using a single dedicated hyperparameter. To obtain insights into this trade-off requires finding a set of non-dominated (Pareto efficient) classifiers. So far, this has been approached through manual hyperparameter tuning. Since the trade-off hyperparameters only provide indirect control over the earliness-accuracy trade-off, manual tuning is tedious and tends to result in many sub-optimal hyperparameter settings. This complicates the search for optimal hyperparameter settings and forms a hurdle for the application of EarlyTSC to real-world problems. To address these issues, we propose an automated approach to hyperparameter tuning and algorithm selection for EarlyTSC, building on developments in the fast-moving research area known as automated machine learning (AutoML). To deal with the challenging task of optimising two conflicting objectives in early time series classification, we propose MultiETSC, a system for multi-objective algorithm selection and hyperparameter optimisation (MO-CASH) for EarlyTSC. MultiETSC can potentially leverage any existing or future EarlyTSC algorithm and produces a set of Pareto optimal algorithm configurations from which a user can choose a posteriori. As an additional benefit, our proposed framework can incorporate and leverage time-series classification algorithms not originally designed for EarlyTSC for improving performance on EarlyTSC; we demonstrate this property using a newly defined, “naïve” fixed-time algorithm. In an extensive empirical evaluation of our new approach on a benchmark of 115 data sets, we show that MultiETSC performs substantially better than baseline methods, ranking highest (avg. rank 1.98) compared to conceptually simpler single-algorithm (2.98) and single-objective alternatives (4.36).

Trajectory time series classification algorithm based on conv-olutional self-attention mechanism

As a kind of time series data in the field of transportation, trajectory has complex multi-dimensional information. How to fully excavate the multi-dimensional information of the trajectory and analyse its complex dynamic pattern is an important problem to be solved in the classification of trajectory time series. In response to the above problems, this paper improves the existing self-attention mechanism, and proposes a trajectory time series classification algorithm based on convolutional self-attention mechanism. According to the characteristics of the trajectory data, the convolutional self-attention mechanism first extracts the characteristics of the trajectory fragments in the trajectory to obtain the flight status characteristics, and then performs the self-attention operation on these characteristics, so as to achieve a better feature extraction effect. The experimental results show that the trajectory time series classification algorithm based on the convolutional self-attention mechanism has better feature extraction ability and classification accuracy than other existing algorithms.

FNIRS Signal Classification Based on Deep Learning in Rock-Paper-Scissors Imagery Task

To explore whether the brain contains pattern differences in the rock–paper–scissors (RPS) imagery task, this paper attempts to classify this task using fNIRS and deep learning. In this study, we designed an RPS task with a total duration of 25 min and 40 s, and recruited 22 volunteers for the experiment. We used the fNIRS acquisition device (FOIRE-3000) to record the cerebral neural activities of these participants in the RPS task. The time series classification (TSC) algorithm was introduced into the time-domain fNIRS signal classification. Experiments show that CNN-based TSC methods can achieve 97% accuracy in RPS classification. CNN-based TSC method is suitable for the classification of fNIRS signals in RPS motor imagery tasks, and may find new application directions for the development of brain–computer interfaces (BCI).

Deep Learning for ECG Analysis: Benchmarks and Insights from PTB-XL.

Electrocardiography (ECG) is a very common, non-invasive diagnostic procedure and its interpretation is increasingly supported by algorithms. The progress in the field of automatic ECG analysis has up to now been hampered by a lack of appropriate datasets for training as well as a lack of well-defined evaluation procedures to ensure comparability of different algorithms. To alleviate these issues, we put forward first benchmarking results for the recently published, freely accessible clinical 12-lead ECG dataset PTB-XL, covering a variety of tasks from different ECG statement prediction tasks to age and sex prediction. Among the investigated deep-learning-based timeseries classification algorithms, we find that convolutional neural networks, in particular resnet- and inception-based architectures, show the strongest performance across all tasks. We find consistent results on the ICBEB2018 challenge ECG dataset and discuss prospects of transfer learning using classifiers pretrained on PTB-XL. These benchmarking results are complemented by deeper insights into the classification algorithm in terms of hidden stratification, model uncertainty and an exploratory interpretability analysis, which provide connecting points for future research on the dataset. Our results emphasize the prospects of deep-learning-based algorithms in the field of ECG analysis, not only in terms of quantitative accuracy but also in terms of clinically equally important further quality metrics such as uncertainty quantification and interpretability. With this resource, we aim to establish the PTB-XL dataset as a resource for structured benchmarking of ECG analysis algorithms and encourage other researchers in the field to join these efforts.

An improved fast shapelet selection algorithm and its application to pervasive EEG

With the rapid development of pervasive devices, a great deal of time series are generated by various sensors, and many time series classification (TSC) algorithms have been proposed to deal with these data. Among them, shapelet-based algorithms have attracted great attention due to its high accuracy and strong interpretability. However, time complexity of shapelet-based algorithms is high. In this paper, we propose an improved Fast Shapelet Selection algorithm based on Clustering (FSSoC), which greatly reduces the time of shapelet selection. Firstly, time series are clustered into several groups with improved k-means, and then some time series are sampled from each cluster with a strategy based on Euclidean Distance sorting. Secondly, Important Data Points (IDPs) of the sampled time series are identified and only the subsequences between two nonadjacent IDPs are added to shapelet candidates. Therefore, the number of shapelet candidates is greatly reduced, which leads to a obviously reduction in time consumption. Thirdly, FSSoC is applied to shapelet transformation algorithm to test classification accuracy and running time, the experiments demonstrate that FSSoC is obviously faster than existing shapelet selection algorithms while keeping a high accuracy. At last, a case study on EEG time series is presented, which verifies the feasibility of FSSoC application to automatically discover representative EEG features.

Classification Algorithm for Person Identification and Gesture Recognition Based on Hand Gestures with Small Training Sets.

Classification algorithms require training data initially labelled by classes to build a model and then to be able to classify the new data. The amount and diversity of training data affect the classification quality and usually the larger the training set, the better the accuracy of classification. In many applications only small amounts of training data are available. This article presents a new time series classification algorithm for problems with small training sets. The algorithm was tested on hand gesture recordings in tasks of person identification and gesture recognition. The algorithm provides significantly better classification accuracy than other machine learning algorithms. For 22 different hand gestures performed by 10 people and the training set size equal to 5 gesture execution records per class, the error rate for the newly proposed algorithm is from 37% to 75% lower than for the other compared algorithms. When the training set consists of only one sample per class the new algorithm reaches from 45% to 95% lower error rate. Conducted experiments indicate that the algorithm outperforms state-of-the-art methods in terms of classification accuracy in the problem of person identification and gesture recognition.

The great multivariate time series classification bake off: a review and experimental evaluation of recent algorithmic advances

Time Series Classification (TSC) involves building predictive models for a discrete target variable from ordered, real valued, attributes. Over recent years, a new set of TSC algorithms have been developed which have made significant improvement over the previous state of the art. The main focus has been on univariate TSC, i.e. the problem where each case has a single series and a class label. In reality, it is more common to encounter multivariate TSC (MTSC) problems where the time series for a single case has multiple dimensions. Despite this, much less consideration has been given to MTSC than the univariate case. The UCR archive has provided a valuable resource for univariate TSC, and the lack of a standard set of test problems may explain why there has been less focus on MTSC. The UEA archive of 30 MTSC problems released in 2018 has made comparison of algorithms easier. We review recently proposed bespoke MTSC algorithms based on deep learning, shapelets and bag of words approaches. If an algorithm cannot naturally handle multivariate data, the simplest approach to adapt a univariate classifier to MTSC is to ensemble it over the multivariate dimensions. We compare the bespoke algorithms to these dimension independent approaches on the 26 of the 30 MTSC archive problems where the data are all of equal length. We demonstrate that four classifiers are significantly more accurate than the benchmark dynamic time warping algorithm and that one of these recently proposed classifiers, ROCKET, achieves significant improvement on the archive datasets in at least an order of magnitude less time than the other three.

Global bridge damage detection using multi-sensor data based on optimized functional echo state networks

While machine learning has been increasingly incorporated into structural damage detection, most existing methods still rely on hand-crafted damage features. For a given structure, the performance of detection is heavily impacted by the quality of features, and choosing the optimal features may be difficult and time-consuming. Various time series classification algorithms studied in machine learning are able to classify structural responses into damage conditions without feature engineering; however, most of them only deal with univariate time series classification and are either inapplicable or ineffective on multivariate (i.e. multi-dimensional) data, thus unable to fully utilize all sensors available on real bridges. To address these limitations, we propose a global bridge damage detection method based on multivariate time series classification with optimized functional echo state networks. In this method, data from multiple sensors are directly used as inputs without feature extraction. Training of the functional echo state network is simple and straightforward, and by leveraging the nonlinear mapping capacity and dynamic memory of functional echo state network, the separability of different classes, that is, classifying accuracy is enhanced compared to conventional classification algorithms. Furthermore, hyperparameters of the functional echo state network are automatically optimized with particle swarm optimization algorithm, which further improves the accuracy while saving the cost of manual tuning. Experimental results on two classical data sets show that functional echo state network achieves high and stable accuracy, which indicate that our method can detect global bridge structural damage efficiently by analyzing multiple sensor data, and is prospected to be applied in real bridge structural health monitoring systems.