Time Series Classification Algorithms Research Articles

Bake off redux: a review and experimental evaluation of recent time series classification algorithms

AbstractIn 2017, a research paper (Bagnall et al. Data Mining and Knowledge Discovery 31(3):606-660. 2017) compared 18 Time Series Classification (TSC) algorithms on 85 datasets from the University of California, Riverside (UCR) archive. This study, commonly referred to as a ‘bake off’, identified that only nine algorithms performed significantly better than the Dynamic Time Warping (DTW) and Rotation Forest benchmarks that were used. The study categorised each algorithm by the type of feature they extract from time series data, forming a taxonomy of five main algorithm types. This categorisation of algorithms alongside the provision of code and accessible results for reproducibility has helped fuel an increase in popularity of the TSC field. Over six years have passed since this bake off, the UCR archive has expanded to 112 datasets and there have been a large number of new algorithms proposed. We revisit the bake off, seeing how each of the proposed categories have advanced since the original publication, and evaluate the performance of newer algorithms against the previous best-of-category using an expanded UCR archive. We extend the taxonomy to include three new categories to reflect recent developments. Alongside the originally proposed distance, interval, shapelet, dictionary and hybrid based algorithms, we compare newer convolution and feature based algorithms as well as deep learning approaches. We introduce 30 classification datasets either recently donated to the archive or reformatted to the TSC format, and use these to further evaluate the best performing algorithm from each category. Overall, we find that two recently proposed algorithms, MultiROCKET+Hydra (Dempster et al. 2022) and HIVE-COTEv2 (Middlehurst et al. Mach Learn 110:3211-3243. 2021), perform significantly better than other approaches on both the current and new TSC problems.

Finding Discriminative Subsequences Via a Coverage Measure and Mutual Information Selection Strategy for Multi-Class Time Series Classification

Time series classification (TSC) has attracted considerable attention from the data mining community over the past decades. One of the effective ways to handle this task is to find discriminative subsequences in time series to train a classifier. Obviously, how to measure the discriminative power of subsequences and find the optimal combination of subsequences is crucial to the accuracy of TSC. In this paper, we introduce a new method, CRMI, to find high-quality discriminative subsequences for multi-class time series classification (MC-TSC). Different from existing methods, there are two significant innovations in the work. At first, we propose a novel measure, named coverage ratio, to evaluate the discriminative power of a subsequence based on a coverage matrix which is figured out by the clustering technique. Second, a heuristic algorithm based on mutual information (MI) is proposed to find the optimal combination of subsequence candidates. The calculation of MI is also based on the coverage matrix. Extensive experiments were conducted on 54 UCR time series datasets with at least 3 categories, and the results show that (1) the proposed algorithm achieves the highest average accuracy and outperforms most of the existing shapelet-based TSC algorithms; (2) compared with existing methods, the proposed algorithm performs better on datasets with a large number of categories.

Breast cancer classification through multivariate radiomic time series analysis in DCE-MRI sequences

Breast cancer is the most prevalent disease that poses a significant threat to women’s health. Despite the Dynamic Contrast-Enhanced MRI (DCE-MRI) has been widely used for breast cancer classification, its diagnostic performance is still suboptimal. In this work, the Radiomic workflow was implemented to classify the whole DCE-MRI sequence based on the distinction in contrast agent uptake between benign and malignant lesions. The radiomic features extracted from each of the seven time instants within the DCE-MRI sequence were fed into a multi-instant features selection strategy to select the discriminative features for time series classification. Several time series classification algorithms including Rocket, MultiRocket, K-Nearest Neighbor, Time Series Forest, and Supervised Time Series Forest were compared. Firstly, a univariate classification was performed to find the five most informative radiomic series, and then, a multivariate time series classification was implemented via a voting mechanism. The Multivariate Rocket model was the most accurate (Accuracy = 0.852, AUC-ROC = 0.852, Specificity = 0.823, Sensitivity = 0.882). The intelligible radiomic features enabled model findings explanations and clinical validation. In particular, the Energy and TotalEnergy were among the most important features, and the most descriptive for the change in signal intensity, which is the main effect of the contrast agent.

Healthcare facilities management: A novel data-driven model for predictive maintenance of computed tomography equipment

BackgroundThe breakdown of healthcare facilities is a huge challenge for hospitals. Medical images obtained by Computed Tomography (CT) provide information about the patients' physical conditions and play a critical role in diagnosis of disease. To deliver high-quality medical images on time, it is essential to minimize the occurrence frequencies of anomalies and failures of the equipment. MethodsWe extracted the real-time CT equipment status time series data such as oil temperature, of three equipment, between May 19, 2020, and May 19, 2021. Tube arcing is treated as the classification label. We propose a dictionary-based data-driven model SAX-HCBOP, where the two methods, Histogram-based Information Gain Binning (HIGB) and Coefficient improved Bag of Pattern (CoBOP), are implemented to transform the data into the bag-of-words paradigm. We compare our model to the existing predictive maintenance models based on statistical and time series classification algorithms. ResultsThe results show that the Accuracy, Recall, Precision and F1-score of the proposed model achieve 0.904, 0.747, 0.417, 0.535, respectively. The oil temperature is identified as the most important feature. The proposed model is superior to other models in predicting CT equipment anomalies. In addition, experiments on the public dataset also demonstrate the effectiveness of the proposed model. ConclusionsThe two proposed methods can improve the performance of the dictionary-based time series classification methods in predictive maintenance. In addition, based on the proposed real-time anomaly prediction system, the model assists hospitals in making accurate healthcare facilities maintenance decisions.

Minirocket Kullanarak Güçlendirilmiş ve Verimli Atriyal Fibrilasyon Tespiti

Atrial Fibrillation (AF) detection from intracardiac Electrogram (EGM) signals is a critical aspect of cardiovascular health monitoring. This study explores the application of Minirocket, a time series classification (TSC) algorithm, for robust and efficient AF detection. A comparative analysis is conducted against a deep learning approach using a subset of the dataset from Rodrigo et al. (2022). The study investigates the robustness of Minirocket in the face of shorter EGM sequences and varying training sizes, essential for real-world applications such as wearable and implanted devices. Empirical runtime analysis further assesses the efficiency of Minirocket in comparison to conventional machine learning (ML) algorithms. The results showcase Minirocket's notable performance, especially in scenarios with shorter signals and varying training sizes, making it a promising candidate for streamlined AF detection in emerging cardiovascular monitoring technologies. This research contributes to the optimization of AF detection algorithms for increased efficiency and adaptability to dynamic clinical scenarios.

Patient deterioration detection using one-class classification via cluster period estimation subtask

Deterioration is the significant degradation of the physical state prior to death. Detecting the deterioration of patients could provide an early warning to their families in instances of homecare, to clinicians treating hospitalized patients and to nurses by clients of retirement homes. Traditional supervised machine learning is not helpful for this purpose because the deterioration has individual differences for each patient, and the model cannot access the information about deterioration from healthy patients. This paper applies one-class classification (OCC) to detect deterioration changes. OCC can provide an early warning because the model can learn from only normal conditions. In particular, a one-class time-series classification (OCTSC) algorithm has been developed by combining K-means clustering with sliding windows and a linear regression subtask. The core idea is to detect the change in the signal period related to heart/breathing rate. For this purpose, clustering is applied to sliding windows, and the period is estimated using linear regression for the time index of arbitrary cluster. The deterioration change is detected by unseen scores computed as the error of linear regression subtask for cluster index.

Deep Learning–Based Time-Series Classification for Robotic Inspection of Pipe Condition Using Non-Contact Ultrasonic Testing

Abstract This journal paper explores the application of Deep Learning (DL)-based Time-Series Classification (TSC) algorithms in ultrasonic testing for pipeline inspection. The utility of Electromagnetic Acoustic Transducers (EMAT) as a non-contact ultrasonic testing technique for compact robotic platforms is emphasized, prioritizing computational efficiency in defect detection over pinpoint accuracy. To address limited sample availability, the study conducts benchmarking of four methods to enable comparative evaluation of classification times. The core of the DL-based TSC approach involves training DL models using varied proportions (60%, 80%, and 100%) of the available training dataset. This investigation demonstrates the adaptability of DL-enabled anomaly detection with shifting data sizes, showcasing the AI-driven process's robustness in identifying pipeline irregularities. The outcomes underscore the pivotal role of artificial intelligence (AI) in facilitating semi-accurate but swift anomaly detection, thereby streamlining subsequent focused inspections on pipeline areas of concern. By synergistically integrating EMAT technology and DL-driven TSC, this research contributes to enhancing the precision and near real-time inspection capabilities of pipeline assessment. This investigation collectively highlights the potential of DL networks to revolutionize pipeline inspection by rapidly and accurately analyzing ultrasound waveform data.

A study on pronunciation assessment of English learners based on temporal classification algorithm

Abstract This paper utilizes a time-series classification algorithm to classify samples, and the selected datasets are utilized to calculate the distance of time-series samples. The information gain is calculated using the information obtained, and the information entropy is determined collectively for the node datasets to generate time series features. The convolution operation is used to obtain the formal representation of the time series classification, and the extracted English pronunciation features are adaptively matched. The evaluation results were determined using a multilayer wavelet feature scale transformation method, and English learners achieved scores of 6 and above in all three tests using the time-series classification algorithm. To master standard English pronunciation, English learners should use the temporal classification algorithm.

Measuring service quality based on customer emotion: An explainable AI approach

This paper develops an explainable artificial intelligence (AI) approach to measuring service quality in voice-based service encounters. Drawing from the psychology and computer science literature, we construct features of a customer's emotion dynamics during a service encounter. Using real-world call center data from a large insurance company, we train an ensemble model with these emotion dynamics features to predict service quality. The model has higher prediction performance than the two benchmark approaches using quality-assurance evaluation and operational indices. Our method for emotion dynamics classification outperforms a host of state-of-the-art time series classification algorithms. We further apply explainable AI methods to identify the most important features of emotion dynamics and show how they are related to service quality. For example, the location where the last emotion episode appears in a service call has a U-shaped relationship to low quality. Finally, to demonstrate utility, we design an IT artifact to automatically measure service quality after service encounters in the call center and use the measure to predict a customer's referral intention.

Data-oriented ensemble predictor based on time series classifiers for fraud detection

Non-Technical Losses (NTL) remain a challenge because electricity theft is prevalent in traditional meters and smart metering systems and structures. They cause financial losses that affect the security of supply and lead to a cooperative burden because NTL are incorporated in the expense tariff and are paid by all customers in countries such as India, China, Brazil, Tunisia, Uruguay, and others. A significant number of methods have been proposed to detect NTL, but they cannot precisely recognize this type of fraud, which enables companies to better identify the fraud’s origin and evaluate its financial impact. In addition, it is important to combine several Time-Series Classification (TSC) algorithms which consider the time-dependent nature of energy consumption data. We propose a data-oriented heterogeneous ensemble predictor based on time series classifiers for NTL detection, called DETECT. It classifies energy consumption samples either as honest or as fraudulent and further categorizes the type of fraud which might have been committed based on the analysis of the consumption patterns of each sample. DETECT is a fraud detection algorithm that has the ability to analyze and classify NTL frauds effectively regardless of the dataset. Using our proposed approach, we obtained a performance improvement with a Detection Rate (DR) value equal to 93.45% and a False Positive Rate (FPR) value equal to 1.61%. DETECT focuses on time-series data, which enables the development of a method that better interprets real-world scenarios and it is more error-resistant.

Choose Wisely: An Extensive Evaluation of Model Selection for Anomaly Detection in Time Series

Anomaly detection is a fundamental task for time-series analytics with important implications for the downstream performance of many applications. Despite increasing academic interest and the large number of methods proposed in the literature, recent benchmark and evaluation studies demonstrated that no overall best anomaly detection methods exist when applied to very heterogeneous time series datasets. Therefore, the only scalable and viable solution to solve anomaly detection over very different time series collected from diverse domains is to propose a model selection method that will select, based on time series characteristics, the best anomaly detection method to run. Existing AutoML solutions are, unfortunately, not directly applicable to time series anomaly detection, and no evaluation of time series-based approaches for model selection exists. Towards that direction, this paper studies the performance of time series classification methods used as model selection for anomaly detection. Overall, we compare 17 different classifiers over 1800 time series, and we propose the first extensive experimental evaluation of time series classification as model selection for anomaly detection. Our results demonstrate that model selection methods outperform every single anomaly detection method while being in the same order of magnitude regarding execution time. This evaluation is the first step to demonstrate the accuracy and efficiency of time series classification algorithms for anomaly detection, and represents a strong baseline that can then be used to guide the model selection step in general AutoML pipelines.

Probability-weighted ensemble support vector machine for intelligent recognition of moving wear debris from joint implant

Friction-induced wear debris from joint implants are effective resources in investigating artificial joint wear and cellular immune response mechanisms. To improve the accuracy of wear debris analysis, an intelligent recognition method is developed for wear debris measurement under motion conditions. In this method, the multi-view image sequence of moving wear debris is captured to acquire the variations of aspect ratio, area, and roundness features. Then multiple SVM models are integrated to identify wear debris types based on weighted probability to improve the accuracy. The proposed method can achieve a classification accuracy of 90.51%, which is better than HIVE-COTE2.0, MultiRocket, and other time series classification algorithms. This method can be applied to monitor wear status of artificial joint articulating surfaces.

Animal behavior classification via deep learning on embedded systems

We develop an end-to-end deep-neural-network-based algorithm for classifying animal behavior using accelerometry data on the embedded system of an artificial intelligence of things (AIoT) device installed in a wearable collar tag. The proposed algorithm jointly performs feature extraction and classification utilizing a set of infinite-impulse-response (IIR) and finite-impulse-response (FIR) filters together with a multilayer perceptron. The utilized IIR and FIR filters can be viewed as specific types of recurrent and convolutional neural network layers, respectively. We evaluate the performance of the proposed algorithm via two real-world datasets collected from total eighteen grazing beef cattle using collar tags. The results show that the proposed algorithm offers good intra- and inter-dataset classification accuracy and outperforms its closest contenders including two state-of-the-art convolutional-neural-network-based time-series classification algorithms, which are significantly more complex. We implement the proposed algorithm on the embedded system of the utilized collar tags’ AIoT device to perform in-situ classification of animal behavior. We achieve real-time in-situ behavior inference from accelerometry data without imposing any strain on the available computational, memory, or energy resources of the embedded system.

Towards a novel method for detecting atypical lying down and standing up behaviors in dairy cows using accelerometers and machine learning

Free-stall cubicles are designed so that cows do not defecate in the bedding material, while still providing comfortable lying places. However, fixed cubicle elements, such as the partition and neck rail, restrict available movement space and may hinder cows from performing natural lying down and standing up movement patterns. Although there are various types of cubicle partitions that differ in shape, dimensions, or materials, there is no method other than visual observations to assess their effects on cow welfare. An automated detection system could improve the efficiency and promote objectivity of such assessments. Therefore, the aim of this research was to explore which atypical lying down and standing up behaviors could be detected using body-mounted accelerometers and machine learning. Three leg- and one head-mounted accelerometer set to record at 20 Hz were fitted to 48 lactating dairy cows (Brown Swiss and Holstein × Swiss Fleckvieh). Lying down and standing up events were simultaneously assessed through video observations, by assigning binary presence/absence labels for atypical behaviors, such as lunging the head sideways when standing up and pawing the bedding material before lying down. Different time series classification algorithms were employed for model development using a nested cross-validation strategy. The best performing classifiers were MiniRocket and the deep learning algorithm InceptionTime. Atypical behaviors performed during standing up events, namely Hesitant head lunge and Crawling backwards, were identified as most promising candidates for accelerometer-based detection. These behaviors were detected with balanced accuracies of 0.67 and 0.74, respectively, and their learning curves indicated that more training data might further improve model performance. Overall, achieved performances were not yet satisfactory for application in the evaluation of new dairy cow housing installations. Potentially, ethograms designed for human observers are not optimal for machine learning and adjustments with machine learnability in mind might be necessary. The behaviors identified as promising are good candidates for further development into an efficient and objective method that could complement human observations in the assessment of dairy cow housing installations.

Fuzzy Cognitive Map-Driven Comprehensive Time-Series Classification.

This article presents a comprehensive approach for time-series classification. The proposed model employs a fuzzy cognitive map (FCM) as a classification engine. Preprocessed input data feed the employed FCM. Map responses, after a postprocessing procedure, are used in the calculation of the final classification decision. The time-series data are staged using the moving-window technique to capture the time flow in the training procedure. We use a backward error propagation algorithm to compute the required model hyperparameters. Four model hyperparameters require tuning. Two are crucial for the model construction: 1) FCM size (number of concepts) and 2) window size (for the moving-window technique). Other two are important for training the model: 1) the number of epochs and 2) the learning rate (for training). Two distinguishing aspects of the proposed model are worth noting: 1) the separation of the classification engine from pre- and post-processing and 2) the time flow capture for data from concept space. The proposed classifier joins the key advantage of the FCM model, which is the interpretability of the model, with the superior classification performance attributed to the specially designed pre- and postprocessing stages. This article presents the experiments performed, demonstrating that the proposed model performs well against a wide range of state-of-the-art time-series classification algorithms.

Scalable classifier-agnostic channel selection for multivariate time series classification

Accuracy is a key focus of current work in time series classification. However, speed and data reduction are equally important in many applications, especially when the data scale and storage requirements rapidly increase. Current multivariate time series classification (MTSC) algorithms need hundreds of compute hours to complete training and prediction. This is due to the nature of multivariate time series data which grows with the number of time series, their length and the number of channels. In many applications, not all the channels are useful for the classification task, hence we require methods that can efficiently select useful channels and thus save computational resources. We propose and evaluate two methods for channel selection. Our techniques work by representing each class by a prototype time series and performing channel selection based on the prototype distance between classes. The main hypothesis is that useful channels enable better separation between classes; hence, channels with a larger distance between class prototypes are more useful. On the UEA MTSC benchmark, we show that these techniques achieve significant data reduction and classifier speedup for similar levels of classification accuracy. Channel selection is applied as a pre-processing step before training state-of-the-art MTSC algorithms and saves about 70% of computation time and data storage with preserved accuracy. Furthermore, our methods enable efficient classifiers, such as ROCKET, to achieve better accuracy than using no selection or greedy forward channel selection. To further study the impact of our techniques, we present experiments on classifying synthetic multivariate time series datasets with more than 100 channels, as well as a real-world case study on a dataset with 50 channels. In both cases, our channel selection methods result in significant data reduction with preserved or improved accuracy.

Rethinking attention mechanism in time series classification

The potential of attention mechanisms in time series classification (TSC) is limited owing to general drawbacks, like weak local perception and quadratic complexity. To promote the performance of attention mechanisms, we present a flexible multi-head linear attention (FMLA) architecture, which enhances locality awareness through layer-wise interactions with deformable convolutional blocks and online knowledge distillation. We develop a simple but effective mask mechanism that helps reduce noise influence in time series and reduces the redundancy of FMLA by probabilistically selecting and masking the positions of each given series. We use incremental ablation studies on 85 UCR2018 datasets to evaluate the performance of the main techniques developed. Experimental results demonstrate that FMLA outperformed 11 state-of-the-art TSC algorithms, obtaining a mean accuracy of 89.37%. FMLA achieved the best performance on 29 short-medium and 7 long time-series datasets regarding accuracy. FMLA has a complexity of ON, where N is the sample length. As N increases from 100 to 1000, the floating-point operations per second grow linearly from 0.13G to 1.34G.

Automated Interictal Epileptiform Discharge Detection from Scalp EEG Using Scalable Time-series Classification Approaches.

Deep learning for automated interictal epileptiform discharge (IED) detection has been topical with many published papers in recent years. All existing works viewed EEG signals as time-series and developed specific models for IED classification; however, general time-series classification (TSC) methods were not considered. Moreover, none of these methods were evaluated on any public datasets, making direct comparisons challenging. This paper explored two state-of-the-art convolutional-based TSC algorithms, InceptionTime and Minirocket, on IED detection. We fine-tuned and cross-evaluated them on a public (Temple University Events - TUEV) and two private datasets and provided ready metrics for benchmarking future work. We observed that the optimal parameters correlated with the clinical duration of an IED and achieved the best area under precision-recall curve (AUPRC) of 0.98 and F1 of 0.80 on the private datasets, respectively. The AUPRC and F1 on the TUEV dataset were 0.99 and 0.97, respectively. While algorithms trained on the private sets maintained their performance when tested on the TUEV data, those trained on TUEV could not generalize well to the private data. These results emerge from differences in the class distributions across datasets and indicate a need for public datasets with a better diversity of IED waveforms, background activities and artifacts to facilitate standardization and benchmarking of algorithms.

Distinguishing normal, neuropathic and myopathic EMG with an automated machine learning approach

ObjectiveDistinguishing normal, neuropathic and myopathic electromyography (EMG) traces can be challenging. We aimed to create an automated time series classification algorithm. MethodsEMGs of healthy controls (HC, n = 25), patients with amyotrophic lateral sclerosis (ALS, n = 20) and inclusion body myositis (IBM, n = 20), were retrospectively selected based on longitudinal clinical follow-up data (ALS and HC) or muscle biopsy (IBM). A machine learning pipeline was applied based on 5-second EMG fragments of each muscle. Diagnostic yield expressed as area under the curve (AUC) of a receiver-operator characteristics curve, accuracy, sensitivity, and specificity were determined per muscle (muscle-level) and per patient (patient-level). ResultsDiagnostic yield of the classification ALS vs. HC was: AUC 0.834 ± 0.014 at muscle-level and 0.856 ± 0.009 at patient-level. For the classification HC vs. IBM, AUC was 0.744 ± 0.043 at muscle-level and 0.735 ± 0.029 at patient-level. For the classification ALS vs. IBM, AUC was 0.569 ± 0.024 at muscle-level and 0.689 ± 0.035 at patient-level. ConclusionsAn automated time series classification algorithm can distinguish EMGs from healthy individuals from those of patients with ALS with a high diagnostic yield. Using longer EMG fragments with different levels of muscle activation may improve performance. SignificanceIn the future, machine learning algorithms may help improve the diagnostic accuracy of EMG examinations.

Toward Continuous Breath Monitoring on a Mobile Phone Using a Frugal Conducting Cloth-Based Smart Mask.

A frugal humidity sensor that can detect changes in the humidity of exhaled breath of individuals has been fabricated. The sensor comprises a humidity-sensitive conducting polymer that is in situ formed on a cloth that acts as a substrate. Interdigitated silver electrodes were screen-printed on the modified cloth, and conducting threads connected the electrodes to the measurement circuit. The sensor's response to changing humidity was measured as a voltage drop across the sensor using a microcontroller. The sensor was capable of discerning between fast, normal, and slow breathing based on the response time. A response time of ∼1.3 s was observed for fast breathing. An Android-based mobile application was designed to collect sensor data via Bluetooth for analysis. A time series classification algorithm was implemented to analyze patterns in breathing. The sensor was later stitched onto a face mask, transforming it into a smart mask that can monitor changes in the breathing pattern at work, play, and sleep.