Time Series Classification Problem Research Articles

Derivative dynamic time warping algorithm with introduced correction for varying load fault diagnosis of nuclear power system steam turbine units

The Derivative Dynamic Time Warping (DDTW) algorithm is improved to address the multi-parameters time series classification problem faced by nuclear power system steam turbine units varying load fault diagnosis. Firstly, the entire load changing process is treated as a single sample rather than multiple time-step-samples. This ensures the complete information on the load changing processes, while avoiding interference from normal data fluctuations during faults. Secondly, Time Series Position Coefficient and Time Series Length Coefficient are proposed to correct the DDTW algorithm from two perspectives: the sequences lengths and the data positions in the sequences. This solves the singularities and timeline scaling problems, thereby preventing interference introduced by data sequences' lengths differences and ''similar data appearing at different times'' problem. The nuclear power system steam turbine unit simulation model was built to obtain load changing processes data under normal and faults statuses. In the varying load fault diagnosis test based on these data, the improved DDTW algorithm achieved an accuracy of 1.38%–12.06% higher than other methods, reaching 87.50%. Finally, The Deep Convolutional Generative Adversarial Networks (DCGAN) model was used to generate data to supplement the limited samples of complete load changing processes, and the accuracy of the novel method increased to 95.51% with the increase of data used to support the comparison.

CTCTime: A New Model for Unidimensional Time Series Classification

One-dimensional time series classification has always been an important research direction, playing an irreplaceable role in various fields. With the rapid development of deep learning, most traditional time series classification methods have gradually been replaced by neural network-based classification methods. At present, in the field of time series classification, models that perform well are mostly based on Convolutional Neural Networks (CNNs), while models with the Transformer architecture, which have shown outstanding performance in natural language processing and computer vision, do not stand out. In order to change this situation, We have investigated the feasibility of training models based on a single Transformer architecture directly on small datasets without additional data processing. Furthermore, we propose a new model called CTCTime, which combines the Transformer architecture with CNNs to address one-dimensional time series classification problems. We compared CTCTime with 13 traditional algorithms on 44 datasets from the UCR archive and with 7 advanced methods on 85 datasets. The UCR datasets are classic datasets for one-dimensional time series classification tasks. The experimental results demonstrate its feasibility, accuracy, and scalability.

Residual Echo State Networks: Residual recurrent neural networks with stable dynamics and fast learning

Residual connections have been established as a staple for modern deep learning architectures. Most of their applications are cast towards feedforward computing. In this paper, we study the architectural bias of residual connections in the context of recurrent neural networks (RNNs), specifically in the temporal dimension. We frame our discussion from the perspective of Reservoir Computing and dynamical system theory, focusing on important aspects of neural computation like memory capacity, long-term information processing, stability, and nonlinear computation capability. Experiments corroborate the striking advantage brought by temporal residual connections for a plethora of different time series processing tasks, comprehending memory-based, forecasting, and classification problems.

Structural damage classification in composite materials using the Wigner-Ville distribution and convolutional neural networks

Detecting damage in composite materials is crucial and has received considerable attention in the field of machine learning. However, challenges remain in addressing backbone issues and classifying specific types of damage. This paper presents a novel deep-learning approach for automatically distinguishing delamination and microcracks in Carbon Fiber-Reinforced Polymer (CFRP). The methodology utilizes signals from piezoelectric transducers transformed into time–frequency representations based on the Wigner-Ville Distribution (WVD). Backbone issues were successfully addressed by transitioning the time series classification problem into a computer vision (CV) context through Convolutional Neural Networks (CNN). The analysis included a thorough examination of delamination and microcracks datasets produced experimentally by the National Aeronautics and Space Administration (NASA), focusing on these two types of damage. The proposed methodology achieved a precision range of 98.3 % to 100 % in damage classification, demonstrating its effectiveness for structural health monitoring in composite materials.

Time Series Classification Based on Forward Echo State Convolution Network

The Echo state network (ESN) is an efficient recurrent neural network that has achieved good results in time series prediction tasks. Still, its application in time series classification tasks has yet to develop fully. In this study, we work on the time series classification problem based on echo state networks. We propose a new framework called forward echo state convolutional network (FESCN). It consists of two parts, the encoder and the decoder, where the encoder part is composed of a forward topology echo state network (FT-ESN), and the decoder part mainly consists of a convolutional layer and a max-pooling layer. We apply the proposed network framework to the univariate time series dataset UCR and compare it with six traditional methods and four neural network models. The experimental findings demonstrate that FESCN outperforms other methods in terms of overall classification accuracy. Additionally, we investigated the impact of reservoir size on network performance and observed that the optimal classification results were obtained when the reservoir size was set to 32. Finally, we investigated the performance of the network under noise interference, and the results show that FESCN has a more stable network performance compared to EMN (echo memory network).

Bake off redux: a review and experimental evaluation of recent time series classification algorithms

AbstractIn 2017, a research paper (Bagnall et al. Data Mining and Knowledge Discovery 31(3):606-660. 2017) compared 18 Time Series Classification (TSC) algorithms on 85 datasets from the University of California, Riverside (UCR) archive. This study, commonly referred to as a ‘bake off’, identified that only nine algorithms performed significantly better than the Dynamic Time Warping (DTW) and Rotation Forest benchmarks that were used. The study categorised each algorithm by the type of feature they extract from time series data, forming a taxonomy of five main algorithm types. This categorisation of algorithms alongside the provision of code and accessible results for reproducibility has helped fuel an increase in popularity of the TSC field. Over six years have passed since this bake off, the UCR archive has expanded to 112 datasets and there have been a large number of new algorithms proposed. We revisit the bake off, seeing how each of the proposed categories have advanced since the original publication, and evaluate the performance of newer algorithms against the previous best-of-category using an expanded UCR archive. We extend the taxonomy to include three new categories to reflect recent developments. Alongside the originally proposed distance, interval, shapelet, dictionary and hybrid based algorithms, we compare newer convolution and feature based algorithms as well as deep learning approaches. We introduce 30 classification datasets either recently donated to the archive or reformatted to the TSC format, and use these to further evaluate the best performing algorithm from each category. Overall, we find that two recently proposed algorithms, MultiROCKET+Hydra (Dempster et al. 2022) and HIVE-COTEv2 (Middlehurst et al. Mach Learn 110:3211-3243. 2021), perform significantly better than other approaches on both the current and new TSC problems.

Exploring the Entropy-Based Classification of Time Series Using Visibility Graphs from Chaotic Maps

The classification of time series using machine learning (ML) analysis and entropy-based features is an urgent task for the study of nonlinear signals in the fields of finance, biology and medicine, including EEG analysis and Brain–Computer Interfacing. As several entropy measures exist, the problem is assessing the effectiveness of entropies used as features for the ML classification of nonlinear dynamics of time series. We propose a method, called global efficiency (GEFMCC), for assessing the effectiveness of entropy features using several chaotic mappings. GEFMCC is a fitness function for optimizing the type and parameters of entropies for time series classification problems. We analyze fuzzy entropy (FuzzyEn) and neural network entropy (NNetEn) for four discrete mappings, the logistic map, the sine map, the Planck map, and the two-memristor-based map, with a base length time series of 300 elements. FuzzyEn has greater GEFMCC in the classification task compared to NNetEn. However, NNetEn classification efficiency is higher than FuzzyEn for some local areas of the time series dynamics. The results of using horizontal visibility graphs (HVG) instead of the raw time series demonstrate the GEFMCC decrease after HVG time series transformation. However, the GEFMCC increases after applying the HVG for some local areas of time series dynamics. The scientific community can use the results to explore the efficiency of the entropy-based classification of time series in “The Entropy Universe”. An implementation of the algorithms in Python is presented.

Monitoring domestic water consumption: a comparative study of model-based and data-driven end-use disaggregation methods

ABSTRACT Monitoring the water usage of different appliances and informing consumers about it has been shown to have an impact on their behavior toward drinking water conservation. The most practical and cost-effective way to accomplish this is through a non-intrusive approach, that locally analyzes data received from a flow sensor at the main water supply pipe of a household. In this work, we present two different methods addressing the challenges of disaggregating end-use consumption and classifying consumption events. The first method is model-based (MB) and uses a combination of dynamic time wrapping and statistical bounds to analyze four water end-use characteristics. The second, learning-based (LB) method is data-driven and formulates the problem as a time-series classification problem without relying on a priori identification of events. We perform an extensive computational study that includes a comparison between an MB and an LB method, as well as an experimental study to demonstrate the application of the LB method on an edge computing device. Both methods achieve similar F1 scores (LB: 71.73%, MB: 71.04%) with the LB being more precise. The embedded LB method achieves a slightly higher score (72.01%) while enhancing on-site real-time processing, improving security and privacy and enabling cost savings.

Fault detection in the gas turbine of the Kirkuk power plant: An anomaly detection approach using DLSTM-Autoencoder

Developing a maintenance schedule for different parts of a power plant can help to prevent serious system damage while also having a direct effect on reducing maintenance time and costs. Moreover, the proposed procedure can provide a better understanding of the health state of the system. The occurrence of problems (anomalous behavior) in power plants usually takes time and happens gradually. Predicting the possibility of anomalous behavior arising can therefore result in the prediction of a failure. In this paper, the authors tried to propose a new method for reducing the volume of input data by extracting the latent layer output of an Autoencoder, which was then applied to identify features. We achieved this by keeping relevant features and learning the anomalous behavior's encoded representation. Furthermore, the latent layer output from the encoder was used to train a Deep Long-Short-Term Memory (DLSTM) model with three hidden layers for fault occurrence prediction by considering an anomaly detection approach. To this end, a threshold approach was used to determine the possibility of a fault occurring in the turbine, with data collected from the Kirkuk gas turbine power plant. Piezoelectric accelerometer CA 202 was utilized as a sensor to collect data in industry. Since most of the recorded data were normal and the amount of abnormal data was small, various techniques, i.e., flipping and jittering, were employed to create synthetic abnormal data and augment the applicable dataset. In summary, according to the proposed methodology, the system is modelled as a time series classification problem to mimic the anomaly detection problem. The results showed that the accuracy of the trained model for detecting turbine vibration as a system abnormality is over 86%. Also, the implementation of the proposed prediction model on the experimental dataset yielded satisfactory results for detecting and predicting the occurrence of faults in Kirkuk gas turbine power plant.

Residual Convolutional Attention Model With Transfer Learning for Detecting Multianomalous Features in Structural Vibration Data

In response to the data anomalies and frequent false alarms caused by harsh environments in long‐term structural health monitoring (SHM), this study has reframed the detection of abnormal vibration data as a time series classification problem. This approach identifies multiple anomalous features, thereby reducing manual detection costs. The novel developed Convolutional Neural Network with Squeeze‐and‐Excitation and Multi‐Head Self‐Attention (CNN–SE–MHSA) employs a deep residual network structure with channel and spatial attention mechanisms, effectively handling the global long‐term dependencies required for anomaly feature learning. It better understands and utilizes feature information across different levels and dimensions, enhancing classification accuracy in complex anomaly situations. Through t‐SNE dimensionality reduction visualization and interpretability analysis, it is demonstrated that the model excels in identifying critical features. Furthermore, by generating simulated data with a variational autoencoder (VAE) and implementing transfer learning strategies based on these data, the issue of low recognition accuracy for complex anomaly data due to data imbalance can be effectively mitigated. In a 25‐day long‐term monitoring experiment of indoor tunnel lining structures, this method demonstrated an average accuracy rate exceeding 96% and a rapid detection capability within 16 min. The results indicate that this method achieves high accuracy in anomaly detection for long‐term monitoring data, even when relying exclusively on time‐domain data.

Time series quantum classifiers with amplitude embedding

Quantum Machine Learning was born during the past decade as the intersection of Quantum Computing and Machine Learning. Today, advances in quantum computer hardware and the design of simulation frameworks able to run quantum algorithms in classic computers make it possible to extend classic artificial intelligence models to a quantum environment. Despite these achievements, several questions regarding the whole quantum machine learning pipeline remain unanswered, for instance the problem of classical data representation on quantum hardware, or the methodologies for designing and evaluating quantum models for common learning tasks such as classification, function approximation, clustering, etc. These problems become even more difficult to solve in the case of Time Series processing, where the context of past historical data may influence the behavior of the decision-making model. In this piece of research, we address the problem of Time Series classification using quantum models, and propose an efficient and compact representation of time series in quantum data using amplitude embedding. The proposal is capable of representing a time series of length n in log2(n)\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$log_2(n)$$\\end{document} computational units, and experiments conducted on benchmark time series classification problems show that quantum models designed for classification can also outperform the accuracy of classic methods.

The Danger of Minimum Exposures: Understanding Cross-App Information Leaks on iOS through Multi-Side-Channel Learning.

Research on side-channel leaks has long been focusing on the information exposure from a single channel (memory, network traffic, power, etc.). Less studied is the risk of learning from multiple side channels related to a target activity (e.g., website visits) even when individual channels are not informative enough for an effective attack. Although the prior research made the first step on this direction, inferring the operations of foreground apps on iOS from a set of global statistics, still less clear are how to determine the maximum information leaks from all target-related side channels on a system, what can be learnt about the target from such leaks and most importantly, how to control information leaks from the whole system, not just from an individual channel. To answer these fundamental questions, we performed the first systematic study on multi-channel inference, focusing on iOS as the first step. Our research is based upon a novel attack technique, called Mischief, which given a set of potential side channels related to a target activity (e.g., foreground apps), utilizes probabilistic search to approximate an optimal subset of the channels exposing most information, as measured by Merit Score, a metric for correlation-based feature selection. On such an optimal subset, an inference attack is modeled as a multivariate time series classification problem, so the state-of-the-art deep-learning based solution, InceptionTime in particular, can be applied to achieve the best possible outcome. Mischief is found to work effectively on today's iOS (16.2), identifying foreground apps, website visits, sensitive IoT operations (e.g., opening the door) with a high confidence, even in an open-world scenario, which demonstrates that the protection Apple puts in place against the known attack is inadequate. Also importantly, this new understanding enables us to develop more comprehensive protection, which could elevate today's side-channel research from suppressing leaks from individual channels to controlling information exposure across the whole system.

ADF & TransApp: A Transformer-Based Framework for Appliance Detection Using Smart Meter Consumption Series

Over the past decade, millions of smart meters have been installed by electricity suppliers worldwide, allowing them to collect a large amount of electricity consumption data, albeit sampled at a low frequency (one point every 30min). One of the important challenges these suppliers face is how to utilize these data to detect the presence/absence of different appliances in the customers' households. This valuable information can help them provide personalized offers and recommendations to help customers towards the energy transition. Appliance detection can be cast as a time series classification problem. However, the large amount of data combined with the long and variable length of the consumption series pose challenges when training a classifier. In this paper, we propose ADF, a framework that uses subsequences of a client consumption series to detect the presence/absence of appliances. We also introduce TransApp, a Transformer-based time series classifier that is first pretrained in a self-supervised way to enhance its performance on appliance detection tasks. We test our approach on two real datasets, including a publicly available one. The experimental results with two large real datasets show that the proposed approach outperforms current solutions, including state-of-the-art time series classifiers applied to appliance detection.

Minimum Recall-Based Loss Function for Imbalanced Time Series Classification

This paper deals with imbalanced time series classification problems. In particular, we propose to learn time series classifiers that maximize the minimum recall of the classes rather than the accuracy. Consequently, we manage to obtain classifiers which tend to give the same importance to all the classes. Unfortunately, for most of the traditional classifiers, learning to maximize the minimum recall of the classes is not trivial (if possible), since it can distort the nature of the classifiers themselves. Neural networks, in contrast, are classifiers that explicitly define a loss function, allowing it to be modified. Given that the minimum recall is not a differentiable function, and therefore does not allow the use of common gradient-based learning methods, we apply and evaluate several smooth approximations of the minimum recall function. A thorough experimental evaluation shows that our approach improves the performance of state-of-the-art methods used in imbalanced time series classification, obtaining higher recall values for the minority classes, incurring only a slight loss in accuracy.

Multiclass Sparse Centroids With Application to Fast Time Series Classification.

In this article, we propose an efficient multiclass classification scheme based on sparse centroids classifiers. The proposed strategy exhibits linear complexity with respect to both the number of classes and the cardinality of the feature space. The classifier we introduce is based on binary space partitioning, performed by a decision tree where the assignation law at each node is defined via a sparse centroid classifier. We apply the presented strategy to the time series classification problem, showing by experimental evidence that it achieves performance comparable to that of state-of-the-art methods, but with a significantly lower classification time. The proposed technique can be an effective option in resource-constrained environments where the classification time and the computational cost are critical or, in scenarios, where real-time classification is necessary.

DEEP LEARNING FOR SIMULTANEOUS IMPUTATION AND CLASSIFICATION OF TIME SERIES INCOMPLETE DATA

Classification with time series data has many practical applications in finance, medicine, manufacturing, energy, transportation, and agriculture. However, time series data is often plagued by missing values, which not only decreases the accuracy but also increases the complexity of classification models. A common approach to address classification with missing data is to use imputation methods to estimate the missing values before constructing classification models. Recurrent neural network models have been shown to be effective in estimating missing values for time series data. On the other hand, convolutional neural network models have demonstrated their effectiveness in building classification models for time series data. However, there has been no research that specifically addresses how to combine these two types of models to simultaneously address the problem of time series classification with incomplete data. Therefore, in this paper, a model combining recurrent neural networks and convolutional neural networks is proposed to build a model that can simultaneously estimate missing values and classify time series data. The experimental results demonstrate that the proposed model performs better than the existing methods for time series classification with incomplete data.

Suspicious Behavior Detection with Temporal Feature Extraction and Time-Series Classification for Shoplifting Crime Prevention.

The rise in crime rates in many parts of the world, coupled with advancements in computer vision, has increased the need for automated crime detection services. To address this issue, we propose a new approach for detecting suspicious behavior as a means of preventing shoplifting. Existing methods are based on the use of convolutional neural networks that rely on extracting spatial features from pixel values. In contrast, our proposed method employs object detection based on YOLOv5 with Deep Sort to track people through a video, using the resulting bounding box coordinates as temporal features. The extracted temporal features are then modeled as a time-series classification problem. The proposed method was tested on the popular UCF Crime dataset, and benchmarked against the current state-of-the-art robust temporal feature magnitude (RTFM) method, which relies on the Inflated 3D ConvNet (I3D) preprocessing method. Our results demonstrate an impressive 8.45-fold increase in detection inference speed compared to the state-of-the-art RTFM, along with an F1 score of 92%,outperforming RTFM by 3%. Furthermore, our method achieved these results without requiring expensive data augmentation or image feature extraction.

Local morphological patterns for time series classification

The key problem of time series classification is the similarity measure between time series. In recent years, efficient and accurate similarity measurement methods of time series have attracted extensive attention from researchers. According to the different similarity measure strategies, the existing time series classification methods can be roughly divided into shape-based (original value) methods and structure-based (symbol transformation) methods. Shape-based methods usually use Euclidean distance (ED), dynamic time warping (DTW), or other methods to measure the global similarity between sequences. The disadvantage of these methods is that their measurement process does not necessarily achieve local sensible matchings of time series, which leads to a decrease in their accuracy and interpretability. To better capture the local information of the sequence, the structure-based methods discretize or symbolize the local value of the time sequence, which leads to the loss of the original information of the sequence. To address these problems, this paper proposes a novel similarity measurement method named dynamic time warping based on the local morphological pattern (MPDTW), which first decomposes the local subsequences of time series using discrete wavelet transforms for extracting the local structure information. Then, the decomposed subsequence will be encoded by the morphological pattern. Finally, the ED between points and their local structure difference based on morphological pattern will be weighted and applied to the DTW algorithm to measure the similarity between sequences. Experiments have been carried out on the classification tasks of the UCR datasets and the results show that our method outperforms the existing baselines.

Neural Network Entropy (NNetEn): Entropy-Based EEG Signal and Chaotic Time Series Classification, Python Package for NNetEn Calculation

Entropy measures are effective features for time series classification problems. Traditional entropy measures, such as Shannon entropy, use probability distribution function. However, for the effective separation of time series, new entropy estimation methods are required to characterize the chaotic dynamic of the system. Our concept of Neural Network Entropy (NNetEn) is based on the classification of special datasets in relation to the entropy of the time series recorded in the reservoir of the neural network. NNetEn estimates the chaotic dynamics of time series in an original way and does not take into account probability distribution functions. We propose two new classification metrics: R2 Efficiency and Pearson Efficiency. The efficiency of NNetEn is verified on separation of two chaotic time series of sine mapping using dispersion analysis. For two close dynamic time series (r = 1.1918 and r = 1.2243), the F-ratio has reached the value of 124 and reflects high efficiency of the introduced method in classification problems. The electroencephalography signal classification for healthy persons and patients with Alzheimer disease illustrates the practical application of the NNetEn features. Our computations demonstrate the synergistic effect of increasing classification accuracy when applying traditional entropy measures and the NNetEn concept conjointly. An implementation of the algorithms in Python is presented.

Automated machine learning approach for time series classification pipelines using evolutionary optimization

Automated machine learning has the ability to improve the efficiency of time series classification due to the ability to combine multiple feature extraction methods and classification models. In the paper, we propose a flexible AutoML approach that combines multiple feature generation strategies (spectral, wavelet, topological, quantile) and classifiers as parts of the modeling pipeline. It allows obtaining a more robust and lightweight solution for the time series classification problem. Generation of the pipeline is based on an evolutionary algorithm.Comparison with approaches with the highest results was conducted on the UEA/UCR archive for modeling quality analysis. The proposed approach allows, on the one hand, to solve the task of time series classification automatically and could be used as part of an industrial data processing pipeline. On the other hand, it could be used as a tool for modeling and studying natural process properties described as time series.