Segmentation Of Multivariate Time Series Research Articles

SIMTSeg: A self-supervised multivariate time series segmentation method with periodic subspace projection and reverse diffusion for industrial process

Subsequences with varied regimes in the industrial multivariate time series (MTS) are closely associated with the dynamic status of the multi-phased industrial process. Time series segmentation (TSS) provides insights into the underlying behavior of industrial systems. However, the complexity of industrial data poses significant challenges to the conventional TSS methods. Motivated by this, a Self-supervised Industrial Multivariate Time-series Segmentation method (SIMTSeg) is presented in this work. An MTS folding module based on Ramanujan periodic subspace projection is first proposed, where the MTS is reshaped into the 3D feature map to realize the compact representation of the intricate data dependencies. Subsequently, a self-supervised module based on the encoder-decoder architecture is adopted to address the problem of deficient and task-specific annotations in industrial data. The folded feature map is denoised step by step following the reverse diffusion process, and finally turns into the segmentation mask without redundant details. The proposed SIMTSeg has been validated by a popular industrial benchmark, the Tennessee Eastman Process, and outperforms the unsupervised data-driven baselines in terms of various performance metrics. SIMTSeg has no prerequisite on the number of segmentation points or regime types, and is capable of giving more meaningful segmentation results that are in line with the high-level semantics.

A Total Variation Based Method for Multivariate Time Series Segmentation

A Total Variation Based Method for Multivariate Time Series Segmentation

A Multivariate Time-Series Segmentation Framework for Flight Condition Recognition

Helicopters usage monitoring has gained significant attention in recent years, due to the safety and cost management implications. At its core there is the Flight Condition Recognition algorithm, which enables to detect the maneuvers carried out by the aircraft through on-board sensors measurements. In this work we propose a multi-variate time-series segmentation framework, which uses supervised learning algorithms, sliding windows and stacking ensembles to produce reliable estimates of the flown flight regimes. We validate the proposed approach on a large dataset of 460 labeled load flights from two distinct helicopter models, demonstrating its efficacy in predicting a range of 49 different maneuver types.

Industrial Transfer Learning for Multivariate Time Series Segmentation: A Case Study on Hydraulic Pump Testing Cycles.

Industrial data scarcity is one of the largest factors holding back the widespread use of machine learning in manufacturing. To overcome this problem, the concept of transfer learning was developed and has received much attention in recent industrial research. This paper focuses on the problem of time series segmentation and presents the first in-depth research on transfer learning for deep learning-based time series segmentation on the industrial use case of end-of-line pump testing. In particular, we investigate whether the performance of deep learning models can be increased by pretraining the network with data from other domains. Three different scenarios are analyzed: source and target data being closely related, source and target data being distantly related, and source and target data being non-related. The results demonstrate that transfer learning can enhance the performance of time series segmentation models with respect to accuracy and training speed. The benefit can be most clearly seen in scenarios where source and training data are closely related and the number of target training data samples is lowest. However, in the scenario of non-related datasets, cases of negative transfer learning were observed as well. Thus, the research emphasizes the potential, but also the challenges, of industrial transfer learning.

The Contours of a Cliophysics. How Can Econophysics Enrich Cliometrics? Case Studies in Debt Issues and Global Capital Markets

In this article we illustrate what the contours of “Cliophysics” are. The term was coined by the “Cliometrics and Complexity” (CAC) team, composed of economists and physicists, and hosted by the Complex Systems Institute of ENS Lyon (IXXI). Cliophysics consists in the application of methodological tools from Complexity and Econophysics to the field of Economic history. More precisely, this new scientific challenge aims at combining Cliometrics, i.e. the application of economic modelling and econometrics to the field of Economic history, together with Complexity analysis and Econophysics. In this article, we highlight how using statistical signal processing, topological analysis and network analysis can enrich Cliometrics. To that end, we present three case studies taken from the research agenda developed within the CAC Team: 1) An original research in Cliophysics involving coïncidence analysis based on symbolic time series, aiming to revisit the classical standards of the link between public debt and economic growth in Economic history; 2) An approach in Cliophysics involving topological representations of asset markets networks, aiming to detect homogeneous eras of international monetary regimes; 3) An approach in Cliophysics involving the segmentation of multivariate time series derived from assets returns, aiming to allow for the identification of the structural breaks in the history of global capital markets. We provide evidence that Cliophysics 1) reveals new stylized facts in Economic history, 2) unveils key moments in the history of capital flows, different from the conventional view, and 3) fills a gap in historical analysis, by focusing on “structure matters in Economic history”.

A multivariate time series segmentation algorithm for analyzing the operating statuses of tunnel boring machines

The segmentation of tunnel boring machine (TBM) time series plays a crucial role in analyzing TBM operating statuses and mining potential information from the collected signals. In this paper, a novel algorithm named SDP–RLR is proposed to segment multivariate TBM time series with outliers caused by harsh operating environments and changeable tunneling statuses. In this algorithm, the 3σ rule is extended to externally studentized residuals of a linear regression and used to identify the outliers in each segment of the input time series. An outlier removal penalty term is added to the segment errors to avoid regarding borderline data points as outliers. As a segmentation optimization algorithm, dynamic programming (DP) is improved to scalable dynamic programming (SDP) in this paper by combining time series in two stages to reduce computational costs. In the first stage, consecutive time points are integrated to reduce the calculations required for DP, while the second stage refines the segmentation results obtained from the first stage. Experiments are conducted on synthetic datasets to evaluate the performance of the SDP–RLR algorithm, and a complete TBM time series in terms of three key variables validates its effectiveness for segmentation and outlier detection tasks. The SDP–RLR algorithm can segment a TBM time series into four different statuses, which assists in analyzing the operating statuses of TBMs. In addition, comparative experiments indicate that the proposed algorithm can handle the segmentation of complex TBM time series when they cannot be successfully processed using the segmentation approach after outlier detection.

Memetic algorithm for multivariate time-series segmentation

In recent years, analyzing time-series data has become an ever important research topic due to an increased number of temporal datasets in science and engineering. Segmentation is widely used in time-series data analysis because it provides a more compact representation by dividing the series into segments. Segmentation approaches based on genetic algorithms have been proposed to extract segments and patterns with a given objective, such as a low rate of change or periodicity from time-series data. However, they may not be effective in obtaining the precise solution because they perform global search. In this study, we propose a memetic algorithm for multivariate time-series segmentation. For efficient local refinement, we calculate a likelihood-based score for all time points and use it in the evolutionary process. Experiments demonstrate that the proposed method is superior to conventional segmentation methods. The source code of the proposed method can be downloaded from https://github.com/hlim-kist/ma_mts

Adaptive G–G clustering for fuzzy segmentation of multivariate time series

In this paper, Gath–Geva (G–G) fuzzy clustering is extended to adaptively segment hydrometeorological multivariate time series. First, KPCA is used to extract principle components of multivariate time series to remove the impacts of redundant and irrelevant variables. Then, taking the time information into account, the segmentation of principle components of multivariate time series is derived with the modified Davies–Bouldin Index and adaptive G–G fuzzy clustering. In the experiment, our proposed algorithm is applied on the real-world hydrometeorological multivariate time series collected every 6 min with length $$N=720$$ . Comparison with the existing segmentation algorithms, our proposed algorithm proves the applicability and usefulness in hydrometeorological multivariate time series analysis.

Hybrid dynamic learning mechanism for multivariate time series segmentation

AbstractTo improve the efficiency of segmentation methods for multivariate time series, a hybrid dynamic learning mechanism for such series' segmentation is proposed. First, an incremental clustering algorithm is used to automatically cluster variables of multivariate time series. Second, common factors are extracted from every cluster by a dynamic factor model as an ensemble description of the system. Third, this common factor series is segmented by dynamic programming. The proposed method can potentially segment multivariate time series and not only performs segmentation better on multivariate time series with a large number of variables but also improves the running accuracy and efficiency of the algorithm, especially when analyzing complex datasets.

Representation learning for unsupervised heterogeneous multivariate time series segmentation and its application

While driving a vehicle, data are collected from a huge number of sensors that generate both categorical and continuous variables with varying scales. In order to understand the status of the vehicles and the drivers’ behaviors, it is crucial to segment and identify different phases within this time series data. However, data often lacks labels to denote different phases, rendering supervised learning based segmentation methods as futile. Consequently, distance based time series segmentation method is a realistic solution for detecting different phases in the sensor data. However, there is no universal distance measure that utilizes both categorical and continuous variables simultaneously to segment the multivariate data.In this paper, we propose a novel unsupervised time series segmentation framework for heterogeneous multivariate data. By applying the distributed representation of the word embedding methods, we transform multivariate heterogeneous data into continuous vectors, allowing them to be segmented by conventional distance metrics such as Euclidean or Cosine distance. Subsequently, similar segments are clustered to generate general patterns. Without any labels or feature engineering, our framework successfully segments and discovers insightful driving patterns from heterogeneous sensor data collected from actual vehicles.

Greedy Gaussian segmentation of multivariate time series

We consider the problem of breaking a multivariate (vector) time series into segments over which the data is well explained as independent samples from a Gaussian distribution. We formulate this as a covariance-regularized maximum likelihood problem, which can be reduced to a combinatorial optimization problem of searching over the possible breakpoints, or segment boundaries. This problem can be solved using dynamic programming, with complexity that grows with the square of the time series length. We propose a heuristic method that approximately solves the problem in linear time with respect to this length, and always yields a locally optimal choice, in the sense that no change of any one breakpoint improves the objective. Our method, which we call greedy Gaussian segmentation (GGS), easily scales to problems with vectors of dimension over 1000 and time series of arbitrary length. We discuss methods that can be used to validate such a model using data, and also to automatically choose appropriate values of the two hyperparameters in the method. Finally, we illustrate our GGS approach on financial time series and Wikipedia text data.

Batch Process Monitoring Based on Fuzzy Segmentation of Multivariate Time-Series

Batch Process Monitoring Based on Fuzzy Segmentation of Multivariate Time-Series

The use of artificial neural network segmentation of multivariate time series for the analysis of geomagnetic disturbances

This study is devoted to a preliminary analysis of the segmentation of multivariate time series of the parameters of the solar wind, interplanetary magnetic field, and the Dst index using artificial neural networks, for the period from November 1997 to March 2014. It is shown that this technique allows progress in understanding the dynamics of the Earth’s magnetosphere.

Dynamic programming approach for segmentation of multivariate time series

In this paper, dynamic programming (DP) algorithm is applied to automatically segment multivariate time series. The definition and recursive formulation of segment errors of univariate time series are extended to multivariate time series, so that DP algorithm is computationally viable for multivariate time series. The order of autoregression and segmentation are simultaneously determined by Schwarz’s Bayesian information criterion. The segmentation procedure is evaluated with artificially synthesized and hydrometeorological multivariate time series. Synthetic multivariate time series are generated by threshold autoregressive model, and in real-world multivariate time series experiment we propose that besides the regression by constant, autoregression should be taken into account. The experimental studies show that the proposed algorithm performs well.

Joint segmentation of multivariate time series with hidden process regression for human activity recognition

The problem of human activity recognition is central for understanding and predicting the human behavior, in particular in a prospective of assistive services to humans, such as health monitoring, well being, security, etc. There is therefore a growing need to build accurate models which can take into account the variability of the human activities over time (dynamic models) rather than static ones which can have some limitations in such a dynamic context. In this paper, the problem of activity recognition is analyzed through the segmentation of the multidimensional time series of the acceleration data measured in the 3-d space using body-worn accelerometers. The proposed model for automatic temporal segmentation is a specific statistical latent process model which assumes that the observed acceleration sequence is governed by sequence of hidden (unobserved) activities. More specifically, the proposed approach is based on a specific multiple regression model incorporating a hidden discrete logistic process which governs the switching from one activity to another over time. The model is learned in an unsupervised context by maximizing the observed-data log-likelihood via a dedicated expectation–maximization (EM) algorithm. We applied it on a real-world automatic human activity recognition problem and its performance was assessed by performing comparisons with alternative approaches, including well-known supervised static classifiers and the standard hidden Markov model (HMM). The obtained results are very encouraging and show that the proposed approach is quite competitive even it works in an entirely unsupervised way and does not requires a feature extraction preprocessing step.

Multivariate time series segmentation for generalized description of dynamic systems operation

In this paper a class of systems that can be in several typical operation modes is considered. The goal of the research is to build a generalized description of such systems. The description is understood as segmentation of multivariate time series of experimental data and identification of typical states models. Two approaches to solving the problem are proposed. The first one is based on evolution models, and the second builds segments using some simple elements. The algorithms are tested on simulated data and applied to describe the behavior of animals in biological experiments.

Structural Break Estimation for Nonstationary Time Series Models

This article considers the problem of modeling a class of nonstationary time series using piecewise autoregressive (AR) processes. The number and locations of the piecewise AR segments, as well as the orders of the respective AR processes, are assumed unknown. The minimum description length principle is applied to compare various segmented AR fits to the data. The goal is to find the “best” combination of the number of segments, the lengths of the segments, and the orders of the piecewise AR processes. Such a “best” combination is implicitly defined as the optimizer of an objective function, and a genetic algorithm is implemented to solve this difficult optimization problem. Numerical results from simulation experiments and real data analyses show that the procedure has excellent empirical properties. The segmentation of multivariate time series is also considered. Assuming that the true underlying model is a segmented autoregression, this procedure is shown to be consistent for estimating the location of the breaks.

Modified Gath–Geva clustering for fuzzy segmentation of multivariate time-series

Partitioning a time-series into internally homogeneous segments is an important data-mining problem. The changes of the variables of a multivariate time-series are usually vague and do not focus on any particular time point. Therefore, it is not practical to define crisp bounds of the segments. Although fuzzy clustering algorithms are widely used to group overlapping and vague objects, they cannot be directly applied to time-series segmentation, because the clusters need to be contiguous in time. This paper proposes a clustering algorithm for the simultaneous identification of local probabilistic principal component analysis (PPCA) models used to measure the homogeneity of the segments and fuzzy sets used to represent the segments in time. The algorithm favors contiguous clusters in time and is able to detect changes in the hidden structure of multivariate time-series. A fuzzy decision making algorithm based on a compatibility criteria of the clusters has been worked out to determine the required number of segments, while the required number of principal components are determined by the screeplots of the eigenvalues of the fuzzy covariance matrices. The application example shows that this new technique is a useful tool for the analysis of historical process data.