Multivariate Time Series Clustering Research Articles

Multivariate time series clustering based on fuzzy cognitive maps and community detection

Most time series clustering methods mainly focus on univariate time series (UTS). Compared with UTS, multivariate time series (MTS) consists of multiple components. Although interest in MTS clustering is increasing, its performance is far from satisfactory. Most traditional MTS clustering methods may have two limitations. First, they do not consider both the temporal features of each component and the relationship between the components. Second, they can only identify the local relationship between adjacent data, but cannot obtain long-distance global relationships and capture clusters of arbitrary shapes. In this paper, we develop a method for MTS clustering based on fuzzy cognitive maps (FCMs) and community detection, termed as MTSC-FCM-CD. To overcome the first limitation, we use FCM to represent MTS; FCM can extract temporal features while preserving the relationship between components in MTS. To overcome the second limitation, we use the community detection algorithm to cluster the global relations, which is different from the traditional nearest neighbor distance-based method. In the calculation process, the similarity between FCMs should be used to build a complex network. Existing methods calculate the similarity between FCMs, only considering the numerical characteristics but ignoring the topological structure. To solve this problem, we design a two-stage similarity measure to build a complex network. In comparison to the existing methods, the experimental results on twenty benchmark datasets demonstrate the effectiveness of MTSC-FCM-CD in MTS clustering.

Multivariate Time Series Feature Extraction and Clustering Framework for Multi-Function Radar Work Mode Recognition

Multi-Function Radars (MFRs) are sophisticated sensors with great agility and flexibility in adapting their transmitted waveform and control parameters. The recognition of MFR work modes based on the intercepted pulse sequences plays an important role in interpreting the functional purpose and threats of a non-cooperative MFRs. However, due to the increased flexibility of MFRs, radar work modes with emerging new modulations and control parameters always appear, and the supervised classification method suffers performance degradation or even failure. Unsupervised learning and clustering of MFR pulse sequences becomes urgent and important. This paper establishes a unified multivariate MFR time series feature extraction and clustering framework for MFR work mode recognition. At first, various features are collected to form the feature set. The feature set includes features extracted through deep learning based on recurrent auto-encoders, multidimensional time series toolkit features, and manually crafted features for radar inter-pulse modulations. Subsequently, several feature selection algorithms, combined with different clustering and classification methods, are used for the selection of an “optimal” feature subset. Finally, the effectiveness and superiority of the proposed framework and selected features are validated through simulated and measured datasets. In the simulated dataset containing 20 classes of work modes, under the most severe non-ideal conditions, we achieve a clustering purity of 73.46% and an NMI of 84.28%. In the measured dataset with seven classes of work modes, we achieve a clustering purity of 86.96% and an NMI of 90.10%.

Improving demand forecasting for customers with missing downstream data in intermittent demand supply chains with supervised multivariate clustering

AbstractIn a collaborative supply chain arrangement like vendor‐managed inventory, information on product demand at the point of sale is expected to be shared among members of the supply chain. However, in practice, obtaining such information can be costly, and some members may be unwilling or unable to provide the necessary access to the data. As such, large collaborative supply chains with multiple members may operate under a mixed‐information scenario where point‐of‐sale demand information is not known for all customers. Other sources of demand information exist and are becoming more available along supply chains using Industry 4.0 technologies and can serve as a substitute, but the data may be noisy, distorted, and partially missing. Under mixed information, leveraging existing customers' point‐of‐sale demand to improve the intermittent demand forecast of customers with missing information has yet to be explored. We propose a supervised demand forecasting method that uses multivariate time series clustering to map multiple sources of demand data. Members with missing downstream demand data have their resulting demand forecast improved by averaging over customers with similar delivery patterns for their final demand forecast. Our results show up to a 10% accuracy improvement over traditional intermittent demand forecasting methods with missing information.

Time series clustering to improve one-class classifier performance

The improvement of one-class classifiers’ performance through clustering of multivariate time series is considered in this paper. Datasets arising from real processes come from the available sensors and are affected by many factors, such as aging of the process, changes in the operation region, and equipment malfunction. Despite that, one expects that the classes represented by such diverse data can be unveiled via trained classifiers. This work hypothesizes that the overall performance can be improved by training sets of one-class classifiers with subsets of data clustered by similarity. The proposed method is applied to one class classifiers since they are trained only with the target class, which is clustered based on time series similarity using Dynamic Time Warping and k-means. The advantages of the techniques are illustrated through their application to a public dataset from the oil industry with instances characterizing eight classes of data represented by five time series. Seven classes are selected to train LSTM classifiers using the variables and instances clustered using time series clustering algorithms. The results show that the increase in the similarity of training data tends to improve the performance of the LSTM classifier, achieving an increase of 10% in the overall performance. In a specific case, where the clustering model raised the similarity by 84%, the classification performance improved by 21%.

Time-Varying Gaussian Markov Random Fields Learning for Multivariate Time Series Clustering

Time-Varying Gaussian Markov Random Fields Learning for Multivariate Time Series Clustering

Short-term exposure sequences and anxiety symptoms: a time series clustering of smartphone-based mobility trajectories

BackgroundShort-term environmental exposures, including green space, air pollution, and noise, have been suggested to affect health. However, the evidence is limited to aggregated exposure estimates which do not allow the capture of daily spatiotemporal exposure sequences. We aimed to (1) determine individuals’ sequential exposure patterns along their daily mobility paths and (2) examine whether and to what extent these exposure patterns were associated with anxiety symptoms.MethodsWe cross-sectionally tracked 141 participants aged 18–65 using their global positioning system (GPS) enabled smartphones for up to 7 days in the Netherlands. We estimated their location-dependent exposures for green space, fine particulate matter, and noise along their moving trajectories at 10-min intervals. The resulting time-resolved exposure sequences were then partitioned using multivariate time series clustering with dynamic time warping as the similarity measure. Respondents’ anxiety symptoms were assessed with the Generalized Anxiety Disorders-7 questionnaire. We fitted linear regressions to assess the associations between sequential exposure patterns and anxiety symptoms.ResultsWe found four distinctive daily sequential exposure patterns across the participants. Exposure patterns differed in terms of exposure levels and daily variations. Regression results revealed that participants with a “moderately health-threatening” exposure pattern were significantly associated with fewer anxiety symptoms than participants with a “strongly health-threatening” exposure pattern.ConclusionsOur findings support that environmental exposures’ daily sequence and short-term magnitudes may be associated with mental health. We urge more time-resolved mobility-based assessments in future analyses of environmental health effects in daily life.

Soft Subspace Based Ensemble Clustering for Multivariate Time Series Data.

Recently, multivariate time series (MTS) clustering has gained lots of attention. However, state-of-the-art algorithms suffer from two major issues. First, few existing studies consider correlations and redundancies between variables of MTS data. Second, since different clusters usually exist in different intrinsic variables, how to efficiently enhance the performance by mining the intrinsic variables of a cluster is challenging work. To deal with these issues, we first propose a variable-weighted K-medoids clustering algorithm (VWKM) based on the importance of a variable for a cluster. In VWKM, the proposed variable weighting scheme could identify the important variables for a cluster, which can also provide knowledge and experience to related experts. Then, a Reverse nearest neighborhood-based density Peaks approach (RP) is proposed to handle the problem of initialization sensitivity of VWKM. Next, based on VWKM and the density peaks approach, an ensemble Clustering framework (SSEC) is advanced to further enhance the clustering performance. Experimental results on ten MTS datasets show that our method works well on MTS datasets and outperforms the state-of-the-art clustering ensemble approaches.

Space-time relationships between COVID-19 vaccinations and human mobility patterns in the United States

As COVID-19 vaccines were administered in early 2021, they helped to mitigate the spread of the COVID-19 virus and signaled an important shift in the pandemic. To better understand how ongoing COVID-19 vaccinations were related to human mobility across the U.S., we identified different mobility-vaccination profiles between January and May 2021 by county in the U.S., using K-means multivariate time-series clustering. The impacts of demographic, socioeconomic, and COVID-19-related variables on different profiles were examined. Results showed 5 different clusters of mobility-vaccination profiles were found for the U.S. One cluster represented counties in larger population centers and metropolitan areas (e.g., Los Angeles and New York) that had estimated 25% higher mobility and 75% higher vaccination rates than rural counties in the Mountain and South U.S. Census regions (e.g., counties in Arkansas and Mississippi), where people were mobile despite not getting vaccinated. Higher education and household income were found to impact counties’ mobility-vaccination profiles. Examination of trip purposes for selected counties returned higher trips to retail/recreation and workplaces for rural counties with relatively lower mobility-vaccination profiles. The results can serve as input for regional and local health officials regarding population responses to a pandemic relevant to economic recovery and future disease prevention.

Interpretable Clustering of Multivariate Time Series with Time2Feat

Interpretable Clustering of Multivariate Time Series with Time2Feat

Multivariate Time Series Clustering based on Graph Convolutional Network

Multivariable time series (MTS) clustering is an important topic in time series data mining. The major challenge of MTS clustering is to capture the temporal correlations and the dependencies between multiple variables. In this paper, we propose a novel MTS clustering approach based on graph convolutional network (GCN), which is a powerful feature extractor for graph structure data. We regard each variable in MTS as a node in the graph and construct edges through the correlation between variables. Furthermore, GCN and deep learning back-ropagation technology are used to continuously learn the relationship between multiple variables. Combining the learned variables with the characteristics of the time dimensions, the comprehensive features can be fused to form effective representation for MTS clustering task. We carry out extensive experimental analysis on four open time series data sets and six benchmark algorithms, which shows the superiority of the proposed method.

A self-supervised learning-based approach to clustering multivariate time-series data with missing values (SLAC-Time): An application to TBI phenotyping

Self-supervised learning approaches provide a promising direction for clustering multivariate time-series data. However, real-world time-series data often include missing values, and the existing approaches require imputing missing values before clustering, which may cause extensive computations and noise and result in invalid interpretations. To address these challenges, we present a Self-supervised Learning-based Approach to Clustering multivariate Time-series data with missing values (SLAC-Time). SLAC-Time is a Transformer-based clustering method that uses time-series forecasting as a proxy task for leveraging unlabeled data and learning more robust time-series representations. This method jointly learns the neural network parameters and the cluster assignments of the learned representations. It iteratively clusters the learned representations with the K-means method and then utilizes the subsequent cluster assignments as pseudo-labels to update the model parameters. To evaluate our proposed approach, we applied it to clustering and phenotyping Traumatic Brain Injury (TBI) patients in the Transforming Research and Clinical Knowledge in Traumatic Brain Injury (TRACK-TBI) study. Clinical data associated with TBI patients are often measured over time and represented as time-series variables characterized by missing values and irregular time intervals. Our experiments demonstrate that SLAC-Time outperforms the baseline K-means clustering algorithm in terms of silhouette coefficient, Calinski Harabasz index, Dunn index, and Davies Bouldin index. We identified three TBI phenotypes that are distinct from one another in terms of clinically significant variables as well as clinical outcomes, including the Extended Glasgow Outcome Scale (GOSE) score, Intensive Care Unit (ICU) length of stay, and mortality rate. The experiments show that the TBI phenotypes identified by SLAC-Time can be potentially used for developing targeted clinical trials and therapeutic strategies.

Clustering multivariate time series using energy distance

A novel methodology is proposed for clustering multivariate time series data using energy distance defined in Székely and Rizzo (2013). Specifically, a dissimilarity matrix is formed using the energy distance statistic to measure the separation between the finite‐dimensional distributions for the component time series. Once the pairwise dissimilarity matrix is calculated, a hierarchical clustering method is then applied to obtain the dendrogram. This procedure is completely nonparametric as the dissimilarities between stationary distributions are directly calculated without making any model assumptions. In order to justify this procedure, asymptotic properties of the energy distance estimates are derived for general stationary and ergodic time series. The method is illustrated in a simulation study for various component time series that are either linear or nonlinear. Finally, the methodology is applied to two examples; one involves the GDP of selected countries and the other is the population size of various states in the U.S.A. in the years 1900–1999.

A Multivariate Time Series Anomaly Detection Method Based on Clustered Particle Swarm Optimization

Due to the advent of 5G and the integration of sensors, sensor nodes need to collect data from multiple sources simultaneously.Traditional anomaly detection methods for single attribute time series detect multivariate data, low detection accuracy, and significant node energy consumption. To avoid these problems, this paper provides an improved FCM multivariate time series clustering method based on the sliding window. Based on the CPSO optimal weight, the reconstruction error realizes anomaly detection. Experiments on different datasets demonstrate the effectiveness of the algorithm. Moreover, compared with the three classical methods, the results show that the proposed algorithm has higher detection accuracy and accuracy.

Clustering glacier behavior - A multi-variate time series clustering approach

Clustering glacier behavior - A multi-variate time series clustering approach

Multivariate Time Series Clustering of Groundwater Quality Data to Develop Data-Driven Monitoring Strategies in a Historically Contaminated Urban Area

As groundwater quality monitoring networks have been expanded over the last decades, significant time series are now available. Therefore, a scientific effort is needed to explore innovative techniques for groundwater quality time series exploitation. In this work, time series exploratory analysis and time series cluster analysis are applied to groundwater contamination data with the aim of developing data-driven monitoring strategies. The study area is an urban area characterized by several superimposing historical contamination sources and a complex hydrogeological setting. A multivariate time series cluster analysis was performed on PCE and TCE concentrations data over a 10 years time span. The time series clustering was performed based on the Dynamic Time Warping method. The results of the clustering identified 3 clusters associated with diffuse background contamination and 7 clusters associated with local hotspots, characterized by specific time profiles. Similarly, a univariate time series cluster analysis was applied to Cr(VI) data, identifying 3 background clusters and 7 hotspots, including 4 singletons. The clustering outputs provided the basis for the implementation of data-driven monitoring strategies and early warning systems. For the clusters associated with diffuse background contaminations and those with constant trends, trigger levels were calculated with the 95° percentile, constituting future threshold values for early warnings. For the clusters with pluriannual trends, either oscillatory or monotonous, specific monitoring strategies were proposed based on trends’ directions. Results show that the spatio-temporal overview of the data variability obtained from the time series cluster analysis helped to extract relevant information from the data while neglecting measurements noise and uncertainty, supporting the implementation of a more efficient groundwater quality monitoring.

CSMVC: A Multiview Method for Multivariate Time-Series Clustering.

Multivariate time-series (MTS) clustering is a fundamental technique in data mining with a wide range of real-world applications. To date, though some approaches have been developed, they suffer from various drawbacks, such as high computational cost or loss of information. Most existing approaches are single-view methods without considering the benefits of mutual-support multiple views. Moreover, due to its data structure, MTS data cannot be handled well by most multiview clustering methods. Toward this end, we propose a consistent and specific non-negative matrix factorization-based multiview clustering (CSMVC) method for MTS clustering. The proposed method constructs a multilayer graph to represent the original MTS data and generates multiple views with a subspace technique. The obtained multiview data are processed through a novel non-negative matrix factorization (NMF) method, which can explore the view-consistent and view-specific information simultaneously. Furthermore, an alternating optimization scheme is proposed to solve the corresponding optimization problem. We conduct extensive experiments on 13 benchmark datasets and the results demonstrate the superiority of our proposed method against other state-of-the-art algorithms under a wide range of evaluation metrics.

Multiway clustering with time-varying parameters

This paper proposes a clustering approach for multivariate time series with time-varying parameters in a multiway framework. Although clustering techniques based on time series distribution characteristics have been extensively studied, methods based on time-varying parameters have only recently been explored and are missing for multivariate time series. This paper fills the gap by proposing a multiway approach for distribution-based clustering of multivariate time series. To show the validity of the proposed clustering procedure, we provide both a simulation study and an application to real air quality time series data.

Mithra: Anomaly Detection as an Oracle for Cyberphysical Systems

Testing plays an essential role in ensuring the safety and quality of cyberphysical systems (CPSs). One of the main challenges in automated and software-in-the-loop simulation testing of CPSs is defining effective oracles that can check that a given system conforms to expectations of desired behavior. Manually specifying such oracles can be tedious, complex, and error-prone, and so techniques for automatically learning oracles are attractive. Characteristics of CPSs, such as limited or no access to source code, behavior that is non-deterministic and sensitive to noise, and that the system may respond differently to input based on its context introduce considerable challenges for automated oracle learning. We present Mithra, a novel, unsupervised oracle learning technique for CPSs that operates on existing telemetry data. It uses a three-step multivariate time series clustering to discover the set of unique, correct behaviors for a CPS, which it uses to construct robust oracles. We instantiate our proposed technique for ArduPilot, a popular, open-source autopilot software. On a set of 24 bugs, we show that Mithra effectively identifies buggy executions with few false positives and outperforms AR-SI, a state-of-the-art CPS oracle learning technique. We demonstrate Mithra’s wider applicability by applying it to an autonomous racer built for the Robot Operating System.

Time2Feat

Clustering multivariate time series is a critical task in many real-world applications involving multiple signals and sensors. Existing systems aim to maximize effectiveness, efficiency and scalability, but fail to guarantee the interpretability of the results. This hinders their application in critical real scenarios where human comprehension of algorithmic behavior is required. This paper introduces Time2Feat, an end-to-end machine learning system for multivariate time series (MTS) clustering. The system relies on inter-signal and intra-signal interpretable features extracted from the time series. Then, a dimensionality reduction technique is applied to select a subset of features that retain most of the information, thus enhancing the interpretability of the results. In addition, domain experts can semi-supervise the process, by providing a small amount of MTS with a target cluster. This process further improves both accuracy and interpretability, narrowing down the number of features used by the clustering process. We demonstrate the effectiveness, interpretability, efficiency, and robustness of Time2Feat through experiments on eighteen benchmarking time series datasets, comparing them with state-of-the-art MTS clustering methods.

Spatial Weighted Robust Clustering of Multivariate Time Series Based on Quantile Dependence With an Application to Mobility During COVID-19 Pandemic

In this article, a fuzzy clustering model for multivariate time series based on the quantile cross-spectral density and principal component analysis is extended by including: 1) a weighting system which assigns a weight to each principal component in accordance with its importance concerning the underlying clustering structure and 2) a penalization term allowing to take into account the spatial information. The iterative solutions of the new model, which employs the exponential distance in order to gain robustness against outlying series, are derived. A simulation study shows that the weighting system substantially enhances the effectiveness of the former approach. The behavior of the extended model in terms of the spatial penalization term is also analyzed. An application involving multivariate time series of mobility indicators concerning COVID-19 pandemic highlights the usefulness of the proposed technique.