Clustering Of Time Series Data Research Articles

Clustering plasma concentration-time curves: applications ofunsupervised learning in pharmacogenomics

ABSTRACT Pharmaceutical researchers are continually searching for techniques to improve both drug development processes and patient outcomes. An area of recent interest is the potential for machine learning (ML) applications within pharmacology. One such application not yet given close study is the unsupervised clustering of plasma concentration-time curves, hereafter, pharmacokinetic (PK) curves. In this paper, we present our findings on how to cluster PK curves by their similarity. Specifically, we find clustering to be effective at identifying similar-shaped PK curves and informative for understanding patterns within each cluster of PK curves. Because PK curves are time series data objects, our approach utilizes the extensive body of research related to the clustering of time series data as a starting point. As such, we examine many dissimilarity measures between time series data objects to find those most suitable for PK curves. We identify Euclidean distance as generally most appropriate for clustering PK curves, and we further show that dynamic time warping, Fréchet, and structure-based measures of dissimilarity like correlation may produce unexpected results. As an illustration, we apply these methods in a case study with 250 PK curves used in a previous pharmacogenomic study. Our case study finds that an unsupervised ML clustering with Euclidean distance, without any subject genetic information, is able to independently validate the same conclusions as the reference pharmacogenomic results. To our knowledge, this is the first such demonstration. Further, the case study demonstrates how the clustering of PK curves may generate insights that could be difficult to perceive solely with population level summary statistics of PK metrics.

1DCAE-TSSAMC: Two-Stage Multi-Dimensional Spatial Features Based Multi-View Deep Clustering for Time Series Data

At present, as a research hotspot for time series data (TSD), the deep clustering analysis of TSD has huge research value and practical significance. However, there still exist the following three problems: (1) For deep clustering based on joint optimization, the inevitably mutual interference existing between deep feature representation learning progress and clustering progress leads to difficult model training especially in the initial stage, the possible feature space distortion, inaccurate and weak feature representation; (2) Existing deep clustering methods are difficult to intuitively define the similarity of time series and rely heavily on complex feature extraction networks and clustering algorithms. (3) Multidimensional time series have the characteristics of high dimensions, complex relationships between dimensions, and variable data forms, thus generating a huge feature space. It is difficult for existing methods to select discriminative features, resulting in generally low accuracy of methods. Accordingly, to address the above three problems, we proposed a novel general two-stage multi-dimensional spatial features based multi-view deep clustering method 1DCAE-TSSAMC (One-dimensional deep convolutional auto-encoder based two-stage stepwise amplification multi-clustering). We conducted verification and analysis based on real-world important multi-scenario, and compared with many other benchmarks ranging from the most classic approaches such as K-means and Hierarchical to the state-of-the-art approaches based on deep learning such as Deep Temporal Clustering (DTC) and Temporal Clustering Network (TCN). Experimental results show that the new method outperforms the other benchmarks, and provides more accurate, richer, and more reliable analysis results, more importantly, with significant improvement in accuracy and spatial linear separability.

Time-series data clustering with load-shape preservation for identifying residential energy consumption behaviors

Categorizing residential energy demand patterns is a principal task for demand-side management (DSM) and energy-saving strategies. While deep learning (DL)-based clustering offers a promising alternative to conventional machine learning (ML), DL’s advantages and disadvantages over ML still remain unclear in identifying energy demand patterns. Moreover, prevalent DL-based clustering can suffer from catastrophic feature distortion when capturing load-shape information from energy-load data, leading to erroneous pattern identification. To address these issues, we propose integrating a load-shape preservation mechanism into representative DL-based clustering and investigate its effectiveness in categorizing energy demand patterns, compared to existing ML and DL. We experiment and compare the three clustering approaches using one-year residential energy-load data. Results show that the proposed DL, equipped with load-shape preservation, outperformed ML quantitatively and closely aligns with the baseline DL’s performance. This is particularly significant considering that the baseline DL prioritizes quantitative enhancements, sometimes compromising load-shape precision. Furthermore, the proposed DL discovered more diverse energy demand patterns than the baseline ML and DL, while producing more human-agreeable results. This finding underscores the pivotal role of load-shape preservation in enhancing data clustering and demand pattern recognition in the real-world. These benefits will facilitate personalized DSM interventions and foster residents’ energy-saving behaviors.

Evolutionary Multi-Tasking Optimization for High-Efficiency Time Series Data Clustering

Evolutionary Multi-Tasking Optimization for High-Efficiency Time Series Data Clustering

Self-Supervised Framework Based on Subject-Wise Clustering for Human Subject Time Series Data

With the widespread adoption of IoT, wearable devices, and sensors, time series data from human subjects are significantly increasing in the healthcare domain. Due to the laborious nature of manual annotation in time series data and the requirement for human experts, self-supervised learning methods are attempted to alleviate the limited label situations. While existing self-supervised methods have been successful to achieve comparable performance to the fully supervised methods, there are still some limitations that need to be addressed, considering the nature of time series data from human subjects: In real-world clinical settings, data labels (e.g., sleep stages) are usually annotated by subject-level, and there is a substantial variation in patterns between subjects. Thus, a model should be designed to deal with not only the label scarcity but also subject-wise nature of data to ensure high performance in real-world scenarios. To mitigate these issues, we propose a novel self-supervised learning framework for human subject time series data: Subject-Aware Time Series Clustering (SA-TSC). In the unsupervised representation learning phase, SA-TSC adopts a subject-wise learning strategy rather than instance-wise learning which randomly samples data instances from different subjects within the batch during training. Specifically, we generate subject-graphs with our graph construction method based on Gumbel-Softmax and perform graph spectral clustering on each subject-graph. In addition, we utilize graph neural networks to capture dependencies between channels and design our own graph learning module motivated from self-supervised loss. Experimental results show the outstanding performance of our SA-TSC with the limited & subject-wise label setting, leading to its high applicability to the healthcare industry. The code is available at: https://github.com/DILAB-HYU/SA-TSC

The ensemble distance on model-based clustering for regions clustering based on rainfall: The case of rainfall in West Java Indonesia

Time series data clusters are being researched thoroughly. The distance metric drives the development of the clustering time series. The ARIMA model is one of the models that can be employed in model-based clustering, although differing model selection criteria can lead to uncertainty in the model. In this investigation, we created a technique for ensemble distance-based time series data clustering. To express the distance between two series, five distances based on the five model selection criteria are utilized. The average of the five distances reflects the distance of two time series data. According to the simulation results, the ensemble distance method could boost clustering accuracy by more than 11%. Based on the pattern of rainfall levels, we applied our methods to find clusters of locations in the Province of West Java (Indonesia). The findings indicate that the rainfall pattern in the same cluster is similar. The cluster model is effective and feasible for representing individual models in a cluster.

Time-constrained Gaussian mixture model for clustering multi-modal chemical process data

Two novel unsupervised learning algorithms were developed for improved clustering of multi-modal time series data that are not separable in feature space, which are common characteristics of chemical process data. The algorithms are extensions of the conventional Gaussian Mixture Model (GMM) and K-means clustering. Both algorithms were adapted to account for the time-dependent nature of chemical process data and thus are termed time-constrained GMM (TCGMM) and time-constrained K-means (TCK-means). The algorithms are evaluated using autoregressive time series data with small step changes in the means and variances; a problem that confounds conventional clustering algorithms. In Case Study 1, step changes in the means and variances are implemented at specific time intervals to create two modes. TCGMM outperforms the other algorithms by obtaining a minimum of 85% accuracy in correctly identifying the modes. The TCGMM algorithm is also tested in a second case study where combinations of mean- and variance-shifts are randomly instantiated based on a conditional probability table (CPT). TCGMM outperforms conventional GMM by an average accuracy of 65.4% versus 46.6% and learns the CPT with an average difference in the main diagonal entries (probabilities of remaining in the same mode) of 1.89% and an average difference in the off-diagonal entries (mode transition probabilities) of 0.664%. Keywords: clustering, TCGMM, TCK-means

Temporal Multi-features Representation Learning-Based Clustering for Time-Series Data

Temporal Multi-features Representation Learning-Based Clustering for Time-Series Data

Bayesian Semiparametric Local Clustering of Multiple Time Series Data

In multiple time series data, clustering the component profiles can identify meaningful latent groups while also detecting interesting change points in their trajectories. Conventional time series clustering methods, however, suffer the drawback of requiring the co-clustered units to have the same cluster membership throughout the entire time domain. In contrast to these “global” clustering methods, we develop a Bayesian “local” clustering method that allows the functions to flexibly change their cluster memberships over time. We design a Markov chain Monte Carlo algorithm to implement our method. We illustrate the method in several real-world datasets, where time-varying cluster memberships provide meaningful inferences about the underlying processes. These include a public health dataset to showcase the more detailed inference our method can provide over global clustering alternatives, and a temperature dataset to demonstrate our method’s utility as a flexible change point detection method. Supplemental materials for this article, including R codes implementing the method, are available online.

Abstract 11546: Functional Co-Clustering of Physical Activity Patterns Among Sexual and Gender Minority Adults

Introduction: Physical activity (PA) is a important modifiable risk factor for cardiovascular disease (CVD) and CVD mortality. There is growing evidence that sexual and gender minority (SGM; lesbian/gay, bisexual, transgender) adults are at higher risk of insufficient PA and CVD. Accelerometers allow for collection of granular PA data that can identify individual- and temporal heterogeneity in daily patterns. To date, no studies have investigated accelerometry-based PA patterns among SGM adults. Goal: To characterize daily PA trajectories among SGM adults and identify distinct phenotypes using a time series-based clustering technique. Methods: We recruited an online sample of healthy SGM adults in the United States who completed continuous wrist-worn accelerometry for 30 days to collect step counts. For clustering, we used a functional latent block models (FLBMs), where each 24-hour period was a Fourier-smoothed data curve. FLBMs allow hierarchical time-series data clustering by simultaneously clustering person- and day-level data. The optimal number of clusters was selected using the integrated completed likelihood (ICL) criterion. Results: Forty-two SGM adults with a mean age of 27.0 years (+/- 7.7) provided 1207 person-level days of accelerometry (range=6-31 days/person). A final model with 4 blocks (2 x 2 clusters) provided the best fit (ICL= -70086.4). Two person-level clusters were identified, which were characterized by differences in the amount and distribution of steps throughout the day were identified. Cluster 1 (n=26) had higher overall steps counts with significant morning and evening increases in step counts. Cluster 2 (n=16) had fewer steps that were within a narrower time period. We further identified 2 day-level clusters. Cluster 2 (n=14) had a wider temporal distribution of step counts and a higher variance on weekend days vs. weekdays relative to Cluster 1 (n=17). Conclusions: This is the first study to elucidate daily PA trajectories in SGM people. Using FLBM, we accounted for individual heterogeneity and relations among days. Findings can help identify individuals at increased risk of physical inactivity and subsequent negative health outcomes, providing important knowledge to inform behavioral interventions.

A hybrid machine learning approach for the load prediction in the sustainable transition of district heating networks

Current district heating networks are undergoing a sustainable transition towards the 4th and 5th generation of district heating networks, characterized by the integration of different types of renewable energy sources (RES) and low operational temperatures, i.e., 55 °C or lower. Due to the lower temperature difference between supply and return, it is necessary to develop novel methods to understand the loads accurately and provide operation scenarios to anticipate demand peaks and increase flexibility in the energy network, both for long- and short-term horizons. In this study, a hybrid machine-learning (ML) method is developed, combining a clustering pre-processing step with a multi-input artificial neural network (ANN) model to predict heat loads in buildings cluster-wise. Specifically, the impact of time-series data clustering, as a pre-processing step, on the performance of ML models was investigated. It was found that data clustering contributes effectively to the reduction of data training costs by limiting the training processes to representative clusters only instead of all datasets. Additionally, low-quality data, including outliers and large measurement gaps, are excluded from the training to enhance the overall prediction performance of the models.

Deep Temporal Contrastive Clustering

Recently the deep learning has shown its advantage in representation learning and clustering for time series data. Despite the considerable progress, the existing deep time series clustering approaches mostly seek to train the deep neural network by some instance reconstruction based or cluster distribution based objective, which, however, lack the ability to exploit the sample-wise (or augmentation-wise) contrastive information or even the higher-level (e.g., cluster-level) contrastiveness for learning discriminative and clustering-friendly representations. In light of this, this paper presents a deep temporal contrastive clustering (DTCC) approach, which for the first time, to our knowledge, incorporates the contrastive learning paradigm into the deep time series clustering research. Specifically, with two parallel views generated from the original time series and their augmentations, we utilize two identical auto-encoders to learn the corresponding representations, and in the meantime perform the cluster distribution learning by incorporating a k-means objective. Further, two levels of contrastive learning are simultaneously enforced to capture the instance-level and cluster-level contrastive information, respectively. With the reconstruction loss of the auto-encoder, the cluster distribution loss, and the two levels of contrastive losses jointly optimized, the network architecture is trained in a self-supervised manner and the clustering result can thereby be obtained. Experiments on a variety of time series datasets demonstrate the superiority of our DTCC approach over the state-of-the-art. The code is available at https://github.com/07zy/DTCC .

Identifying responders to elamipretide in Barth syndrome: Hierarchical clustering for time series data

BackgroundBarth syndrome (BTHS) is a rare genetic disease that is characterized by cardiomyopathy, skeletal myopathy, neutropenia, and growth abnormalities and often leads to death in childhood. Recently, elamipretide has been tested as a potential first disease-modifying drug. This study aimed to identify patients with BTHS who may respond to elamipretide, based on continuous physiological measurements acquired through wearable devices.ResultsData from a randomized, double-blind, placebo-controlled crossover trial of 12 patients with BTHS were used, including physiological time series data measured using a wearable device (heart rate, respiratory rate, activity, and posture) and functional scores. The latter included the 6-minute walk test (6MWT), Patient-Reported Outcomes Measurement Information System (PROMIS) fatigue score, SWAY Balance Mobile Application score (SWAY balance score), BTHS Symptom Assessment (BTHS-SA) Total Fatigue score, muscle strength by handheld dynamometry, 5 times sit-and-stand test (5XSST), and monolysocardiolipin to cardiolipin ratio (MLCL:CL). Groups were created through median split of the functional scores into “highest score” and “lowest score”, and “best response to elamipretide” and “worst response to elamipretide”. Agglomerative hierarchical clustering (AHC) models were implemented to assess whether physiological data could classify patients according to functional status and distinguish non-responders from responders to elamipretide. AHC models clustered patients according to their functional status with accuracies of 60–93%, with the greatest accuracies for 6MWT (93%), PROMIS (87%), and SWAY balance score (80%). Another set of AHC models clustered patients with respect to their response to treatment with elamipretide with perfect accuracy (all 100%).ConclusionsIn this proof-of-concept study, we demonstrated that continuously acquired physiological measurements from wearable devices can be used to predict functional status and response to treatment among patients with BTHS.

Fractal dimension based geographical clustering of COVID-19 time series data

Understanding the local dynamics of COVID-19 transmission calls for an approach that characterizes the incidence curve in a small geographical unit. Given that incidence curves exhibit considerable day-to-day variation, the fractal structure of the time series dynamics is investigated for the Flanders and Brussels Regions of Belgium. For each statistical sector, the smallest administrative geographical entity in Belgium, fractal dimensions of COVID-19 incidence rates, based on rolling time spans of 7, 14, and 21 days were estimated using four different estimators: box-count, Hall-Wood, variogram, and madogram. We found varying patterns of fractal dimensions across time and location. The fractal dimension is further summarized by its mean, variance, and autocorrelation over time. These summary statistics are then used to cluster regions with different incidence rate patterns using k-means clustering. Fractal dimension analysis of COVID-19 incidence thus offers important insight into the past, current, and arguably future evolution of an infectious disease outbreak.

Spatiotemporal Sequence-to-Sequence Clustering for Electric Load Forecasting

Massive electrical load exhibits many patterns making it difficult for forecast algorithms to generalise well. Most learning algorithms produce a better forecast for dominant patterns in the case of weekday consumption and otherwise for less dominant patterns in weekend and holiday consumption. In view of this, there is the need to cluster the load patterns, so learning algorithms can focus on the patterns independently to produce forecasts with better accuracy for all cases. However, clustering time-series data breaks the time-series dependency, making model training difficult. This paper presents a novel sequence-to-sequence cluster framework to reform time-series dependency after clustering; this enables independent clusters to be modelled using Convolutional Neural Network-Gated Recurrent Unit, which learns spatiotemporal features for future forecasts. A real-world dataset by the Korea Power Exchange composed of nationwide consumption is used for case studies and experiments. Experimental results verify that the proposed study effectively improves the accuracy of electric load forecasting by about 50%, with a WAPE of 0.67%. The proposed method also speeds up the training process of the forecast algorithm by about 35%, given that only a subset of the dataset is trained due to clustering. Korea Water Resources Corporation has implemented the proposed method for load forecasting and system marginal price estimation.

Optimization-Assisting Dual-Step Clustering of Time Series Data

This paper aims to propose a new time series data clustering with the following steps: (1) data reduction and (2) clustering. The main objective of the time series data clustering is to minimize the dataset size via a prototype defined for same time series data in every group that significantly reduced the complexities. Initially, the time series dataset in the data reduction step is subjected to preprocessing process. Further, in the proposed probability based distance measure evaluation, the time series data is grouped into subclusters. In the clustering step, the proposed shape based similarity measure is performed. Moreover, the clustering process is carried out by optimized k-mean clustering in which the center point is optimally tuned by a new customized whale optimization algorithm (CWOA). At last, the performance of the adopted model is computed to other traditional models with respect to various measures such as sensitivity, accuracy, FPR, conentropy, precision, FNR, specificity, MCC, entropy, F-measure, and Rand index, respectively.

A Fast Weighted Fuzzy C-Medoids Clustering for Time Series Data Based on P-Splines.

The rapid growth of digital information has produced massive amounts of time series data on rich features and most time series data are noisy and contain some outlier samples, which leads to a decline in the clustering effect. To efficiently discover the hidden statistical information about the data, a fast weighted fuzzy C-medoids clustering algorithm based on P-splines (PS-WFCMdd) is proposed for time series datasets in this study. Specifically, the P-spline method is used to fit the functional data related to the original time series data, and the obtained smooth-fitting data is used as the input of the clustering algorithm to enhance the ability to process the data set during the clustering process. Then, we define a new weighted method to further avoid the influence of outlier sample points in the weighted fuzzy C-medoids clustering process, to improve the robustness of our algorithm. We propose using the third version of mueen’s algorithm for similarity search (MASS 3) to measure the similarity between time series quickly and accurately, to further improve the clustering efficiency. Our new algorithm is compared with several other time series clustering algorithms, and the performance of the algorithm is evaluated experimentally on different types of time series examples. The experimental results show that our new method can speed up data processing and the comprehensive performance of each clustering evaluation index are relatively good.

Hydrological Time Series Clustering: A Case Study of Telemetry Stations in Thailand

Water level data from telemetry stations typically demonstrate diverse behaviors over time. Specific characteristics can be observed among distinct station groups that are different from others. Clustering time series data into a specified number of groups based on their similarity is an initial step for further analysis in water management analytics. Our main goal in this work is to develop a clustering framework based on a combination of feature representations, feature reduction techniques, as well as clustering algorithms. Thorough experiments on multiple combinations of these methods were conducted and compared. Based on collected water level data in Thailand, UMAP reduced representations of engineered features using HAC clustering with euclidean distance outperformed other methods. Its performance reached 0.8 Fowlkes-Mallows score. Out of 81 stations, only nine unclear cases were incorrectly clustered. Distinct behaviors with abrupt and frequent fluctuations could be perfectly identified.

P1317: TIME SERIES CLUSTERING OF T CELL SUBSETS DISSECTS HETEROGENEITY IN IMMUNE RECONSTITUTION AND SURVIVAL AMONG RECIPIENTS OF MUD-HCT WITH ATG OR PTCY

Background: Proliferating alloreactive T cells have a central role in the induction of acute GVHD (aGVHD) making them a promising target in preventing alloreactivity. T cell depleting regimens such as anti-T-lymphocyte globulin (ATG) or post-transplant cyclophosphamide (PTCy) effectively eliminate these cells and reduce aGVHD after hematopoietic cell transplantation (HCT) with marginal differences in clinical outcome. Comparative immune reconstitution analyses could contribute to answer the question, which agent would be most appropriate for an individual patient. However, not only effects relating to the different T cell depleting regimens, i.e. ATG or PTCy may be relevant but the general heterogeneity of T cell reconstitution. Aims: Here, we aimed at dissecting this heterogeneity with an approach called time-series clustering to better understand the impact of both regimens in individual patients and to consequently identify distinct patient subsets which benefit the most from each protocol. Methods: We retrospectively compared immune reconstitution data of 339 recipients of matched-unrelated donor (MUD) HCT with either ATG (n=304) or PTCy (n=35) as T cell depletion via conventional analysis on the cohort level and for its individual heterogeneity via time-series clustering. This analysis leveraged the approach of dynamic time warping (DTW) to determine the distance measure later used for the partitional clustering of individual patient time-series data. The performance of this methods was evaluated by the silhouette coefficient (Sil) which indicates a separation of clusters with values between 0 (overlapping clusters) and +1 (best separation); A 10-fold resampling was used to assess model robustness. Cluster information were used for subsequent analysis of clinical outcomes. All analysis were performed using R packages R stats, dtwclust, survival, survminer and cmprsk. Results: Comparative analysis of cellular reconstitution revealed distinct prominent T cell population after each protocol. While patients receiving PTCy as GVHD prophylaxis presented with higher levels of regulatory T cells, ATG patients had higher levels of γδ T- or NK-T cells. Time-series clustering of T cell subpopulations that associate with GVHD successfully dissected each population’s heterogeneity. For ATG patients the clustering revealed two distinct clusters with its optimal model configuration showing a good and robust silhouette coefficient (7_1: Sil=0.524) and a balanced patient distribution (Fig. 1A). In this model, clustering of ATG patients was driven by αβ- and activated T cells as those revealed higher absolute counts and a greater difference in shape over time as compared to regulatory- and γδ T cells. Patients in ATG cluster 1, showing higher absolute counts of αβ- and activated T cells compared to cluster 2 were associated to a significantly higher 1-year OS (98% vs. 79%, p=0.0023) due to lower NRM (p=0.032) and relapse (p=0.01). The clustering of PTCy patients distinguished two clusters with lower silhouette coefficients had a strong impact of αβ- and activated T cells (Fig.1B). Comparing the overall survival probability of both ATG clusters with the PTCy cohort revealed a significant difference driven by the ATG clusters (Fig 1C). Image:Summary/Conclusion: Beyond a differential impact of ATG and PTCy on immune reconstitution our analysis identified phenotypes that reproducibly associated with impaired clinical outcomes within the same T cell depletion platform. This provides guidance for individually choosing the most appropriate agent in the MUD setting.

The use of infrared spectroscopy and chemometrics to investigate deterioration in vegetable tanned leather: potential applications in heritage science

Vegetable tanned leather presents a unique challenge to conservators and curators of heritage collections, as little is known about how its physical and chemical properties change upon deterioration. Developing a better understanding of deterioration processes would be incredibly valuable in informing the conservation, storage, and restoration of leather objects. Fourier Transform infrared spectroscopy (FTIR) used with attenuated total reflectance (ATR) is increasingly applied in the heritage sector due to its relative ease of application and potential to be non-destructive. However, whilst FTIR has been applied successfully to the understanding of deterioration in other protein-based materials such as parchment, its application to the analysis of leather has been limited, largely due to the highly complex spectra obtained. Here, we have developed multivariate statistical methods for the analysis of FTIR data obtained from a time-series of leather samples artificially degraded at different pH values. Principal component analysis (PCA), Partial Least Squares Discriminant Analysis (PLS-DA) and k-means clustering, when used together, are demonstrated as powerful tools in identifying early subtle differences in the FTIR spectra as leather degrades, identifying differences occurring over time and between different environmental conditions. We show that k-means clustering of time series data was able to highlight some areas of the spectrum that might be indicative of degradation, which more common chemometric techniques could not. The methods we describe here have the potential to widen the application of FTIR as a fast, non-destructive and reliable tool for assessing the condition of archaeological and historical leather objects, ultimately leading to better informed conservation, storage and restoration of these objects.