Temporal gap statistic: A new internal index to validate time series clustering

Abstract

Abstract Unsupervised Machine Learning techniques have been developed to find out structures in datasets without considering any prior information. In such a context, the main challenge is to confirm whether the obtained structure indeed contains relevant data patterns. Aiming at solving this issue, there are several validation indexes proposed under different categories (e.g. internal, external, and relative) that allow to, for example, compare clustering algorithms or define the best parameter configurations. However, most of those indices are applied to data characterized for being collected in an independent and identically distributed manner. Thus, after performing a Systematic Literature Review, we noticed there are few researches investigating validation indexes specifically designed to deal with time-dependent data. The absence of researches for such context has motivated this work that was devoted to developing a new internal index based on Gap Statistic. Our index supports the estimation of the optimal number of clusters in a dataset only composed of time series. To reach this goal, we performed three important modifications in Gap Statistic: i) the use of a measure to calculate the distance between time series; ii) the adoption of a clustering method based on medoid; and iii) the modeling of time series in phase space using Dynamical System tools. Our results emphasize the importance of the proposed index, by accurately clustering sets of chaotic time series.