True Number Of Clusters Research Articles

Parameter-Agnostic Deep Graph Clustering

Deep graph clustering, efficiently dividing nodes into multiple disjoint clusters in an unsupervised manner, has become a crucial tool for analyzing ubiquitous graph data. Existing methods have acquired impressive clustering effects by optimizing the clustering network under the parametric condition—predefining the true number of clusters ( K tr ). However, K tr is inaccessible in pure unsupervised scenarios, in which existing methods are incapable of inferring the number of clusters ( K ), causing limited feasibility. This article proposes the first Parameter-Agnostic Deep Graph Clustering method (PADGC), which consists of two core modules: K -guidence clustering and topological-hierarchical inference, to infer K efficiently and gain impressive clustering predictions. Specifically, K -guidence clustering is employed to optimize the cluster assignments and discriminative embeddings in a mutual promotion manner under the latest updated K , even though K may deviate from K tr . In turn, such optimized cluster assignments are utilized to explore more accurate K in the topological-hierarchical inference, which can split the dispersive clusters and merge the coupled ones. In this way, these two modules are complementarily optimized until generating the final convergent K and discriminative cluster assignments. Extensive experiments on several benchmarks, including graphs and images, can demonstrate the superiority of our method. The mean values of our inferred K , in 11 out of 12 datasets, deviates from K tr by less than 1. Our method can also achieve competitive clustering effects with existing parametric deep graph clustering.

Evaluating multivariate time-series clustering using simulated ecological momentary assessment data

During an Ecological Momentary Assessment (EMA) study, through repeated digital questionnaires, we have the opportunity to collect multiple multivariate time-series (MTS) data for all participants. Although, it is common that individual data is analyzed per participant, the richness of such dataset poses the question of whether meaningful groups of individuals could be uncovered to better understand the underlying processes on an individual and a group level. Such grouping could be obtained by clustering. Therefore, this paper examines the performance of various clustering approaches for grouping individuals based on the similarity of their raw time-series data patterns. Clustering is an unsupervised task, where the true underlying groups are not usually available, making the result difficult to evaluate. Therefore, in the current paper, simulated irregular time-series data, resembling EMA, are used to validate the performance of several methods under different clustering-related choices, such as the distance metric. Data are generated with a varying number of clusters, total number of individuals and time-points as well as number of variables and proportions of noisy variables, while their time-series represent well-shaped patterns, typically observed in emotional behavior. After applying clustering to all simulated datasets, clustering performance was first assessed by comparing the true and predicted labels, while the impact of the different datasets’ parameters was also examined. Because ground truth labels are not always available, or do not even exist, in real-world scenarios, clustering evaluation through distance-based and distance-free measures was further investigated. Overall, all clustering methods (e.g. k-means, Hierarchical clustering, Fuzzy k-medoids) proved reliable in different configurations, revealing the true number of clusters. Moreover, kernel-based methods appeared more efficient when highly noisy variables are involved, becoming more promising for real-world data. As a second part, an illustration of two specific simulated scenarios (datasets) is provided, showing, in more detail, all different analysis steps before drawing a conclusion about the choice of the optimal number of clusters.

Weighted adaptively ensemble clustering method based on fuzzy Co-association matrix

Although many existing co-association (CA) matrix-based ensemble clustering methods have witnessed their superiority in unsupervised learning, but these methods still suffer the following problems: (1) the CA matrix may only focus on strengthening the co-association relations of the pairwise samples in the same cluster and ignoring other information (e.g., inter-cluster association relationship). (2) the CA matrix should be directly used to obtain the ensemble clustering result without using other clustering methods (e.g., spectral clustering). (3). the weights of each base clustering result should be assigned adaptively to reflect the importance of each base clustering result. In order to circumvent the above problems, we propose a novel weighted adaptively ensemble clustering method based on fuzzy co-association matrix (called WEC-FCA) in our study. Concretely, the proposed method WEC-FCA first proposes a novel CA matrix called fuzzy co-association (FCA) matrix to reflect both the co-association relations and inter-cluster association relationships of the pairwise samples in dataset. Then by introducing the rank constraint into the FCA matrix, a novel optimalization framework is invented to obtain the optimal ensemble FCA matrix having the same total number of components as the true number of clusters, so as to be directly used to obtain the ensemble clustering result. Besides, the weights of each base clustering result can be assigned adaptively during the optimization process by using the Shannon entropy. Experimental results on both synthetic and real datasets confirm that the proposed method WEC-FCA keeps at least comparable clustering performance to the state-of-the-art ensemble clustering methods.

Cluster ensemble selection and consensus clustering: A multi-objective optimization approach

Cluster ensembles have emerged as a powerful tool to obtain clusters of data points by combining a library of clustering solutions into a consensus solution. In this paper, we address the cluster ensemble selection problem and design a multi-objective optimization-based solution framework to produce consensus solutions. Given a library of clustering solutions, we first design a preprocessing procedure that measures the agreement of each clustering solution with the other solutions and eliminates the ones that may mislead the process. We then develop a multi-objective optimization algorithm that selects representative clustering solutions from the preprocessed library with respect to size, coverage, and diversity criteria and combines them into a single consensus solution, for which the true number of clusters is assumed to be unknown. We conduct experiments on different benchmark data sets. The results show that our approach yields more accurate consensus solutions compared to full-ensemble and the existing approaches for most data sets. We also present an application on the customer segmentation problem, where our approach is used to segment customers and to find a consensus solution for each segment, simultaneously.

On k-means iterations and Gaussian clusters

Nowadays, k-means remains arguably the most popular clustering algorithm (Jain, 2010; Vouros et al., 2021). Two of its main properties are simplicity and speed in practice. Here, our main claim is that the average number of iterations k-means takes to converge (τ¯) is in fact very informative. We find this to be particularly interesting because τ¯ is always known when applying k-means but has never been, to our knowledge, used in the data analysis process. By experimenting with Gaussian clusters, we show that τ¯ is related to the structure of a data set under study. Data sets containing Gaussian clusters have a much lower τ¯ than those containing uniformly random data. In fact, we go considerably further and demonstrate a pattern of inverse correlation between τ¯ and the clustering quality. We illustrate the importance of our findings through two practical applications. First, we describe the cases in which τ¯ can be effectively used to identify irrelevant features present in a given data set or be used to improve the results of existing feature selection algorithms. Second, we show that there is a strong relationship between τ¯ and the number of clusters in a data set, and that this relationship can be used to find the true number of clusters it contains.

Prediction Strength for Clustering Activity Patterns Using Accelerometer Data

Background: Clustering, a class of unsupervised machine learning methods, has been applied to physical activity data recorded by accelerometers to discover unique patterns of physical activity and health outcomes. The prediction strength metric provides a criterion to determine the optimal number of clusters for clustering methods. The aim of this study is to provide specific guidance for applying prediction strength to time series accelerometer data. Methods: For this purpose, we designed an extensive simulation study. We created a synthetic data set of accelerometer data using data from a childhood obesity management trial. We evaluated the role of a prespecified threshold of the prediction strength metric as a key input parameter. We compared the recommended threshold (between 0.8 and 0.9) with an approach we developed (Local Maxima). Results: The choice of threshold had a large impact on performance. When the noise level increased (greater overlap between true clusters), lower thresholds outperformed the recommended threshold, which tended to underestimate the true number of clusters. In addition, we found that sorting the data by magnitude of intensity in windows within the time series of interest prior to clustering alleviated sensitivity to threshold choice. Furthermore, for accelerometer data, we recommend that the Local Maxima approach be utilized together with a graphical evaluation of the prediction strength metric function over values of k. Finally, we strongly suggest sorting of the data prior to clustering if sorting retains meaning for the research question at hand. Conclusion: Our recommendations can help future researchers discover more robust patterns from accelerometer data.

A novel cluster validity index based on augmented non-shared nearest neighbors

The true cluster number of the dataset in practical applications is rarely known in advance. Therefore, it is necessary to use a cluster validity index to evaluate the clustering results and determine the optimal cluster number. However, the performance of existing cluster validity indices is vulnerable to various factors such as cluster shape and density. To solve the above issues, this paper proposes a new cluster validity index based on augmented non-shared nearest neighbors (ANCV). The ANCV index is based on the following principles: (1) Within-cluster compactness can be measured by the distance between the pairs of data points with fewer shared nearest neighbors. (2) The distances between the pairs of data points at the intersection of clusters can be used to estimate the between-cluster separation. On this basis, the above point pairs are further extended to their augmented non-shared nearest neighbors, thereby forming small clusters. Then, the average distance within and between these clusters is calculated respectively to estimate the within-cluster compactness and between-cluster separation. Finally, the optimal number of clusters is determined by the difference between the between-cluster separation and the within-cluster compactness. Experimental results on both 12 two-dimensional synthetic datasets and 10 real datasets from UCI have shown that the ANCV index performs the best among all compared indices.

PDBI: A partitioning Davies-Bouldin index for clustering evaluation

Clustering validation and identifying the optimal number of clusters are crucial in expert and intelligent systems. However, the commonly used cluster validity indices (CVI) are not relevant enough to measure data structures. They do not embed the necessary mechanisms to be as effective as that of the clustering algorithm used to give the clustering results. This paper proposes a novel CVI called PDBI (Partitioning Davies-Bouldin Index) initially inspired from the native idea of the Davies-Bouldin Index (DBI). PDBI is based on a strategy that consists in dividing each cluster into sub-clusters that redefine the concepts of internal homogeneity and cluster separation via the integration of sophisticated mechanisms. This strategy makes it possible to process a relevant CVI even in the case of complex data structures and in presence of clusters with noisy patterns. PDBI is deterministic, runs independently of a given clustering algorithm and generates a normalized score between 0 and 1. Numerous tests were carried out using 2-dimensional benchmark data sets and data generated in higher dimensions with consistent ground truths. The experimental comparisons with the state-of-the-art validity indices demonstrate the efficiency of the proposal in discovering the true number of clusters and dealing with various sorts of data sets. The PDBI demonstration as well as illustrations can be found on the author’s website11A demonstration is available at: http://r-riad.net/

Cluster Analysis on Locally Asymptotically Self-Similar Processes with Known Number of Clusters

We conduct cluster analysis of a class of locally asymptotically self-similar stochastic processes with finite covariance structures, which includes Brownian motion, fractional Brownian motion, and multifractional Brownian motion as paradigmatic examples. Given the true number of clusters, a new covariance-based dissimilarity measure is introduced, based on which we obtain approximately asymptotically consistent algorithms for clustering locally asymptotically self-similar stochastic processes. In the simulation study, clustering data sampled from fractional and multifractional Brownian motions with distinct Hurst parameters illustrates the approximated asymptotic consistency of the proposed algorithms. Clustering global financial markets’ equity indexes returns and sovereign CDS spreads provides a successful real world application. Implementations in MATLAB of the proposed algorithms and the simulation study are publicly shared in GitHub.

Determine the number of clusters by data augmentation

Determining the number of clusters is crucial for the successful application of clustering. In this paper, we propose a new order-determination method called the data augmentation estimator (DAE), for the general model-based clustering. The estimator is based on a novel idea that augments data with an independently generated small cluster, which enables us to justify how the instability of clustering changes with the number of clusters assumed in clustering. The pattern of instability provides an alternative characterization of the true number of clusters to the commonly used goodness-of-fit measure. By combining the two sources of information appropriately, the proposed estimator reaches asymptotic consistency under general conditions and is easily implementable. It is also more efficient than the conventional BIC-type approaches that use the goodness-of-fit measure only. These properties are illustrated by the simulation studies and real data examples at the end.

A new validity clustering index-based on finding new centroid positions using the mean of clustered data to determine the optimum number of clusters

Clustering, an unsupervised pattern classification method, plays an important role in identifying input dataset structures. It partitions input datasets into clusters or groups where either the optimum number of clusters is known in prior or automatically determined. In the case of automatic clustering, the performance is evaluated using a cluster validity index (CVI), which determines the optimum number of clusters in the data. From previous works, the improper cluster centroids positioning produced by clustering algorithms could reduce the performance of the validation process and performance produced by the previous state-of-the-art CVIs. In addition, those previous CVIs can only work properly with certain clustering algorithms and simple datasets structures, which their performances will reduce if they are applied to other clustering algorithms as well as more complex datasets. This study proposes an efficient CVI, namely, the validity clustering index based on finding the mean of clustered data (VCIM). The proposed approach combines the properties of the score function index and the mean to determine new cluster centroid positions. The performance of the VCIM index is compared with well-known CVIs on both artificial and real-life datasets. The obtained results on artificial datasets show that the proposed VCIM index outperforms the other CVIs in determining the true number of clusters for the five conventional clustering algorithms, namely, K-means, Fuzzy C-mean, agglomerative hierarchical average linkage clustering, variance-based differential evolution, and density peaks clustering and Particle swarm optimization (PDPC) algorithms. For the 14 real-word datasets, the proposed VCIM index correctly determined the optimum number of clusters for 11 out of 14 for the K-means clustering algorithm, 9 out of 14 for both Fuzzy clustering and agglomerative hierarchical average linkage clustering algorithms, 12 out of 14 for the variance-based differential evolution algorithm and 11 out of 14 datasets for PDPC. The obtained results using the proposed VCIM show its significance when combined with clustering algorithms and nominate its potential in various clustering applications.

SC-MEB: spatial clustering with hidden Markov random field using empirical Bayes.

Spatial transcriptomics has been emerging as a powerful technique for resolving gene expression profiles while retaining tissue spatial information. These spatially resolved transcriptomics make it feasible to examine the complex multicellular systems of different microenvironments. To answer scientific questions with spatial transcriptomics and expand our understanding of how cell types and states are regulated by microenvironment, the first step is to identify cell clusters by integrating the available spatial information. Here, we introduce SC-MEB, an empirical Bayes approach for spatial clustering analysis using a hidden Markov random field. We have also derived an efficient expectation-maximization algorithm based on an iterative conditional mode for SC-MEB. In contrast to BayesSpace, a recently developed method, SC-MEB is not only computationally efficient and scalable to large sample sizes but is also capable of choosing the smoothness parameter and the number of clusters. We performed comprehensive simulation studies to demonstrate the superiority of SC-MEB over some existing methods. We applied SC-MEB to analyze the spatial transcriptome of human dorsolateral prefrontal cortex tissues and mouse hypothalamic preoptic region. Our analysis results showed that SC-MEB can achieve a similar or better clustering performance to BayesSpace, which uses the true number of clusters and a fixed smoothness parameter. Moreover, SC-MEB is scalable to large ‘sample sizes’. We then employed SC-MEB to analyze a colon dataset from a patient with colorectal cancer (CRC) and COVID-19, and further performed differential expression analysis to identify signature genes related to the clustering results. The heatmap of identified signature genes showed that the clusters identified using SC-MEB were more separable than those obtained with BayesSpace. Using pathway analysis, we identified three immune-related clusters, and in a further comparison, found the mean expression of COVID-19 signature genes was greater in immune than non-immune regions of colon tissue. SC-MEB provides a valuable computational tool for investigating the structural organizations of tissues from spatial transcriptomic data.

Detecting the number of clusters in a network

Abstract Many clustering algorithms for complex networks depend on the choice for the number of clusters and it is often unclear how to make this choice. The number of eigenvalues located outside a circle in the spectrum of the non-backtracking matrix was conjectured to be an estimator of the number of clusters in a graph. We compare the estimate of the number of clusters obtained from the spectrum of the non-backtracking matrix with three estimators based on the concept of modularity and evaluate the methods on several benchmark graphs. We find that the non-backtracking method detects the number of clusters better than the modularity-based methods for the graphs in our simulation study, especially when the clusters have slightly different sizes. The estimates of the non-backtracking method are narrowly distributed around the true number of clusters for all benchmark graphs considered. Additionally, for graphs without a clustering structure, the non-backtracking method detects exactly one cluster, which is a convenient property of an estimator of the number of clusters. However, the lack of a well-defined concept of a cluster prevents sharp conclusions.

A density-peak-based clustering algorithm of automatically determining the number of clusters

Clustering is a typical and important method to discover new structures and knowledge from data sets. Most existing clustering methods need to know the number of clusters in advance, which is difficult. Some algorithms claim they do not need to know the number of clusters in advance. Among these algorithms, however, some need to manually determine the cluster centers in a decision graph, which is not easy; some assume that the number of initial cluster centers given is greater than the actual number of classes, but in fact the true number of clusters is not known. In order to tackle this issue, we propose a density-peak-based clustering algorithm of automatically determining the number of clusters. First, we design a density metric by using a continuous function which can well distinguish the densities of different data points. Then, we design a pre-clustering method which can get the initial cluster centers and the corresponding clusters. Furthermore, we propose an automatic clustering method which can automatically determine the final cluster centers and the corresponding clusters. Experiments are conducted on widely used data sets, and the results show the effectiveness of the proposed method.

CNAK: Cluster number assisted K-means

The K-means clustering algorithm is well-known for its easy computational approach. In this algorithm, essential cluster-level information is captured by the K cluster centroids. However, how many such centroids can reveal the structure of the underlying data depends upon the choice of K. In this paper, we propose a clustering algorithm in which the number of cluster K can be learned as well as it performs the clustering. Our work revolves around two observations: i) a large-sized random sampled dataset may have a similar distribution as the original data, and ii) for the true number of clusters their centroids, generated from a sampled datasets, approximate the cluster centroids generated from the original dataset. The first observation has paved the way to provide a scalable solution, and the second one forms the key aspect of building the proposed algorithm. We have tested our method on several real and synthetic datasets. Our method can solve a few pertinent issues of clustering a dataset: 1) detection of a single cluster in the absence of any other cluster in a dataset, 2) the presence of hierarchy, 3) clustering of a high dimensional dataset, 4) robustness over dataset having cluster imbalance, and 5) robustness to noise. We have observed significant improvement in speed and quality for predicting cluster numbers as well as the composition of clusters in a large dataset.

Clustering with statistical error control

AbstractThis article presents a clustering approach that allows for rigorous statistical error control similar to a statistical test. We develop estimators for both the unknown number of clusters and the clusters themselves. The estimators depend on a tuning parameter α which is similar to the significance level of a statistical hypothesis test. By choosing α, one can control the probability of overestimating the true number of clusters, while the probability of underestimation is asymptotically negligible. In addition, the probability that the estimated clusters differ from the true ones is controlled. In the theoretical part of the article, formal versions of these statements on statistical error control are derived in a baseline model with convex clusters. A simulation study and two applications to temperature and gene expression microarray data complement the theoretical analysis.

Efficient synthetical clustering validity indexes for hierarchical clustering

Clustering validation and identifying the optimal number of clusters are of great importance in expert and intelligent systems. However, the commonly used similarity measures for validating are not versatile to measure the complex data structure, in reality, some of which are not as effective as that of the used clustering algorithm which gives the clustering results. This paper studies the validity indexes for the hierarchical clustering algorithm and proposes a unified validity index framework. For the single-linkage agglomerative hierarchical clustering we propose two efficient synthetical clustering validity (SCV) indexes using the minimum spanning tree to calculate the intra-cluster compactness to overcome the deficiencies of the measurements in the existing validity indexes. For the general hierarchical clustering, a self-adaptive similarity measure strategy and two generalized synthetical clustering validity (GSCV) indexes, which are the extension of the proposed SCV indexes, are developed. The proposed SCV and GSCV indexes constitute a unified validity index framework, where SCV index is a special case of GSCV index, can avoid the incompatibility of the similarity measure between the clustering and validation. The experimental comparisons with the state-of-the-art validity indexes on artificial and real-world data sets demonstrate the efficiency of the proposed validity indexes in discovering the true number of clusters and dealing with various sorts of data sets, including imbalanced data sets.

Nonnegative self-representation with a fixed rank constraint for subspace clustering

A number of approaches to graph-based subspace clustering, which assumes that the clustered data points were drawn from an unknown union of multiple subspaces, have been proposed in recent years. Despite their successes in computer vision and data mining, most neglect to simultaneously consider global and local information, which may improve clustering performance. On the other hand, the number of connected components reflected by the learned affinity matrix is commonly inconsistent with the true number of clusters. To this end, we propose an adaptive affinity matrix learning method, nonnegative self-representation with a fixed rank constraint (NSFRC), in which the nonnegative self-representation and an adaptive distance regularization jointly uncover the intrinsic structure of data. In particular, a fixed rank constraint as a prior is imposed on the Laplacian matrix associated with the data representation coefficients to urge the true number of clusters to exactly equal the number of connected components in the learned affinity matrix. Also, we derive an efficient iterative algorithm based on an augmented Lagrangian multiplier to optimize NSFRC. Extensive experiments conducted on real-world benchmark datasets demonstrate the superior performance of the proposed method over some state-of-the-art approaches.

A hierarchical Gamma Mixture Model-based method for estimating the number of clusters in complex data

This paper proposes a new method for estimating the true number of clusters and initial cluster centers in a dataset with many clusters. The observation points are assigned to the data space to observe the clusters through the distributions of the distances between the observation points and the objects in the dataset. A Gamma Mixture Model (GMM) is built from a distance distribution to partition the dataset into subsets, and a GMM tree is obtained by recursively partitioning the dataset. From the leaves of the GMM tree, a set of initial cluster centers are identified and the true number of clusters is estimated. This method is implemented in the new GMM-Tree algorithm. Two GMM forest algorithms are further proposed to ensemble multiple GMM trees to handle high dimensional data with many clusters. The GMM-P-Forest algorithm builds GMM trees in parallel, whereas the GMM-S-Forest algorithm uses a sequential process to build a GMM forest. Experiments were conducted on 32 synthetic datasets and 15 real datasets to evaluate the performance of the new algorithms. The results have shown that the proposed algorithms outperformed the existing popular methods: Silhouette, Elbow and Gap Statistic, and the recent method I-nice in estimating the true number of clusters from high dimensional complex data.

On the Persistence of Clustering Solutions and True Number of Clusters in a Dataset

Typically clustering algorithms provide clustering solutions with prespecified number of clusters. The lack of a priori knowledge on the true number of underlying clusters in the dataset makes it important to have a metric to compare the clustering solutions with different number of clusters. This article quantifies a notion of persistence of clustering solutions that enables comparing solutions with different number of clusters. The persistence relates to the range of dataresolution scales over which a clustering solution persists; it is quantified in terms of the maximum over two-norms of all the associated cluster-covariance matrices. Thus we associate a persistence value for each element in a set of clustering solutions with different number of clusters. We show that the datasets where natural clusters are a priori known, the clustering solutions that identify the natural clusters are most persistent - in this way, this notion can be used to identify solutions with true number of clusters. Detailed experiments on a variety of standard and synthetic datasets demonstrate that the proposed persistence-based indicator outperforms the existing approaches, such as, gap-statistic method, X-means, Gmeans, PG-means, dip-means algorithms and informationtheoretic method, in accurately identifying the clustering solutions with true number of clusters. Interestingly, our method can be explained in terms of the phase-transition phenomenon in the deterministic annealing algorithm, where the number of distinct cluster centers changes (bifurcates) with respect to an annealing parameter.