Clustering Ensemble Selection Research Articles

Cluster ensemble selection based on maximum quality-maximum diversity

Diversity and quality are two important factors that affect clustering ensemble performance. Some base clusterings are irrelevant and redundant, which decreases the performance of the clustering ensemble. By removing irrelevant and redundant clusterings, diversity and quality are increased simultaneously, which often leads to a more accurate ensemble solution. The relationship between quality and diversity as an optimization problem is a challenge. Based on the minimum redundancy-maximum relevance (mRMR) criterion, pair-wise and non-pair-wise methods are proposed. In the pair-wise method, each clustering is weighted in contrast to other base clusterings, whereas in the non-pair-wise method, virtual labeling is obtained using a consensus function, and then based on this labeling, each clustering is weighted. To evaluate the performance of these methods, several experiments were conducted on 10 real datasets, and the obtained results were compared to those of full ensembles. The results showed that the proposed methods led to a more significant performance improvement compared with full ensembles and other clustering ensemble selection methods.

Dual-level clustering ensemble algorithm with three consensus strategies

Clustering ensemble (CE), renowned for its robust and potent consensus capability, has garnered significant attention from scholars in recent years and has achieved numerous noteworthy breakthroughs. Nevertheless, three key issues persist: (1) the majority of CE selection strategies rely on preset parameters or empirical knowledge as a premise, lacking adaptive selectivity; (2) the construction of co-association matrix is excessively one-sided; (3) the CE method lacks a more macro perspective to reconcile the conflicts among different consensus results. To address these aforementioned problems, a dual-level clustering ensemble algorithm with three consensus strategies is proposed. Firstly, a backward clustering ensemble selection framework is devised, and its built-in selection strategy can adaptively eliminate redundant members. Then, at the base clustering consensus level, taking into account the interplay between actual spatial location information and the co-occurrence frequency, two modified relation matrices are reconstructed, resulting in the development of two consensus methods with different modes. Additionally, at the CE consensus level with a broader perspective, an adjustable Dempster–Shafer evidence theory is developed as the third consensus method in present algorithm to dynamically fuse multiple ensemble results. Experimental results demonstrate that compared to seven other state-of-the-art and typical CE algorithms, the proposed algorithm exhibits exceptional consensus ability and robustness.

Cluster ensemble selection and consensus clustering: A multi-objective optimization approach

Cluster ensembles have emerged as a powerful tool to obtain clusters of data points by combining a library of clustering solutions into a consensus solution. In this paper, we address the cluster ensemble selection problem and design a multi-objective optimization-based solution framework to produce consensus solutions. Given a library of clustering solutions, we first design a preprocessing procedure that measures the agreement of each clustering solution with the other solutions and eliminates the ones that may mislead the process. We then develop a multi-objective optimization algorithm that selects representative clustering solutions from the preprocessed library with respect to size, coverage, and diversity criteria and combines them into a single consensus solution, for which the true number of clusters is assumed to be unknown. We conduct experiments on different benchmark data sets. The results show that our approach yields more accurate consensus solutions compared to full-ensemble and the existing approaches for most data sets. We also present an application on the customer segmentation problem, where our approach is used to segment customers and to find a consensus solution for each segment, simultaneously.

A fuzzy clustering ensemble selection based on active full-link similarity

A fuzzy clustering ensemble selection based on active full-link similarity

Examining unsupervised ensemble learning using spectroscopy data of organic compounds.

One solution to the challenge of choosing an appropriate clustering algorithm is to combine different clusterings into a single consensus clusteringresult, known as cluster ensemble (CE). This ensemble learning strategy can provide more robust and stable solutions across different domains and datasets. Unfortunately, not all clusterings in the ensemble contribute to the final data partition. Cluster ensemble selection (CES) aims at selecting a subset from a large library of clustering solutions to form a smaller cluster ensemble that performs as well as or better than the set of all available clustering solutions. In this paper, we investigate four CES methods for the categorization of structurally distinct organic compounds using high-dimensional IR and Raman spectroscopy data. Single quality selection (SQI) forms a subset of the ensemble by selecting the highest quality ensemble members. The Single Quality Selection (SQI) method is used with various quality indices to select subsets by including the highest quality ensemble members. The Bagging method, usually applied in supervised learning,ranksensemble members by calculating the normalized mutual information (NMI) between ensemble members and consensus solutions generated from a randomly sampled subset of the full ensemble. The hierarchical cluster and select method (HCAS-SQI) uses the diversity matrix of ensemble members to select a diverse set of ensemble members with the highest quality. Furthermore, a combining strategy can be used to combine subsets selected using multiple quality indices (HCAS-MQI) for the refinement of clustering solutions in the ensemble. The IR + Raman hybrid ensemble library is created by merging two complementary "views" of the organic compounds. This inherently more diverse library gives the best full ensemble consensus results. Overall, the Bagging method is recommended because it provides the most robust results that are better than or comparable to the full ensemble consensus solutions.

Multi-Objective Cluster Ensemble Based on Filter Refinement Scheme

Cluster ensemble improves the robustness and stability of clustering performances by utilizing multiple solutions. Although traditional cluster ensemble methods have achieved promising performances, they are not adaptive enough to cope with data sets that have multiple levels of complexities. Besides, these methods may contain noisy and redundancy members which have negative effects. To mitigate the above issues, in this paper, we propose a multi-objective filter refinement scheme (MOFRS). First, we perform various clustering methods on different representations of data to generate diverse solutions. Second, we propose a solution filter to select a proper method and reduce the number of initial partitions for a given data set. Third, four stability indices are designed to split instances into stable and unstable groups. Fourth, objective functions based on diversity and quality are utilized to quantify the goodness of base clustering solutions. Finally, we design an improvement oriented multi-objective evolutionary algorithm to optimize these objective functions. Extensive experimental results conducted on 27 real-world data sets show that MOFRS outperforms most cluster ensemble selection methods, and achieves statistically significant improvements, compared with full ensemble methods.

Semi-supervised Selective Clustering Ensemble based on constraint information

Clustering is an important research direction in data mining. However, there is no one clustering algorithm that can be applied efficiently in all situation. Clustering ensemble is the best way to solve the above-mentioned problems. It combines the results of multiple clustering algorithms, and the final result is significantly better than a single clustering algorithm. Although there is a lot of constraint information, the existing clustering ensemble algorithm does not utilize it. This paper uses constraint information in consensus function and proposes a Semi-supervised Selective Clustering Ensemble based on Chameleon (SSCEC) and Semi-supervised Selective Clustering Ensemble based on Ncut (SSCEN) to solve the above problem. SSCEC uses the chameleon algorithm as consensus function, and processes constraint information in subgraph partition and subgraph combining. SSCEN uses the Normalized cut algorithm as consensus function, and processes constraint information in the process of graph dichotomy. The experiment results show that our proposed two semi-supervised member selection clustering ensemble algorithms are better than other semi-supervised algorithms.

Clustering ensemble selection based on the extended Jaccard measure

Clustering ensemble selection has shown high efficiency in the improvement of the quality of clustering solutions. This technique comprises two important metrics: diversity and quality. It has been empirically proved that ensembles of higher effectiveness can be achieved through taking into consideration the diversity and quality simultaneously. However, the relationships between these two metrics in base clusterings have remained uncertain. This paper suggests a new hierarchical selection algorithm using a diversity/quality measure based on the Jaccard similarity measure. In the proposed algorithm, the selection of the subsets of the clustering partitions is done based on their diversity measures. The proposed diversity measure (in two types of pair-wise diversity and hybrid diversity) is applied to the proposed algorithm. Hypergraph-partitioning algorithm (HGPA), cluster-based similarity partition algorithm (CSPA), and meta-clustering algorithm (MCLA) were used to obtain the consensus solution and cluster ensemble selection results with a hierarchical method. The experimental results on 14 datasets showed that selecting a subset of base clusterings using the proposed algorithm led to more accurate results compared to those of the full ensemble. The effectiveness and robustness of the proposed algorithm were demonstrated in comparison with the full ensemble. The comparative results showed that the proposed method by new diversity measure outperformed the full ensemble.

From clustering to clustering ensemble selection: A review

Clustering, as an unsupervised learning, is aimed at discovering the natural groupings of a set of patterns, points, or objects. In clustering algorithms, a significant problem is the absence of a deterministic approach based on which users can decide which clustering method best matches a given set of input data. This is due to using certain criteria for optimization. Clustering ensemble as a knowledge reuse offers a solution to solve the challenges inherent in clustering. It seeks to explore results of high stability and robustness by composing computed solutions achieved by base clustering algorithms without getting access to the features. Combining base clusterings together degrades the quality of the final solution when low-quality ensemble members are used. Several researchers in this field have suggested the concept of clustering ensemble selection for the aim of selecting a subset of base clustering based on quality and diversity. While clustering ensemble makes a combination of all ensemble members, clustering ensemble selection chooses a subset of ensemble members and forms a smaller cluster ensemble that performs better than the clustering ensemble. This survey includes the historical development of data clustering that makes an overview on basic clustering techniques, discusses clustering ensemble algorithms including ensemble generation mechanisms and consensus function, and point out clustering ensemble selection techniques with considering quality and diversity.

The Research on Clustering Ensembles Selection Algorithm based on Semi-supervised K-means Clustering

Selective clustering ensemble algorithm can eliminate the inferior quality clustering member’s influence and can achieve a better clustering solution relative to the clustering ensemble algorithm. For high dimensional data clustering, in this paper, a novel selective ensemble algorithm based on semi-supervised K-means clustering is proposed. In this paper, through a large number of experiments to verify the validity of the proposed algorithm for dealing with high dimensional data clustering. The new algorithm can achieve statistically significant performance improvement over other clustering algorithms.

Social Network Optimization for Cluster Ensemble Selection

This paper studies the cluster ensemble selection problem for unsupervised learning. Given a large ensemble of clustering solutions, our goal is to select a subset of solutions to form a smaller yet better performing cluster ensemble than using all available solutions. The common way of aggregating the chosen solutions is accumulating the information of the selected results to a similarity matrix. This paper suggests transforming the similarity matrix to a modularity matrix and then applying a new consensus function which optimizes modularity measure in it. We represent the modularity maximization problem as a 0-1 quadratic program which can be exactly solved for small datasets. We also established a new greedy algorithm, namely sum linkage, to optimize the objective function specially for large scale datasets in a very short time. We show that the proposed consensus partition gets much closer to the actual cluster structure than the partitions obtained from the direct application of common cluster ensemble methods. The promising results compared with other most cited consensus functions show the excellent efficiency of the proposed method.

Multiple clustering and selecting algorithms with combining strategy for selective clustering ensemble

Clustering ensemble can overcome the instability of clustering and improve clustering performance. With the rapid development of clustering ensemble, we find that not all clustering solutions are effective in their final result. In this paper, we focus on selection strategy in selective clustering ensemble. We propose a multiple clustering and selecting approach (MCAS), which is based on different original clustering solutions. Furthermore, we present two combining strategies, direct combining and clustering combining, to combine the solutions selected by MCAS. These combining strategies combine results of MCAS and get a more refined subset of solutions, compared with traditional selective clustering ensemble algorithms and single clustering and selecting algorithms. Experimental results on UCI machine learning datasets show that the algorithm that uses multiple clustering and selecting algorithms with combining strategy performs well on most datasets and outperforms most selective clustering ensemble algorithms.

Cluster ensemble selection using balanced normalized mutual information

A bad partition in an ensemble will be removed by a cluster ensemble selection framework from the final ensemble. It is the main idea in cluster ensemble selection to remove these partitions (bad partitions) from the selected ensemble. But still, it is likely that one of them contains some reliable clusters. Therefore, it may be reasonable to apply the selection phase on cluster level. To do this, a cluster evaluation metric is needed. Some of these metrics have been recently introduced; each of them has its limitations. The weak points of each method have been addressed in the paper. Subsequently, a new metric for cluster assessment has been introduced. The new measure is named Balanced Normalized Mutual Information (BNMI) criterion. It balances the deficiency of the traditional NMI-based criteria. Additionally, an innovative cluster ensemble approach has been proposed. To create the consensus partition considering the elected clusters, a set of different aggregation-functions (called also consensus-functions) have been utilized: the ones which are based upon the co-association matrix (CAM), the ones which are based on hyper graph partitioning algorithms, and the ones which are based upon intermediate space. The experimental study indicates that the state-of-the-art cluster ensemble methods are outperformed by the proposed cluster ensemble approach.

Cluster ensemble selection using voting

Cluster ensemble selection using voting

Transfer Clustering Ensemble Selection.

Clustering ensemble (CE) takes multiple clustering solutions into consideration in order to effectively improve the accuracy and robustness of the final result. To reduce redundancy as well as noise, a CE selection (CES) step is added to further enhance performance. Quality and diversity are two important metrics of CES. However, most of the CES strategies adopt heuristic selection methods or a threshold parameter setting to achieve tradeoff between quality and diversity. In this paper, we propose a transfer CES (TCES) algorithm which makes use of the relationship between quality and diversity in a source dataset, and transfers it into a target dataset based on three objective functions. Furthermore, a multiobjective self-evolutionary process is designed to optimize these three objective functions. Finally, we construct a transfer CE framework (TCE-TCES) based on TCES to obtain better clustering results. The experimental results on 12 transfer clustering tasks obtained from the 20newsgroups dataset show that TCE-TCES can find a better tradeoff between quality and diversity, as well as obtaining more desirable clustering results.

Cluster’s Quality Evaluation and Selective Clustering Ensemble

Clustering ensemble has drawn much attention in recent years due to its ability to generate a high quality and robust partition result. Weighted clustering ensemble and selective clustering ensemble are two general ways to further improve the performance of a clustering ensemble method. Existing weighted clustering ensemble methods assign the same weight to each cluster in a partition of the ensemble. Since the qualities of the clusters in a partition are different, the clusters should be weighted differently. To address this issue, this article proposes a new measure to calculate the similarity between a cluster and a partition. Theoretically, this measure is effective in handling two problems in measuring the quality of a cluster, which are defined as the symmetric problem and the context meaning problem. In addition, some properties of the proposed measure are analyzed. This measure can be easily expanded to a clustering performance measure that calculates the similarity between two partitions. As a result of this measure, we propose a novel selective clustering ensemble framework, which considers the differences between the objective of the ensemble selection stage and the object of the ensemble integration stage in the selective clustering ensemble. To verify the performance of the new measure, we compare the performance of the measure with the two existing measures in weighting clusters. The experiments show that the proposed measure is more effective. To verify the performance of the novel framework, four existing state-of-the-art selective clustering ensemble frameworks are employed as references. The experiments show that the proposed framework is statistically better than the others on 17 UCI benchmark datasets, 8 document datasets, and the Olivetti Face Database.

Clustering ensemble selection considering quality and diversity

It is highly likely that there is a partition that is judged by a stability measure as a bad one while it contains one (or more) high quality cluster(s); and then it is totally neglected. So, inspiring from the evaluation of partitions, researchers turn to define measures for evaluation of clusters. Many stability measures have been proposed such as Normalized Mutual Information to validate a partition. The defined measures are based on Normalized Mutual Information. The drawback of the commonly used approach will be discussed in this paper and a criterion is proposed to assess the association between a cluster and a partition which is called Edited Normalized Mutual Information, ENMI criterion. The ENMI criterion compensates the drawback of the common Normalized Mutual Information (NMI) measure. Also, a clustering ensemble method that is based on aggregating a subset of primary clusters is proposed. The proposed method uses the Average ENMI as fitness measure to select a number of clusters. The clusters that satisfy a predefined threshold of the mentioned measure are selected to participate in the final ensemble. To combine the chosen clusters a set of consensus function methods are employed. One class of the used consensus functions is the co-association based consensus functions. Since the Evidence Accumulation Clustering, EAC, method can’t derive the co-association matrix from a subset of clusters, Extended EAC, EEAC, is employed to construct the co-association matrix from the chosen subset of clusters. The second class of the used consensus functions is based on hyper graph partitioning algorithms. The other class of the used consensus functions considers the chosen clusters as a new feature space and uses a simple clustering algorithm to extract the consensus partitioning. The empirical studies show that the proposed method outperforms other well-known ensembles.

A comparative study of clustering ensemble algorithms

Since clustering ensemble was proposed, it has rapidly attracted much attention. This paper makes an overview of recent research on clustering ensemble about generative mechanism, selective clustering ensemble, consensus function and application. Twelve clustering ensemble algorithms are described and compared to choose a basic one. The experiment shows that using k-means with different initializations as generative mechanism and average-linkage agglomerative clustering as consensus function is the best one. As ensemble size increases, the performance of clustering ensemble improves. The basic clustering ensemble algorithm with suitable ensemble size is compared with clustering algorithms and the experiment shows that clustering ensemble is better than clustering. The influence of diversity on clustering ensemble is instructive to selecting members. The experiment shows that selecting members in high quality and big diversity for low-dimensional data sets, and selecting members in high quality and median diversity for high-dimensional data sets are better than traditional clustering ensemble.

Method for detecting abnormal behaviour of users based on selective clustering ensemble

With the development of cloud computing in the mobile communications industry, the credibility between users and the cloud has become an important obstacle to the development of mobile cloud services. Therefore, the user's abnormal behaviour detection is particularly important. The authors propose a selective clustering ensemble algorithm based on fractal dimension. In their proposed algorithm, they firstly use the sliding window to dynamically obtain data to improve the accuracy of user behaviour acquisition. Secondly, they use the initial clustering stage and incremental clustering stage to produce clustered members. Thirdly, they also use the Duun_index to select the clustered members, then the selection of high-quality clustered members with the voting algorithm to get the final result. Finally, they use the average difference of change to determine whether the current behaviour is abnormal based on the clustering results. The experimental results show that the proposed scheme has a better performance than the traditional clustering algorithm in clustering accuracy. Moreover, their algorithm can improve the detection rate and accuracy rate, respectively.

Two-Level-Oriented Selective Clustering Ensemble Based on Hybrid Multi-Modal Metrics

The purpose of selective clustering ensemble is to select a subset of base clustering partitions with predictive performance and combine these partitions into more accurate and stable final results. Traditional approaches tend to utilize the well-known validity criteria such as NMI to evaluate the quality and diversity of base clustering partitions in the selection process. However, the characteristics of the original data and the data structure itself are commonly neglected. Furthermore, the generation process of base clustering partitions is more concerned with diversity and less consideration of quality. To tackle these problems, we propose a new selective clustering ensemble scheme. In the process of generating base clustering partitions, k-means and hierarchical clustering algorithm alternately combined with random projection method are employed to generate diverse base partitions. Meanwhile, in order to improve the quality of base clustering partitions, we propose a new selection strategy for the number of clusters $k$ in k-means algorithm. In the clustering selection process, both diversity and quality of the base clustering partitions are evaluated by multi-modal metrics from two levels: clustering labels and data structure. Based on five UCI benchmark datasets, experimental results demonstrate that the proposed method not only can generate but also select base clustering partitions with both diversity and quality. Experimental analyses show the validity and stability of the proposed scheme.