Clustering Ensemble Algorithm Research Articles

Cluster Ensemble Based on Iteratively Refined Co-Association Matrix

Cluster ensemble aims at discovering the intrinsic structure of a given dataset robustly and stably, and achieves this by combining multiple base partitions into a single final one. Some cluster ensemble algorithms in the literature are based on a co-association matrix, which can be viewed as a space transformation of the original dataset. However, the co-association matrix does not always depict the cluster structure well. In this paper, we propose a method to refine the co-association matrix and make it describe the structure more accurately. The main idea is to define an inter-cluster similarity with the co-association matrix, then repeatedly combine the most similar cluster pair of a base partition. In turn, the co-association matrix is updated in terms of the combined cluster pair. Furthermore, based on the refined co-association matrix, three consensus schemes are designed to generate the final clustering. The experimental results on eight synthetic datasets and eight real datasets demonstrate that the refined co-association matrix depicts the cluster structure more accurate than the original one, and the proposed ensemble schemes with the refined matrix can produce clusterings with high quality compared with the several state-of-the-art methods.

An Information-Theoretical Framework for Cluster Ensemble

Cluster ensemble is a very important tool that aggregates several base clusterings to generate a single output clustering with improved robustness and stability. However, the quality of the final clustering is often affected by uncertainties on the generation and integration of base clusterings. In this paper, we develop an information-theoretical framework which makes an effort to obtain a final clustering with high consensus on both the original data set and the base clustering set by minimizing the two uncertainties of cluster ensemble. In this framework, we provide a weighted consensus measure based on information entropy to evaluate the quality of a clustering, the similarity between clusters and the similarity between objects. Based on the measure, we propose three weighted cluster ensemble algorithms with different ensemble strategies in the framework, including the weighted feature consensus algorithm, the weighted relabeling consensus algorithm and the weighted pairwise-similarity consensus algorithm. In the experimental analysis, we compare the proposed algorithms with other existing clustering ensemble algorithms on several data sets. The comparison results illustrate the proposed algorithms are very effective and robust.

An ensemble clusterer of multiple fuzzy k-means clusterings to recognize arbitrarily shaped clusters

Fuzzy cluster ensemble is an important research component of ensemble learning, which is used to aggregate several fuzzy base clusterings to generate a single output clustering with improved robustness and quality. However, since clustering is unsupervised, where “accuracy” does not have a clear meaning, it is difficult for existing ensemble methods to integrate multiple fuzzy $k$ -means clusterings to find arbitrarily shaped clusters. To overcome the deficiency, we propose a new ensemble clusterer (algorithm) of multiple fuzzy $k$ -means clusterings based on a local hypothesis. In the new algorithm, we study the extraction of local-credible memberships from a base clustering, the production of multiple base clusterings with different local-credible spaces, and the construction of cluster relation based on indirect overlap of local-credible spaces. The proposed ensemble clusterer not only inherits the scalability of fuzzy $k$ -means but also overcomes the inability to find arbitrarily shaped clusters. We compare the proposed algorithm with other cluster ensemble algorithms on several synthetical and real datasets. The experimental results illustrate the effectiveness and efficiency of the proposed algorithm.

Multi-view data ensemble clustering: a cluster-level perspective

Ensemble clustering has recently emerged a powerful clustering analysis technology for multi-view data. From the existing work, these techniques held great promise, but most of them are inefficient for large data. Some researchers have also proposed efficient ensemble clustering algorithms, but these algorithms devote to data objects with the same feature spaces, which are not satisfied for multi-view data. To overcome these deficiencies, an efficient ensemble clustering algorithm for multi-view mixed data is developed from the cluster-level perspective. Firstly, a set of clustering solutions are produced with the K-prototypes clustering algorithm on each view multiple times, respectively. Then, a cluster-cluster similarity matrix is constructed by considering all the clustering solutions. Next, the METIS algorithm is conduct meta-clustering based on the similarity matrix. After that, the final clustering results are obtained by applying majority voting to assign the objects to their corresponding clusters based on the meta-clustering. The corresponding time complexity of the proposed algorithm is analysed as well. Experimental results on several multi-view datasets demonstrated the superiority of our proposed algorithm.

Review of Traditional and Ensemble Clustering Algorithms for High Dimensional Data

High-dimensional data is explained by a huge quantity of features, introduces new issues to clustering. The so-named 'high dimensionality', creates initially to explain the common increase in time complexity of several computational issues, so the performances of the general clustering algorithms are unsuccessful. Accordingly, several works have been focused on introducing new techniques and clustering algorithms for handling higher dimensionality data. Regular to all clustering algorithms is the fact with the purpose of they need a various fundamental evaluation of similarity among data objects. However still, the existing clustering algorithms have some open research issues. In this review work, we provide a summary of the result of high-dimensional data space and their implications for various clustering algorithms. It also presents a detailed overview of many clustering algorithms with several types: subspace methods, modelbased clustering, density-based clustering methods; partition based clustering methods, etc., including a more detailed description of recent work of their own advantages and disadvantages for solving higher dimensionality data problem. The scope of the future work to extend the present clustering methods and algorithms are also discussed at end of the work.

An enhanced deterministic K-Means clustering algorithm for cancer subtype prediction from gene expression data

BackgroundClustering algorithms with steps involving randomness usually give different results on different executions for the same dataset. This non-deterministic nature of algorithms such as the K-Means clustering algorithm limits their applicability in areas such as cancer subtype prediction using gene expression data. It is hard to sensibly compare the results of such algorithms with those of other algorithms. The non-deterministic nature of K-Means is due to its random selection of data points as initial centroids. MethodWe propose an improved, density based version of K-Means, which involves a novel and systematic method for selecting initial centroids. The key idea of the algorithm is to select data points which belong to dense regions and which are adequately separated in feature space as the initial centroids. ResultsWe compared the proposed algorithm to a set of eleven widely used single clustering algorithms and a prominent ensemble clustering algorithm which is being used for cancer data classification, based on the performances on a set of datasets comprising ten cancer gene expression datasets. The proposed algorithm has shown better overall performance than the others. ConclusionThere is a pressing need in the Biomedical domain for simple, easy-to-use and more accurate Machine Learning tools for cancer subtype prediction. The proposed algorithm is simple, easy-to-use and gives stable results. Moreover, it provides comparatively better predictions of cancer subtypes from gene expression data.

SCE: A Manifold Regularized Set-Covering Method for Data Partitioning

Cluster analysis plays a very important role in data analysis. In these years, cluster ensemble, as a cluster analysis tool, has drawn much attention for its robustness, stability, and accuracy. Many efforts have been done to combine different initial clustering results into a single clustering solution with better performance. However, they neglect the structure information of the raw data in performing the cluster ensemble. In this paper, we propose a Structural Cluster Ensemble (SCE) algorithm for data partitioning formulated as a set-covering problem. In particular, we construct a Laplacian regularized objective function to capture the structure information among clusters. Moreover, considering the importance of the discriminative information underlying in the initial clustering results, we add a discriminative constraint into our proposed objective function. Finally, we verify the performance of the SCE algorithm on both synthetic and real data sets. The experimental results show the effectiveness of our proposed method SCE algorithm.

Particle Swarm Clustering in clustering ensembles: Exploiting pruning and alignment free consensus

A clustering ensemble combines in a consensus function the partitions generated by a set of independent base clusterers. In this study both the employment of particle swarm clustering (PSC) and ensemble pruning (i.e., selective reduction of base partitions) using evolutionary techniques in the design of the consensus function is investigated. In the proposed ensemble, PSC plays two roles. First, it is used as a base clusterer. Second, it is employed in the consensus function; arguably the most challenging element of the ensemble. The proposed consensus function exploits a representation for the base partitions that makes cluster alignment unnecessary, allows for the combination of partitions with different number of clusters, and supports both disjoint and overlapping (fuzzy, probabilistic, and possibilistic) partitions. Results on both synthetic and real-world data sets show that the proposed ensemble can produce statistically significant better partitions, in terms of the validity indices used, than the best base partition available in the ensemble. In general, a small number of selected base partitions (below 20% of the total) yields the best results. Moreover, results produced by the proposed ensemble compare favorably to those of state-of-the-art clustering algorithms, and specially to swarm based clustering ensemble algorithms.

Fast Constrained Spectral Clustering and Cluster Ensemble with Random Projection.

Constrained spectral clustering (CSC) method can greatly improve the clustering accuracy with the incorporation of constraint information into spectral clustering and thus has been paid academic attention widely. In this paper, we propose a fast CSC algorithm via encoding landmark-based graph construction into a new CSC model and applying random sampling to decrease the data size after spectral embedding. Compared with the original model, the new algorithm has the similar results with the increase of its model size asymptotically; compared with the most efficient CSC algorithm known, the new algorithm runs faster and has a wider range of suitable data sets. Meanwhile, a scalable semisupervised cluster ensemble algorithm is also proposed via the combination of our fast CSC algorithm and dimensionality reduction with random projection in the process of spectral ensemble clustering. We demonstrate by presenting theoretical analysis and empirical results that the new cluster ensemble algorithm has advantages in terms of efficiency and effectiveness. Furthermore, the approximate preservation of random projection in clustering accuracy proved in the stage of consensus clustering is also suitable for the weighted k-means clustering and thus gives the theoretical guarantee to this special kind of k-means clustering where each point has its corresponding weight.

A Model of Telecommunication Network Performance Anomaly Detection Based on Service Features Clustering

It proves to be increasingly promising to evaluate the network coverage and network capacity via applying data generated by network interaction provided by the mobile operators in recent years. Self-organized optimization of network coverage and network capacity is a key solution to cope with the rapid growth of mobile data services and user expectation. In this paper, a network performance anomaly detection model based on service feature clustering is proposed. First, the complexity of the wireless network environment and the difference of user behavior are fully considered, an ensemble clustering algorithm is adopted for scene classification by combining the features of various data services with multi-dimensional physical scene characteristics, and, then, the cell categories with different characteristics are marked. Second, each category matches the corresponding network indicators and the weights of each indicators, which are trained from historical data. Finally, network performance anomaly detection is conducted on the basis of these indicators in every scene so that a new approach to evaluate the performance of wireless network can be realized. We apply this method to our long term evolution network, and the actual operation results have confirmed that the algorithm in question is highly effective and relevant.

A Hybrid Spectral Clustering and Deep Neural Network Ensemble Algorithm for Intrusion Detection in Sensor Networks.

The development of intrusion detection systems (IDS) that are adapted to allow routers and network defence systems to detect malicious network traffic disguised as network protocols or normal access is a critical challenge. This paper proposes a novel approach called SCDNN, which combines spectral clustering (SC) and deep neural network (DNN) algorithms. First, the dataset is divided into k subsets based on sample similarity using cluster centres, as in SC. Next, the distance between data points in a testing set and the training set is measured based on similarity features and is fed into the deep neural network algorithm for intrusion detection. Six KDD-Cup99 and NSL-KDD datasets and a sensor network dataset were employed to test the performance of the model. These experimental results indicate that the SCDNN classifier not only performs better than backpropagation neural network (BPNN), support vector machine (SVM), random forest (RF) and Bayes tree models in detection accuracy and the types of abnormal attacks found. It also provides an effective tool of study and analysis of intrusion detection in large networks.

Multigranulation information fusion: A Dempster-Shafer evidence theory-based clustering ensemble method

Clustering analysis is a fundamental technique in machine learning, which is also widely used in information granulation. Multiple clustering systems granulate a data set into multiple granular structures. Therefore, clustering ensemble can serve as an important branch of multigranulation information fusion. Many approaches have been proposed to solve the clustering ensemble problem. This paper focuses on the direct approaches which involve two steps: finding cluster correspondence and utilizing a fusion strategy to produce a final result. The existing direct approaches mainly discuss the process of finding cluster correspondence, while the fusing process is simply done by voting. In this paper, we mainly focus on the fusing process and propose a Dempster-Shafer evidence theory-based clustering ensemble algorithm. The advantage of the algorithm is that the information of an object’s surrounding cluster structure is taken into consideration by using its neighbors to describe it. First, we find neighbors of each object and generate its label probability outputs in every base partition. Second, these label probability outputs are integrated based on DS theory. Theoretically, our method is superior to other voting methods. Besides, several experiments show that the proposed algorithm is statistically better than seven other clustering ensemble methods.

Discovering patterns in traveler behaviour using segmentation

We consider the problem of finding common behavioral patterns among travelers in an airline network through the process of clustering. Travelers can be characterized at relational or transactional level. In this article, we focus on the transactional level characterization; our unit of analysis is a single trip, rather than a customer relationship comprising multiple trips. We begin by characterizing a trip in terms of a number of features that pertain to the booking and travel behavior. Trips thus characterized are then grouped using an ensemble clustering algorithm that aims to find stable clusters as well as discover subgroup structures within groups. A multidimensional analysis of trips based on these groupings leads us to discover non-trivial patterns in traveler behaviour that can then be exploited for better revenue management.

Cluster ensemble framework based on the group method of data handling

Cluster ensemble is a powerful method for improving both the robustness and the stability of unsupervised classification solutions. This paper introduced group method of data handling (GMDH) to cluster ensemble, and proposed a new cluster ensemble framework, which named cluster ensemble framework based on the group method of data handling (CE-GMDH). CE-GMDH consists of three components: an initial solution, a transfer function and an external criterion. Several CE-GMDH models can be built according to different types of transfer functions and external criteria. In this study, three novel models were proposed based on different transfer functions: least squares approach, cluster-based similarity partitioning algorithm and semidefinite programming. The performance of CE-GMDH was compared among different transfer functions, and with some state-of-the-art cluster ensemble algorithms and cluster ensemble frameworks on synthetic and real datasets. Experimental results demonstrate that CE-GMDH can improve the performance of cluster ensemble algorithms which used as the transfer functions through its unique modelling process. It also indicates that CE-GMDH achieves a better or comparable result than the other cluster ensemble algorithms and cluster ensemble frameworks.

Ensemble Clustering Algorithm with Supervised Classification of Clinical Data for Early Diagnosis of Coronary Artery Disease

Ensemble Clustering Algorithm with Supervised Classification of Clinical Data for Early Diagnosis of Coronary Artery Disease

Hierarchical cluster ensemble model based on knowledge granulation

Cluster ensemble has been shown to be very effective in unsupervised classification learning by generating a large pool of different clustering solutions and then combining them into a final decision. However, the task of it becomes more difficult due to the inherent complexities among base cluster results, such as uncertainty, vagueness and overlapping. Granular computing is one of the fastest growing information-processing paradigms in the domain of computational intelligence and human-centric systems. As the core part of granular computing, the rough set theory dealing with inexact, uncertain, or vague information, has been widely applied in machine learning and knowledge discovery related areas in recent years. From these perspectives, in this paper, a hierarchical cluster ensemble model based on knowledge granulation is proposed with the attempt to provide a new way to deal with the cluster ensemble problem together with ensemble learning application of the knowledge granulation. A novel rough distance is introduced to measure the dissimilarity between base partitions and the notion of knowledge granulation is improved to measure the agglomeration degree of a given granule. Furthermore, a novel objective function for cluster ensembles is defined and the corresponding inferences are made. A hierarchical cluster ensemble algorithm based on knowledge granulation is designed. Experimental results on real-world data sets demonstrate the effectiveness for better cluster ensemble of the proposed method.

Abstract P6-08-15: Deconstructing the TNM staging system for breast cancer

Abstract Background The TNM staging system is a standard classification for recording extent of disease in breast cancer. However, with progress in understanding tumor biology, it is unknown how new prognostic factors that will eventually be integrated with the TNM will affect its predictive ability. Our objective was to show the impact on 10-year survival rates for breast cancer as different combinations of prognostic factors are integrated into the TNM. Methods: Data were obtained from the Surveillance, Epidemiology, and End Results (SEER) Program of the National Cancer Institute for the years 1991 through 2000. After exclusions, 132,339 cases of female breast cancer were available. An ensemble clustering algorithm was used to calculate survival after including additional prognostic factors listed in SEER in the TNM. Combinations of the following 6 factors were sequentially added to the TNM: tumor grade, ER/PR status, age at diagnosis, racial/ethnic group, and histological tumor type. Results: Survival rates amongst some tumors with the same TNM stage varied as new factors were integrated into the TNM. Factors associated with favorable outcome usually were associated with better survival than factors associated with less favorable outcome for each stage group with varying degrees. There were 4 different tumor combinations that represented 4 different TNM stages that all corresponded to a 90% 10-year survival when additional factors were added to the TNM stage. Integration of additional prognostic factors led to a crossover in survival of some stage groups. In one combination (T1, N2, grade 1, ER+, PR+, age <50: 131111) patients who were assigned stage IIIA had a 10-year rate of 90%, which qualifies for a stage I category. Survival Crossover in TNM Staging10-year Survival (%)Prognostic Factor CombinationTNM StageNumber of Patients90113IA1899390112222IA22469021211IIA510190122112IIA40199031111IIB68290131111IIIA82582232221IIB11035822322213IIB92583331111IIIA1285814211111IIIC64**Abbreviated table. Prognostic Factor Combination in order from left to right: T, N, grade, ER status, PR status, age, race, histological type. * T1 = 1; T2 = 2; T3 = 3; N0 = 1; N1= 2; N2 = 3; N3 = 4; Grade 1 = 1; Grade 2 = 2; Grade 3 = 3; ER+ = 1; ER- = 2; PR+ = 1; PR- = 2; Age <= 50 = 1; Age >50 = 2; Conclusions: Integrating new prognostic factors into the TNM always changed the outcome. Survival rates, therefore, are relative and depend on the selection of prognostic factors. Adding new factors selected different cohorts from the population which had a heterogeneous population of cancer survivors. These cohorts usually had different survival rates compared with the overall population from which they were drawn. Integrating combinations of prognostic factors revealed frequent crossover of stage groups at 10 years, which is a violation of a staging system and could impact the interpretation of clinical trials. Citation Format: Jigar A Patel, Matthew T Hueman, Dechang Chen, Donald E Henson. Deconstructing the TNM staging system for breast cancer [abstract]. In: Proceedings of the Thirty-Seventh Annual CTRC-AACR San Antonio Breast Cancer Symposium: 2014 Dec 9-13; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2015;75(9 Suppl):Abstract nr P6-08-15.

A H-K CLUSTERING ALGORITHM FOR HIGH DIMENSIONAL DATA USING ENSEMBLE LEARNING

Advances made to the traditional clustering algorithms solves the various problems such as curse of dimensionality and sparsity of data for multiple attributes. The traditional H-K clustering algorithm can solve the randomness and apriority of the initial centers of K-means clustering algorithm. But when we apply it to high dimensional data it causes the dimensional disaster problem due to high computational complexity. All the advanced clustering algorithms like subspace and ensemble clustering algorithms improve the performance for clustering high dimension dataset from different aspects in different extent. Still these algorithms will improve the performance form a single perspective. The objective of the proposed model is to improve the performance of traditional H-K clustering and overcome the limitations such as high computational complexity and poor accuracy for high dimensional data by combining the three different approaches of clustering algorithm as subspace clustering algorithm and ensemble clustering algorithm with H-K clustering algorithm.

An efficient clustering ensemble selection algorithm

Clustering ensemble selection has been confirmed that it can always achieve better result than traditional clustering ensemble algorithms. However, many selective clustering ensemble algorithms cannot eliminate the inferior quality partitions' influence and the accuracy of clustering is not high. In order to solve the problems, the paper proposes a new selective clustering ensemble algorithm. The algorithm, firstly, uses clustering validity evaluation to evaluate all available clustering ensemble partitions and selects the best quality as reference partition; secondly, the paper defines selection strategy via the quality and diversity; lastly, the paper proposes the adaptive weight strategy of ensemble members. The experimental results show that the new algorithm is effective and clustering performance could be significantly improved.

A H-K Clustering Algorithm For High Dimensional Data Using Ensemble Learning

Advances made to the traditional clustering algorithms solves the various problems such as curse of dimensionality and sparsity of data for multiple attributes. The traditional H-K clustering algorithm can solve the randomness and apriority of the initial centers of K-means clustering algorithm. But when we apply it to high dimensional data it causes the dimensional disaster problem due to high computational complexity. All the advanced clustering algorithms like subspace and ensemble clustering algorithms improve the performance for clustering high dimension dataset from different aspects in different extent. Still these algorithms will improve the performance form a single perspective. The objective of the proposed model is to improve the performance of traditional H-K clustering and overcome the limitations such as high computational complexity and poor accuracy for high dimensional data by combining the three different approaches of clustering algorithm as subspace clustering algorithm and ensemble clustering algorithm with H-K clustering algorithm.