Clustering Ensemble Algorithm Research Articles

On pruning the search space for clustering ensemble problems

Clustering ensemble has become a very popular technique in the past few years due to its potentialities for improving the clustering results. Roughly speaking it consists in the combination of different partitions of the same set of objects in order to obtain a consensus one. A common way of defining the consensus partition is as the solution of the median partition problem. In this way, the consensus partition is defined as the solution of a complex optimization problem. In this paper, we study possible prunes of the search space for this optimization problem. Particularly, we introduce a new prune that allows a dramatic reduction of the search space. We also provide a characterization of the family of dissimilarity measures that can be used to take advantage of this prune and we present two measures that fit into this family. We carry out an experimental study on synthetic data by comparing, under different circumstances, the size of the original search space and the size after the proposed prunes. Outstanding reductions are obtained, which can be beneficial for the development of clustering ensemble algorithms. We also compare, on real data, the behavior of a simulated annealing-based ensemble algorithm in the original partition space and in the two proposed pruned spaces. In all cases, the proposed prunes allow the algorithm to find solutions closer to the theoretical optimum.

A Novel Ensemble based Cluster Analysis using Similarity Matrices and Clustering Algorithm (SMCA)

today's world data analytics is gaining popularity due to user's motivation towards online data storage. This storage is not organized because of content types and data handling schemes complexity. User aims to retrieve data in lesser time with logical outcomes as desired can be achieved by applying data mining. Clustering in data mining is one of the known categorization approach used for formation of groups of similar elements having certain properties in common with other elements. This formation sometime creates noisy result in terms of formatted clusters. It depends on various factors such as distance measures, proximity values, objective functions, categorical or numerical attribute types etc. Over the last few years various schemes are suggested by different authors for improving the performance of tradition clustering algorithms. Among them, one is ensemble based clustering. Ensemble uses the mechanism for criteria selection from newly formed clusters with a defined portioning and joining methods to generate a single result instead of multiple solutions. The generation results are affected by various environmental parameters such as number of cluster, partitioning types, proximity values, objective function etc. This paper propose a novel SMCA based ensemble clustering algorithm for improvements over the existing issues defined in the paper. At the primary level of work and analytical evaluations, it shows the promising results in near future.

Active Sampling for Constrained Clustering

Constrained clustering is a framework for improving clustering performance by using constraints about data pairs. Since performance of constrained clustering depends on the set of constraints used, a method is needed to select good constraints that promote clustering performance. In this paper, we propose an active sampling method working with a constrained cluster ensemble algorithm that aggregates clustering results that a modified COP-Kmeans iteratively produces by changing the priorities of constraints. Our method follows the approach of uncertainty sampling and measures uncertainty using variations of clustering results where data pairs are clustered together in some results but not in others. It selects the data pair to be labeled that has the most variable result during cluster ensemble process. Experimental results show that our method outperforms random sampling. We further investigate the effect of important parameters.

Identifying functional homogeneity in a dynamic environment: Application to soft-substrate fish assemblages off the Portuguese coast

In fish community assessment, measures of functional integrity have shown to be the most suited to detect anthropogenic impacts in a background of natural variation. However, minimising background noise involves the establishment of homogeneous areas, which is still mostly done based on taxonomic approaches or by relying on abiotic factors. The present study proposes a framework to identify functional homogeneity in demersal soft-substrate fish assemblages along the Portuguese coast, by relying on functional metrics such as trophic guild composition. Using metric pairwise constrained k-means (MPCK-means), sampling stations were clustered based on individual categories of functional metrics, while taking into account the minimum achievable homogeneity contained in 5years of bottom trawl surveys. All individual clustering solutions resulting from different functional categories were ultimately combined into a final set of seven assemblage types using an ensemble clustering algorithm. Clustering patterns found were linked to depth, substrate type and latitudinal variation. Biomass-based metrics performed better in comparison to abundance-based metrics, but the combination of both achieved the best results in the definition of a final set of seven assemblage types along the coast.

Enhanced clustering ensemble algorithm based on characteristics of data sets

The popular clustering ensemble algorithms cannot give the appropriate treatment program in the light of the different characteristics of the different data sets.A new clustering ensemble algorithm — Enhanced Clustering Ensemble algorithm based on Characteristics of Data sets(ECECD) was proposed for overcoming this defect.ECECD was composed of generation of base clustering,selection of base clustering and consensus function.It selected a special range of ensemble members to form the final ensemble and produced the final clustering based on the characteristic of the data set.Three Benchmark data sets including ecoli,leukaemia and Vehicle were clustered in the experiment,and the clustering errors gained by the proposed algorithm were 0.014,0.489 and 0.361 respectively,which were always the minimum compared with that of the other algorithms such as Bagging based Structure Ensemble Approach(BSEA),Hybrid Cluster Ensemble(HCE) and Cluster-Oriented Ensemble Classifier(COES).The Normalized Mutual Information(NMI) values of the proposed algorithm were also always higher than that of these algorithms when increasing candidate base clusterings.Therefore,compared with these popular clustering ensemble algorithms,the proposed algorithm has the highest clustering precision and the strongest scalability.

Ensemble clustering by means of clustering embedding in vector spaces

Ensemble clustering is a recently evolving research direction in cluster analysis and has found several different application domains. In this work the complex ensemble clustering problem is reduced to the well-known Euclidean median problem by clustering embedding in vector spaces. The Euclidean median problem is solved by the Weiszfeld algorithm and an inverse transformation maps the Euclidean median back into the clustering domain. In the experiment study different evaluation strategies are considered. The proposed embedding strategy is compared to several state-of-art ensemble clustering algorithms and demonstrates superior performance.

To improve the quality of cluster ensembles by selecting a subset of base clusters

Conventional clustering ensemble algorithms employ a set of primary results; each result includes a set of clusters which are emerged from data. Given a large number of available clusters, one is faced with the following questions: (a) can we obtain the same quality of results with a smaller number of clusters instead of full ensemble? (b) If so, which subset of clusters is more efficient to be used in the ensemble? In this paper, these two questions are going to be answered. We explore a clustering ensemble approach combined with a cluster stability criterion as well as a dataset simplicity criterion to discover the finest subset of base clusters for each kind of datasets. Also, a novel method is proposed in order to accumulate the selected clusters and to extract final partitioning. Although it is expected that by reducing the size of ensemble the performance decreases, our experimental results show that our selecting mechanism generally lead to superior results.

Applicability of Ensemble Clustering and Ensemble Classification Algorithm for User Navigation Pattern Prediction

Applicability of Ensemble Clustering and Ensemble Classification Algorithm for User Navigation Pattern Prediction

Protein complex detection via weighted ensemble clustering based on Bayesian nonnegative matrix factorization.

Detecting protein complexes from protein-protein interaction (PPI) networks is a challenging task in computational biology. A vast number of computational methods have been proposed to undertake this task. However, each computational method is developed to capture one aspect of the network. The performance of different methods on the same network can differ substantially, even the same method may have different performance on networks with different topological characteristic. The clustering result of each computational method can be regarded as a feature that describes the PPI network from one aspect. It is therefore desirable to utilize these features to produce a more accurate and reliable clustering. In this paper, a novel Bayesian Nonnegative Matrix Factorization(NMF)-based weighted Ensemble Clustering algorithm (EC-BNMF) is proposed to detect protein complexes from PPI networks. We first apply different computational algorithms on a PPI network to generate some base clustering results. Then we integrate these base clustering results into an ensemble PPI network, in the form of weighted combination. Finally, we identify overlapping protein complexes from this network by employing Bayesian NMF model. When generating an ensemble PPI network, EC-BNMF can automatically optimize the values of weights such that the ensemble algorithm can deliver better results. Experimental results on four PPI networks of Saccharomyces cerevisiae well verify the effectiveness of EC-BNMF in detecting protein complexes. EC-BNMF provides an effective way to integrate different clustering results for more accurate and reliable complex detection. Furthermore, EC-BNMF has a high degree of flexibility in the choice of base clustering results. It can be coupled with existing clustering methods to identify protein complexes.

Region-of-interest extraction in low depth of field images using ensemble clustering and difference of Gaussian approaches

In this paper, a two-stage unsupervised segmentation approach based on ensemble clustering is proposed to extract the focused regions from low depth-of-field (DOF) images. The first stage is to cluster image blocks in a joint contrast-energy feature space into three constituent groups. To achieve this, we make use of a normal mixture-based model along with standard expectation-maximization (EM) algorithm at two consecutive levels of block size. To avoid the common problem of local optima experienced in many models, an ensemble EM clustering algorithm is proposed. As a result, relevant blocks closely conforming to image objects are extracted. In stage two, a binary saliency map is constructed from the relevant blocks at the pixel level, which is based on difference of Gaussian (DOG) and binarization methods. Then, a set of morphological operations is employed to create the region-of-interest (ROI) from the map. Experimental results demonstrate that the proposed approach achieves an F-measure of 91.3% and is computationally 3 times faster than the existing state-of-the-art approach.

Rough subspace-based clustering ensemble for categorical data

Clustering categorical data arising as an important problem of data mining has recently attracted much attention. In this paper, the problem of unsupervised dimensionality reduction for categorical data is first studied. Based on the theory of rough sets, the attributes of categorical data are decomposed into a number of rough subspaces. A novel clustering ensemble algorithm based on rough subspaces is then proposed to deal with categorical data. The algorithm employs some of rough subspaces with high quality to cluster the data and yields a robust and stable solution by exploiting the resulting partitions. We also introduce a cluster index to evaluate the solution of clustering algorithm for categorical data. Experimental results for selected UCI data sets show that the proposed method produces better results than those obtained by other methods when being evaluated in terms of cluster validity indexes.

Clustering Ensemble: A Multiobjective Genetic Algorithm based Approach

Abstract Clustering ensemble refers to the problem of obtaining a final clustering of some data set from a set of input clustering solutions. In this article, the clustering ensemble problem has been modeled as a multiobjective optimization problem and a multiobjective evolutionary algorithm has been used for this purpose. The proposed multiobjective evolutionary clustering ensemble algorithm (MOECEA) evolves a clustering solution from the input clusterings by optimizing two criteria simultaneously. The first objective is to maximize the similarity of the resultant clustering with all the input clusterings, where the similarity between two clustering solutions is computed using adjusted Rand index. The second criteria is to minimize the standard deviation among the similarity scores in order to prevent the evolved clustering solution to be very similar with one of the input clusterings and very dissimilar with the others. The performance of the proposed algorithm has been compared with that of other well-known existing cluster ensemble algorithms for a number of artificial and real-life data sets.

Consensus Function Based on Matrix Factorization

Clustering ensemble has been known as an effective method to improve the robustness and stability of clustering analysis. Clustering ensemble solves the problem in two steps:firstly,generating a large set of clustering partitions based on the clustering algorithms;secondly,combining them using a consensus function to get the final clustering result. The key technology of clustering ensemble is the proper consensus function. Recent research proposed using the matrix factorization to solve clustering ensemble. In this paper, we firstly analyze some traditional matrix factorization algorithms; secondly, we propose a new consensus function using binary nonnegative matrix factorization (BMF) and give the optimization algorithm of BMF; lastly, we propose the new framework of clustering ensemble algorithm and give some experiments on UCI Machine Learning Repository. The experiments show that the new algorithm is effective and clustering performance could be significantly improved.

A New Unsupervised Feature Ranking Method for Gene Expression Data Based on Consensus Affinity

Feature selection is widely established as one of the fundamental computational techniques in mining microarray data. Due to the lack of categorized information in practice, unsupervised feature selection is more practically important but correspondingly more difficult. Motivated by the cluster ensemble techniques, which combine multiple clustering solutions into a consensus solution of higher accuracy and stability, recent efforts in unsupervised feature selection proposed to use these consensus solutions as oracles. However,these methods are dependent on both the particular cluster ensemble algorithm used and the knowledge of the true cluster number. These methods will be unsuitable when the true cluster number is not available, which is common in practice. In view of the above problems, a new unsupervised feature ranking method is proposed to evaluate the importance of the features based on consensus affinity. Different from previous works, our method compares the corresponding affinity of each feature between a pair of instances based on the consensus matrix of clustering solutions. As a result, our method alleviates the need to know the true number of clusters and the dependence on particular cluster ensemble approaches as in previous works. Experiments on real gene expression data sets demonstrate significant improvement of the feature ranking results when compared to several state-of-the-art techniques.

Projective clustering ensembles

A considerable amount of work has been done in data clustering research during the last four decades, and a myriad of methods has been proposed focusing on different data types, proximity functions, cluster representation models, and cluster presentation. However, clustering remains a challenging problem due to its ill-posed nature: it is well known that off-the-shelf clustering methods may discover different patterns in a given set of data, mainly because every clustering algorithm has its own bias resulting from the optimization of different criteria. This bias becomes even more important as in almost all real-world applications, data is inherently high-dimensional and multiple clustering solutions might be available for the same data collection. In this respect, the problems of projective clustering and clustering ensembles have been recently defined to deal with the high dimensionality and multiple clusterings issues, respectively. Nevertheless, despite such two issues can often be encountered together, existing approaches to the two problems have been developed independently of each other. In our earlier work Gullo et al. (Proceedings of the international conference on data mining (ICDM), 2009a) we introduced a novel clustering problem, called projective clustering ensembles (PCE): given a set (ensemble) of projective clustering solutions, the goal is to derive a projective consensus clustering, i.e., a projective clustering that complies with the information on object-to-cluster and the feature-to-cluster assignments given in the ensemble. In this paper, we enhance our previous study and provide theoretical and experimental insights into the PCE problem. PCE is formalized as an optimization problem and is designed to satisfy desirable requirements on independence from the specific clustering ensemble algorithm, ability to handle hard as well as soft data clustering, and different feature weightings. Two PCE formulations are defined: a two-objective optimization problem, in which the two objective functions respectively account for the object- and feature-based representations of the solutions in the ensemble, and a single-objective optimization problem, in which the object- and feature-based representations are embedded into a single function to measure the distance error between the projective consensus clustering and the projective ensemble. The significance of the proposed methods for solving the PCE problem has been shown through an extensive experimental evaluation based on several datasets and comparatively with projective clustering and clustering ensemble baselines.

A novel fuzzy C-means algorithm to generate diverse and desirable cluster solutions used by genetic-based clustering ensemble algorithms

One of the most significant discussions in the field of machine learning today is on the clustering ensemble. The clustering ensemble combines multiple partitions generated by different clustering algorithms into a single clustering solution. Genetic algorithms are known for their high ability to solve optimization problems, especially the problem of the clustering ensemble. To date, despite the major contributions to find consensus cluster partitions with application of genetic algorithms, there has been little discussion on population initialization through generative mechanisms in genetic-based clustering ensemble algorithms as well as the production of cluster partitions with favorable fitness values in first phase clustering ensembles. In this paper, a threshold fuzzy C-means algorithm, named TFCM, is proposed to solve the problem of diversity of clustering, one of the most common problems in clustering ensembles. Moreover, TFCM is able to increase the fitness of cluster partitions, such that it improves performance of genetic-based clustering ensemble algorithms. The fitness average of cluster partitions generated by TFCM are evaluated by three different objective functions and compared against other clustering algorithms. In this paper, a simple genetic-based clustering ensemble algorithm, named SGCE, is proposed, in which cluster partitions generated by the TFCM and other clustering algorithms are used as the initial population used by the SGCE. The performance of the SGCE is evaluated and compared based on the different initial populations used. The experimental results based on eleven real world datasets demonstrate that TFCM improves the fitness of cluster partitions and that the performance of the SGCE is enhanced using initial populations generated by the TFCM.

Regionalization of Rainfall Using RCDA Cluster Ensemble Algorithm in India

The magnitude and frequency of precipitation is of great significance in the field of hydrologic and hydraulic design and has wide applications in varied areas. However, the availability of precipitation data is limited to a few areas, where the rain gauges are successfully and efficiently installed. The magnitude and frequency of precipitation in ungauged sites can be assessed by grouping areas with similar characteristics. The procedure of grouping of areas having similar behaviour is termed as Regionalization. In this paper, RCDA cluster ensemble algorithm is employed to identify the homogeneous regions of rainfall in India. Cluster ensemble methods are commonly used to enhance the quality of clustering by combining multiple clustering schemes to produce a more robust scheme delivering similar homogeneous regions. The goal is to identify, analyse and describe hydrologically similar regions using RCDA cluster ensemble algorithm. RCDA cluster ensemble algorithm, which is based on discriminant analysis. The algorithm takes H base clustering schemes each with K clusters, obtained by any clustering method, as input and constructs discriminant function for each one of them. Subsequently, all the data tuples are predicted using H discriminant functions for cluster membership. Tuples with consistent predictions are assigned to the clusters, while tuples with inconsistent predictions are analyzed further and either assigned to clusters or declared as noise. RCDA algorithm has been compared with Best of K-means and Clue cluster ensemble of R software using traditional clustering quality measures. Further, domain knowledge based comparison has also been performed. All the results are encouraging and indicate better regionalization of the rainfall in different parts of India.

Regionalization of River Basins Using Cluster Ensemble

In the wake of global water scarcity, forecasting of water quantity and quality, regionalization of river basins has attracted serious attention of the hydrology researchers. It has become an important area of research to enhance the quality of prediction of yield in river basins. In this paper, we analyzed the data of Godavari basin, and regionalize it using a cluster ensemble method. Cluster Ensemble methods are commonly used to enhance the quality of clustering by combining multiple clustering schemes to produce a more robust scheme delivering similar homogeneous basins. The goal is to identify, analyse and describe hydrologically similar catchments using cluster analysis. Clustering has been done using RCDA cluster ensemble algorithm, which is based on discriminant analysis. The algorithm takes H base clustering schemes each with K clusters, obtained by any clustering method, as input and constructs discriminant function for each one of them. Subsequently, all the data tuples are predicted using H discriminant functions for cluster membership. Tuples with consistent predictions are assigned to the clusters, while tuples with inconsistent predictions are analyzed further and either assigned to clusters or declared as noise. Clustering results of RCDA algorithm have been compared with Best of k-means and Clue cluster ensemble of R software using traditional clustering quality measures. Further, domain knowledge based comparison has also been performed. All the results are encouraging and indicate better regionalization of the Godavari basin data.

Ensemble of clustering algorithms for large datasets

The ensemble clustering algorithm ECCA (Ensemble of Combined Clustering Algorithms) for processing large datasets is proposed and theoretically substantiated. Results of an experimental study of the algorithm on simulated and real data proving its effectiveness are presented

Weighted association based methods for the combination of heterogeneous partitions

Co-association matrix has been a useful tool in many clustering ensemble techniques as a similarity measure between objects. In this paper, we introduce the weighted-association matrix, which is more expressive than the traditional co-association as a similarity measure, in the sense that it integrates information from the set of partitions in the clustering ensemble as well as from the original data of object representations. The weighted-association matrix is the core of the two main contributions of this paper: a natural extension of the well-known evidence accumulation cluster ensemble method by using the weighted-association matrix and a kernel based clustering ensemble method that uses a new data representation. These methods are compared with simple clustering algorithms as well as with other clustering ensemble algorithms on several datasets. The obtained results ratify the accuracy of the proposed algorithms.