Ensemble Clustering Research Articles

Creating granular climate zones for future-proof building design in the UK

Climate zones play an important role in promoting climate responsive building design and implementing climate-specific prescriptions in national building standards and regulations. The existing studies on climate zoning are subject to several limitations, i.e. the incapability of distinguishing microclimates and the lack of consideration of climate change. In this research, we propose a two-tiered ensemble clustering method for the identification of granular climate zones using the projections of future climate. The first tier identifies primary climate zones using a combination of climatic features and geographical locations, whereas the second tier identifies distinct local variations within each primary climate zone using the temperature related features. The proposed ensemble clustering model is applied to the UK to create a mapping of granular climate zones for future proofing building design. The method identified 14 distinct primary zones and distinguished microclimates at a range of scales from large urban areas, such as the Greater London Area, to national parks, such as Dartmoor and the Pennines. The identified mapping resolves two major obstacles in the creation and usage of weather data for building performance assessment in the UK, i.e. the lack of guidance for selecting weather files, and the absence of scientific rationale for representing the UK climate using the current 14 locations.

Towards improving community detection in complex networks using influential nodes

Abstract Complex network analysis is inspired by empirical studies of real-world networks such as computer networks, technology networks and social networks. The analysis of community structure in complex networks is understood as an important issue by the research society. A community is a set of nodes in a network where the density of connections is high. The insight in the literature shows many approaches to identify influential nodes, but these approaches only lead to finding community centres. Meanwhile, clustering techniques are effectively used for community detection, where they can reveal group structure and hidden connections by considering topological and demographic information. This article presents an ensemble clustering algorithm based on influential nodes to improve community detection in complex networks. Considering different characteristics of the network, the proposed method seeks to discover common interests between users and their behaviours to identify the most suitable communities. First, a set of influential nodes are identified as community centres. Then, these centres are considered as cluster centres. After that, primary clusters are created based on the determined centres. Finally, the primary clusters are reclustered to form the final clusters. Here, the final clusters are considered as communities of users in the network. The simulation has been performed on real-world networks and the results confirm the effectiveness of the proposed method. Specifically, the communities identified by the proposed method are 2.1% better than the best existing state-of-the-art method in terms of modularity. Keywords: complex network; community detection; influential nodes; ensemble clustering.

Dual-level clustering ensemble algorithm with three consensus strategies

Clustering ensemble (CE), renowned for its robust and potent consensus capability, has garnered significant attention from scholars in recent years and has achieved numerous noteworthy breakthroughs. Nevertheless, three key issues persist: (1) the majority of CE selection strategies rely on preset parameters or empirical knowledge as a premise, lacking adaptive selectivity; (2) the construction of co-association matrix is excessively one-sided; (3) the CE method lacks a more macro perspective to reconcile the conflicts among different consensus results. To address these aforementioned problems, a dual-level clustering ensemble algorithm with three consensus strategies is proposed. Firstly, a backward clustering ensemble selection framework is devised, and its built-in selection strategy can adaptively eliminate redundant members. Then, at the base clustering consensus level, taking into account the interplay between actual spatial location information and the co-occurrence frequency, two modified relation matrices are reconstructed, resulting in the development of two consensus methods with different modes. Additionally, at the CE consensus level with a broader perspective, an adjustable Dempster–Shafer evidence theory is developed as the third consensus method in present algorithm to dynamically fuse multiple ensemble results. Experimental results demonstrate that compared to seven other state-of-the-art and typical CE algorithms, the proposed algorithm exhibits exceptional consensus ability and robustness.

Fuzzy self-consistent clustering ensemble

The subject of clustering ensemble (CE) has emerged as a pivotal area of exploration within unsupervised learning, primarily due to its remarkable capacity to integrate multiple base clusterings. However, the conventional CE framework exhibits limitations in effectively handling fuzzy relations and ensuring robustness. This study introduces an innovative and robust fuzzy self-consistent clustering ensemble (FSCE) model. Departing from customary approaches to processing base clustering outcomes, the FSCE model considers the scalable dummy variable representation of base clustering results as a novel feature attributes intrinsic to the original dataset. Subsequently, a γ-fuzzy operator ε+ is formulated, enabling the adjustment of coupling strength contingent upon the uncertainties inherent in practical problems. Leveraging this operator, the model reappraises the fuzzy relationships among objects, thereby engendering a corresponding relation matrix. This matrix profoundly enhances the model's capacity to contend with uncertain relationships. The resultant relation matrix assumes a pivotal role as the primary input in the consensus function, culminating in the derivation of the ensemble outcome. Moreover, the FSCE model deviates from the conventional CE paradigm by introducing a reallocation strategy for fuzzy objects within ensemble outcome. Through comprehensive experimentation, we substantiate that the introduction of γ-fuzzy operator ε+ offers a viable novel approach to enhancing both the overall performance and the fuzzy interpretability of the CE model. The FSCE model distinctly excels in consensus formation and robustness when juxtaposed against eight archetypal and state-of-the-art clustering models.

Parameter-Free Robust Ensemble Framework of Fuzzy Clustering

The ensemble of fuzzy clustering can address the problems presented in the base clustering, such as fluctuations in results due to random initialization and performance degradation due to outliers. However, the performance of fuzzy clustering ensembles is still hampered by some challenges that include misaligned membership matrices, loss of information in the co-similarity matrix, large storage space, unstable ensemble results due to an additional re-clustering, the need for original data information for assistance, etc. To address these issues, we propose a parameter-free robust ensemble framework for fuzzy clustering. After obtaining the set of membership matrices, we cascade these membership matrices and mine the latent spectral matrix of the raw data. Benefiting from this step, we obtain global features of the dataset without knowing the specific data. Then, our framework uses transition matrices to solve the alignment problem, avoiding the storage of large-scale matrices. Most importantly, we introduce a robust weighted mechanism in the optimization model, where each base clustering is adaptively adjusted and the effect of outliers is suppressed by a robust function. In addition, the model yields the results as a membership matrix, which produces the exact partition results directly without any subsequent clustering operations. Finally, since our model is a parameter-free model, the setting of hyper-parameters is avoided and the applicability of the model is improved as well. The effective algorithm of the optimization model is derived and its time complexity and convergence are analyzed. The results of competitive experiments on benchmark data show that the proposed ensemble framework is effective compared to state-of-the-art methods.

Semi-supervised clustering ensemble based on genetic algorithm model

Semi-supervised clustering ensemble based on genetic algorithm model

Proposal of a Visualization System for a Hierarchical Clustering Algorithm: The Visualize Proximity Matrix

Visual data mining is a new and effective strategy for dealing with the growing phenomenon of information overload. There is an urgent need for effective visual data mining because the growth of data sets from different domains and sources has made exploring, managing and analyzing vast volumes of data increasingly difficult. Identifying location-based trends and anomalies in data from a numeric output is challenging. The outputs of data mining procedures are often quite difficult to interpret. There is no human input to data exploration or direct involvement in the data mining process. Thus, it is difficult to analyze, explore, understand and view the data. In data mining, massive data sets are clustered using an iterative process that includes human input but data mining algorithms can miss some knowledge and observations. In addition, many users cannot take the appropriate action to address problems or opportunities because the huge amounts of data prevent them from staying aware of what is happening in and around their environments. Visual data mining can help in dealing with information overload and it is effective in analyzing large complex data sets. Visual data mining assists users in gaining a deep visual understanding of data. In the information age, users need to study and observe vast amounts of data to acquire important knowledge, and thus the need for visual and interactive analytical tools is particularly pressing. Visual cluster analysis has long utilized shaded similarity matrices, and this study investigated how they can be used in clustering visualization. We focused on proposing an agglomerative hierarchical clustering method. Ensemble cluster visualization is also presented for handling large data sets. This study proposes the adoption of a shaded similarity matrix to visually cluster knowledge discovered using data mining. Using the technology acceptance model as the measuring tool, we questioned respondents to evaluate the visualization prototype. The findings demonstrated that the visualization was effective and easy to use, and satisfied users.

Enhancing diversity and robustness of clustering ensemble via reliability weighted measure

Enhancing diversity and robustness of clustering ensemble via reliability weighted measure

Ensemble clustering with low-rank optimal Laplacian matrix learning

The co-association (CA) matrix that describes connection relationship between instances is of importance for ensemble clustering. Existing ensemble clustering methods demonstrate that Laplacian matrix can help to improve the quality of CA matrix and finally produce better clustering results. However, they usually consider the first-order information only and ignore the higher-order information that exists among base clustering results. To address this issue, this paper introduces the theory of higher-order connectivity in ensemble clustering for the first time and further proposes a low-rank optimal Laplacian matrix learning (LROLML) approach for ensemble clustering. Specifically, the proposed LROLML first constructs a set of multi-order Laplacian matrices and then learns the optimal Laplacian matrix from the Laplacian matrix set with different orders. The optimal Laplacian matrix mines more higher-order link information, thereby improving the ensemble performance. We use alternating direction method of multipliers (ADMM) to solve the optimization problem and then apply the average-link hierarchical agglomerative clustering to get the final result. We conducted extensive experiments on 20 popular datasets to validate the proposed LROLML, which is compared with two types of results of basic k-means clustering and seven state-of-the-art ensemble clustering models. The results demonstrate the LROLML is superior to the compared models, showing that the LROLML is promising for ensemble clustering.

An air combat maneuver pattern extraction based on time series segmentation and clustering analysis

Target maneuver recognition is a prerequisite for air combat situation awareness, trajectory prediction, threat assessment and maneuver decision. To get rid of the dependence of the current target maneuver recognition method on empirical criteria and sample data, and automatically and adaptively complete the task of extracting the target maneuver pattern, in this paper, an air combat maneuver pattern extraction based on time series segmentation and clustering analysis is proposed by combining autoencoder, G-G clustering algorithm and the selective ensemble clustering analysis algorithm. Firstly, the autoencoder is used to extract key features of maneuvering trajectory to remove the impacts of redundant variables and reduce the data dimension; Then, taking the time information into account, the segmentation of Maneuver characteristic time series is realized with the improved FSTS-AEGG algorithm, and a large number of maneuver primitives are extracted; Finally, the maneuver primitives are grouped into some categories by using the selective ensemble multiple time series clustering algorithm, which can prove that each class represents a maneuver action. The maneuver pattern extraction method is applied to small scale air combat trajectory and can recognize and correctly partition at least 71.3% of maneuver actions，indicating that the method is effective and satisfies the requirements for engineering accuracy. In addition, this method can provide data support for various target maneuvering recognition methods proposed in the literature, greatly reduce the workload and improve the recognition accuracy.

Consensus clustering with missing labels (ccml): a consensus clustering tool for multi-omics integrative prediction in cohorts with unequal sample coverage.

Multi-omics data integration is a complex and challenging task in biomedical research. Consensus clustering, also known as meta-clustering or cluster ensembles, has become an increasingly popular downstream tool for phenotyping and endotyping using multiple omics and clinical data. However, current consensus clustering methods typically rely on ensembling clustering outputs with similar sample coverages (mathematical replicates), which may not reflect real-world data with varying sample coverages (biological replicates). To address this issue, we propose a new consensus clustering with missing labels (ccml) strategy termed ccml, an R protocol for two-step consensus clustering that can handle unequal missing labels (i.e. multiple predictive labels with different sample coverages). Initially, the regular consensus weights are adjusted (normalized) by sample coverage, then a regular consensus clustering is performed to predict the optimal final cluster. We applied the ccml method to predict molecularly distinct groups based on 9-omics integration in the Karolinska COSMIC cohort, which investigates chronic obstructive pulmonary disease, and 24-omics handprint integrative subgrouping of adult asthma patients of the U-BIOPRED cohort. We propose ccml as a downstream toolkit for multi-omics integration analysis algorithms such as Similarity Network Fusion and robust clustering of clinical data to overcome the limitations posed by missing data, which is inevitable in human cohorts consisting of multiple data modalities. The ccml tool is available in the R language (https://CRAN.R-project.org/package=ccml, https://github.com/pulmonomics-lab/ccml, or https://github.com/ZhoulabCPH/ccml).

Distributed Ensemble Clustering in Networked Multi-Agent Systems

Ensemble clustering, a paradigm that deals with combining the results of multiple clusterings into a single solution, has been widely studied in recent years. The goal of this study is to propose a novel distributed ensemble clustering method that is applicable for use in networked multi-agent systems. The adopted setting supports both object-distributed and feature-distributed clusterings. It is not limited to specific types of algorithms used for obtaining local data labels. The method assumes local processing of local data by the individual agents and neighbor-wise communication of the processed information between the neighboring agents in the network. Using the proposed communication scheme, all agents are able to achieve reliable global results in a fully decentralized way. The network communication design is based on the multi-agent consensus averaging algorithm applied to clustering similarity matrices. It provably results in the fastest convergence to the desired asymptotic values. Several simulation examples illustrate the performance of the proposed distributed solution in different scenarios, including diverse datasets, networks, and applications within the multimedia domain. They show that the obtained performance is very close to that of the corresponding centralized solution.

Weighted ensemble clustering with multivariate randomness and random walk strategy

Ensemble clustering algorithms have made significant progress in recent years due to their excellent performance. However, most of these algorithms face two challenges: one is to focus on the selection of subspaces since there is limited discussion on how to construct a potential metric space, the other is to treat basic clustering equally without fully considering the local connection between clusters when constructing the cooperative association matrix. To solve these issues, we propose a weighted ensemble clustering algorithm with multiple randomness and random walk strategy. We define the free exponential similarity kernel to create a diverse set of random metric spaces coupled with random subspaces and use spectral clustering to generate base clustering. Moreover, we use random walk strategy to discover the local connection between clusters and weight the collaborative association matrix. Finally, the collaborative association matrix uses consensus functions based on hierarchical clustering and meta clustering to obtain clustering results. On this basis, two specific ensemble clustering algorithms WECMR-HC and WECMR-MC are proposed. Theoretical analysis and experimental results demonstrate that our proposed algorithm outperforms existing ensemble algorithms.

Weighted adaptively ensemble clustering method based on fuzzy Co-association matrix

Although many existing co-association (CA) matrix-based ensemble clustering methods have witnessed their superiority in unsupervised learning, but these methods still suffer the following problems: (1) the CA matrix may only focus on strengthening the co-association relations of the pairwise samples in the same cluster and ignoring other information (e.g., inter-cluster association relationship). (2) the CA matrix should be directly used to obtain the ensemble clustering result without using other clustering methods (e.g., spectral clustering). (3). the weights of each base clustering result should be assigned adaptively to reflect the importance of each base clustering result. In order to circumvent the above problems, we propose a novel weighted adaptively ensemble clustering method based on fuzzy co-association matrix (called WEC-FCA) in our study. Concretely, the proposed method WEC-FCA first proposes a novel CA matrix called fuzzy co-association (FCA) matrix to reflect both the co-association relations and inter-cluster association relationships of the pairwise samples in dataset. Then by introducing the rank constraint into the FCA matrix, a novel optimalization framework is invented to obtain the optimal ensemble FCA matrix having the same total number of components as the true number of clusters, so as to be directly used to obtain the ensemble clustering result. Besides, the weights of each base clustering result can be assigned adaptively during the optimization process by using the Shannon entropy. Experimental results on both synthetic and real datasets confirm that the proposed method WEC-FCA keeps at least comparable clustering performance to the state-of-the-art ensemble clustering methods.

Cluster ensemble selection and consensus clustering: A multi-objective optimization approach

Cluster ensembles have emerged as a powerful tool to obtain clusters of data points by combining a library of clustering solutions into a consensus solution. In this paper, we address the cluster ensemble selection problem and design a multi-objective optimization-based solution framework to produce consensus solutions. Given a library of clustering solutions, we first design a preprocessing procedure that measures the agreement of each clustering solution with the other solutions and eliminates the ones that may mislead the process. We then develop a multi-objective optimization algorithm that selects representative clustering solutions from the preprocessed library with respect to size, coverage, and diversity criteria and combines them into a single consensus solution, for which the true number of clusters is assumed to be unknown. We conduct experiments on different benchmark data sets. The results show that our approach yields more accurate consensus solutions compared to full-ensemble and the existing approaches for most data sets. We also present an application on the customer segmentation problem, where our approach is used to segment customers and to find a consensus solution for each segment, simultaneously.

A novel auto-pruned ensemble clustering via SOCP

A novel auto-pruned ensemble clustering via SOCP

LiteWSEC: A Lightweight Framework for Web-Scale Spectral Ensemble Clustering

Spectral Clustering (SC) is an effective clustering method for its excellent performance in partitioning non-linearly distributed data. On the other hand, Ensemble Clustering (EC), a different clustering technology, can promote cluster quality by ensembling the results of base clusterings. In this work, we concentrate on an EC framework that utilizes SC as the base method. Nevertheless, SC suffers from scalability due to its high computational complexity in constructing the Laplacian graph and computing the corresponding eigendecomposition. In the past decades, many efforts have been made to it. However, SC suffers from the scalability issue in processing extensive data, especially in web-scale scenarios. Additionally, EC requires multiple clustering results as the ensemble bases, which further aggravates resource consumption. To address this issue, LiteWSEC, a simple yet efficient Lightweight Framework for Web-scale Spectral Ensemble Clustering, is proposed to cluster web-scale data with limited resource requirements. It adopts the Web-scale Spectral Clustering (WSC) as the base method, which has minimal space overhead without computing overall embedding explicitly. LiteWSEC is highly flexible in the memory requirement, which is adaptive to the available resource. It can partition web-scale data (e.g., <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$n = 8,000~k$</tex-math></inline-formula> ) in an resource-limited host (e.g., memory is restricted to 1 GB). Experiments on real-world, large-scale, and web-scale datasets demonstrate both the efficiency and effectiveness of LiteWSEC over state-of-the-art SC and EC methods.

Soft Subspace Based Ensemble Clustering for Multivariate Time Series Data.

Recently, multivariate time series (MTS) clustering has gained lots of attention. However, state-of-the-art algorithms suffer from two major issues. First, few existing studies consider correlations and redundancies between variables of MTS data. Second, since different clusters usually exist in different intrinsic variables, how to efficiently enhance the performance by mining the intrinsic variables of a cluster is challenging work. To deal with these issues, we first propose a variable-weighted K-medoids clustering algorithm (VWKM) based on the importance of a variable for a cluster. In VWKM, the proposed variable weighting scheme could identify the important variables for a cluster, which can also provide knowledge and experience to related experts. Then, a Reverse nearest neighborhood-based density Peaks approach (RP) is proposed to handle the problem of initialization sensitivity of VWKM. Next, based on VWKM and the density peaks approach, an ensemble Clustering framework (SSEC) is advanced to further enhance the clustering performance. Experimental results on ten MTS datasets show that our method works well on MTS datasets and outperforms the state-of-the-art clustering ensemble approaches.

Fuzzy Ensemble Clustering Based on Self-Coassociation and Prototype Propagation

Fuzzy Ensemble Clustering Based on Self-Coassociation and Prototype Propagation

An improved weighted ensemble clustering based on two-tier uncertainty measurement

Existing locally weighted ensemble clustering algorithms strive to weight each cluster and take into account the differences among all clusters, but they tend to ignore the basic cluster labels. The purpose of this paper is to combine the influence of cluster level and the base clustering level in a unified ensemble clustering framework. A novel two-level weighted ensemble cluster method (TWEC) is proposed, which inserts a global weighting strategy into a local ensemble cluster learning framework. First, the cluster uncertainty based on an entropy criterion is refined by considering the base clustering labels for each cluster. Then, the two-level uncertainty is converted to cluster reliability via improved ensemble-driven cluster validity measure. Finally, two novel consensus functions are developed. Experiments validate the effectiveness of the proposed TWEC framework by comparing it with ten comparison algorithms on fourteen real-world datasets and twenty synthetic datasets. The results show that TWEC framework can improve the robustness and stability of clustering.