Cluster Ensemble Model Research Articles

Fuzzy self-consistent clustering ensemble

The subject of clustering ensemble (CE) has emerged as a pivotal area of exploration within unsupervised learning, primarily due to its remarkable capacity to integrate multiple base clusterings. However, the conventional CE framework exhibits limitations in effectively handling fuzzy relations and ensuring robustness. This study introduces an innovative and robust fuzzy self-consistent clustering ensemble (FSCE) model. Departing from customary approaches to processing base clustering outcomes, the FSCE model considers the scalable dummy variable representation of base clustering results as a novel feature attributes intrinsic to the original dataset. Subsequently, a γ-fuzzy operator ε+ is formulated, enabling the adjustment of coupling strength contingent upon the uncertainties inherent in practical problems. Leveraging this operator, the model reappraises the fuzzy relationships among objects, thereby engendering a corresponding relation matrix. This matrix profoundly enhances the model's capacity to contend with uncertain relationships. The resultant relation matrix assumes a pivotal role as the primary input in the consensus function, culminating in the derivation of the ensemble outcome. Moreover, the FSCE model deviates from the conventional CE paradigm by introducing a reallocation strategy for fuzzy objects within ensemble outcome. Through comprehensive experimentation, we substantiate that the introduction of γ-fuzzy operator ε+ offers a viable novel approach to enhancing both the overall performance and the fuzzy interpretability of the CE model. The FSCE model distinctly excels in consensus formation and robustness when juxtaposed against eight archetypal and state-of-the-art clustering models.

Rank Based Iterative Clustering (RBIC) for indoor localization

In the current era, technological advancements have resulted in a spike in various sensor data-intensive applications including indoor localization services. WiFi fingerprint-based localization for large indoor areas is primarily contingent on laborious site surveys. To ease the flow of research, grossly labeled data collected using a crowd-sourced method could be clustered. However, for a real-time localization approach, the challenge is to generate the clusters containing data samples that are really physically close. In this paper, a two-phase semi-supervised localization approach has been proposed, which is general enough to be applied to indoor localization datasets. In the offline phase, a Rank-Based Iterative Clustering (RBIC) algorithm has been proposed that generates a clustered dataset with a negligible chance of containing physically apart location points within a common cluster. RBIC can be viewed as a clustering ensemble model. Different clustering algorithms are selected as baseline algorithms and assigned unique ranks depending on well-known clustering scores to be fed as input to RBIC. In the online phase, the users’ location is estimated using machine learning (ML) classifiers based on the dynamic received signal strength indicator (RSSI) vector received through its handheld device transceiver. The system is evaluated with three benchmark datasets for WiFi based indoor localization. For the first dataset, JUIndoorLoc, 94% to 99% localization accuracy is achieved for individual supervised classifiers. For the second and third datasets, the ranges of obtained localization accuracies are 96% to 99% and 95% to 98% respectively.

EEG-based driver states discrimination by noise fraction analysis and novel clustering algorithm.

Driver states are reported as one of the principal factors in driving safety. Distinguishing the driving driver state based on the artifact-free electroencephalogram (EEG) signal is an effective means, but redundant information and noise will inevitably reduce the signal-to-noise ratio of the EEG signal. This study proposes a method to automatically remove electrooculography (EOG) artifacts by noise fraction analysis. Specifically, multi-channel EEG recordings are collected after the driver experiences a long time driving and after a certain period of rest respectively. Noise fraction analysis is then applied to remove EOG artifacts by separating the multichannel EEG into components by optimizing the signal-to-noise quotient. The representation of data characteristics of the EEG after denoising is found in the Fisher ratio space. Additionally, a novel clustering algorithm is designed to identify denoising EEG by combining cluster ensemble and probability mixture model (CEPM). The EEG mapping plot is used to illustrate the effectiveness and efficiency of noise fraction analysis on the denoising of EEG signals. Adjusted rand index (ARI) and accuracy (ACC) are used to demonstrate clustering performance and precision. The results showed that the noise artifacts in the EEG were removed and the clustering accuracy of all participants was above 90%, resulting in a high driver fatigue recognition rate.

Anomaly Detection and Classification in Streaming PMU Data in Smart Grids

The invention of Phasor Measurement Units (PMUs) produce synchronized phasor measurements with high resolution real time monitoring and control of power system in smart grids that make possible. PMUs are used in transmitting data to Phasor Data Concentrators (PDC) placed in control centers for monitoring purpose. A primary concern of system operators in control centers is maintaining safe and efficient operation of the power grid. This can be achieved by continuous monitoring of the PMU data that contains both normal and abnormal data. The normal data indicates the normal behavior of the grid whereas the abnormal data indicates fault or abnormal conditions in power grid. As a result, detecting anomalies/abnormal conditions in the fast flowing PMU data that reflects the status of the power system is critical. A novel methodology for detecting and categorizing abnormalities in streaming PMU data is presented in this paper. The proposed method consists of three modules namely, offline Gaussian Mixture Model (GMM), online GMM for identifying anomalies and clustering ensemble model for classifying the anomalies. The significant features of the proposed method are detecting anomalies while taking into account of multivariate nature of the PMU dataset, adapting to concept drift in the flowing PMU data without retraining the existing model unnecessarily and classifying the anomalies. The proposed model is implemented in Python and the testing results prove that the proposed model is well suited for detection and classification of anomalies on the fly.

Geometric consistent fuzzy cluster ensemble with membership reconstruction for image segmentation

In recent years, to handle the ubiquitous uncertainty and unknown noise, different fuzzy clustering methods have been introduced to solve image segmentation problem for various applications, such as natural image, satellite image, multichannel (typically remote sensing) image. In fact, it is hard to know which one is the best for specified task. Cluster ensemble method is proposed to solve the problem of choosing a particular fuzzy clustering algorithm, or a special image processing before clustering, that best suits the given image segmentation task. In fuzzy cluster ensemble, membership vectors generated from different fuzzy clustering methods are merged into one vector as an input object, which is also a combination of data partitions. However, this kind of input object may lose detail information from original target image and may cause inaccurate edges in segmentation results. Moreover, by means of treating it as a new representation of original data, the membership vector of fuzzy cluster ensemble should be intuitively geometric consistent with the original target image. In this paper, by holding this view, we develop a geometric consistent fuzzy cluster ensemble model for spatial data, which involves a constraint between the membership and its reconstruction, to improve the clustering performance on image segmentation. In the proposed model, a pre-determined gradient-preserving weight is used in the membership reconstruction item to make the membership matrix be geometric consistent with the original target image. A semi-implicit optimization iterative algorithm is adopted to solve the proposed geometric consistent model. Experimental results demonstrate the effectiveness of proposed model in synthetic and real-world image segmentation problems over several state-of-the-art methods.

A novel self-directed learning framework for cluster ensemble

This study looked into the problems that occur when a clustering or cluster ensemble model with unsupervised or semi-supervised learning is used in a real-world setting. However, validating the obtained results is a challenging task. Therefore, we propose a self-directed learning (SDL) framework for cluster ensemble, which improves the traditional ensemble framework by assisting the consensus function in achieving the highest assessment of clustering performance. The SDL is built on two models: predicting test set labels (PTL) and detecting best results (DBR). The PTL model assists in predicting the test set labels; the PTL is based on consistently ensembling the outcomes until it obtains satisfactory results. The DBR identifies the correct answers for each data object when a single model provides several outcomes for a single dataset. To highlight the power of our proposed framework, we consider multiple performance measurements; one of them is termed as correction ratio (CR). We compare SDL with several cluster ensemble models. The results show that the proposed framework outperforms other models with 18% accuracy on average and achieves similar results for other performance indicators.

A Generative Clustering Ensemble Model and Its Application in IoT Data Analysis

Data analysis is the foundation of Internet of Things (IoT) based applications, and clustering is an effective technology of data analysis. Clustering ensemble integrates multiple base clustering results to obtain a consensus result and thus improves the clustering performance in stability and robustness. However, it is difficult for existing clustering ensemble algorithms to achieve a satisfying ensemble result, when the base clustering results are unreliable. Concerning this problem, we develop a new clustering ensemble model in this paper, which has several advantages compared with traditional algorithms: (i) structure information about the data is effectively extracted from the base clusterings; (ii) data characteristics and structure information are integrated in an elegant fashion, in the production of the consensus clustering result; and (iii) our model has the generative ability that makes the model achieve outstanding performance when training samples are insufficient. In our model, the structural information is extracted by explicating the coupling relationships between base clusterings and between samples in clustering members. Then, data characteristics and structure information are combined in a generative graph representation learning framework. And the objectives of representation learning and consensus clustering are integrated into a unified optimization model, in which the prior distribution of the data is approximated by a Gaussian mixture model (GMM). Extensive experiments are conducted on multiple IoT datasets; the results prove that our model not only performs better than the conventional clustering ensemble algorithms but also outperforms the state-of-the-art deep clustering methods.

Markov clustering ensemble

Clustering ensemble is an unsupervised ensemble learning method that is very important in machine learning, since it integrates multiple weak base clustering results to produce a strong consistency result. This paper proposes the Markov clustering ensemble (MCE) model to solve the weak stability and robustness of soft clustering ensemble. First, the base clustering algorithms are regarded as new features of the original datasets. Then, the results of the base clustering algorithms are the values of these features, which can break through the framework of consensus cluster ensemble. Second, as the base clustering results are discrete data, the maximum information coefficient is applied to measure their similarity. Accordingly, a graph-based cluster ensemble model can be constructed with row vectors as vertices and the similarity between row vectors as edges. Then, the Markov process can be applied to infer the graph-based cluster ensemble model; therefore, the MCE algorithm is designed according to the inference. To test the performance of the MCE algorithm, clustering algorithms and clustering ensemble algorithms are used to conduct comparative experiments on ten datasets. The experimental results show that the MCE algorithm outperforms the other algorithms in terms of accuracy and purity.

A novel clustering ensemble model based on granular computing

Clustering ensemble is one of the popular methods in the field of data mining for discovering hidden patterns in unlabeled datasets. Researches have shown that selecting base clustering results with certain differences and high quality to participate in the fusion process can improve the quality of the final result. However, the existing inherent characteristics of uncertainty, ambiguity, and overlap of the base clustering results make the selection of the base clustering members more difficult. The accuracy of the final results is easily disturbed by low-quality base clustering members. From the perspective of granular computing, a novel clustering ensemble model is proposed. The similarity among ensemble members is measured by granularity distance, so the quality of the base clustering results is ensured meanwhile the difference among them is enlarged, which is beneficial to improve the accuracy of the final result. According to the dividing ability of knowledge granularity, the method of elements generation for the co-association matrix is optimized and improved. The results obtained from the improved sample similarity measurement are more consistent with the structure of the real data. Compared with the traditional single clustering algorithm and some popular clustering ensemble methods, experiments show that the proposed model improves the quality of the final clustering result and has good expandability.

Clustering Ensemble Model Based on Self-Organizing Map Network.

This paper proposes a clustering ensemble method that introduces cascade structure into the self-organizing map (SOM) to solve the problem of the poor performance of a single clusterer. Cascaded SOM is an extension of classical SOM combined with the cascaded structure. The method combines the outputs of multiple SOM networks in a cascaded manner using them as an input to another SOM network. It also utilizes the characteristic of high-dimensional data insensitivity to changes in the values of a small number of dimensions to achieve the effect of ignoring part of the SOM network error output. Since the initial parameters of the SOM network and the sample training order are randomly generated, the model does not need to provide different training samples for each SOM network to generate a differentiated SOM clusterer. After testing on several classical datasets, the experimental results show that the model can effectively improve the accuracy of pattern recognition by 4%∼10%.

Similarity Analysis of Small- and Medium-Sized Watersheds Based on Clustering Ensemble Model

Similarity analysis of small- and medium-sized watersheds mainly depends on manual work, and there is no complete automated analysis method. In order to solve this problem, we propose a similarity analysis method based on clustering ensemble model. First, the iterative clustering ensemble construction algorithm with weighted random sampling (WRS-CCE) is proposed to get great clustering collectives. Then, we combine spectral clustering with the fuzzy C-means method to design a consensus function for small- and medium-sized watershed data sets. Finally, the similarity analysis of small- and medium-sized watersheds is carried out according to the clustering results. Experiments show that the proposed clustering ensemble model can effectively find more potential similar watersheds and can output the similarity of these watersheds.

Constraint projections for semi‐supervised spectral clustering ensemble

SummaryCluster ensemble combines multiple base clustering results in a suitable way to improve the accuracy of the clustering result. In the conventional cluster ensemble frameworks, pairwise constraints and constraint projections have not been used together, and spectral clustering algorithm is rarely adopted to serve as the consensus function. In this paper, we design a constraint projections for semi‐supervised spectral clustering ensemble (CPSSSCE) model. It takes advantages of spectral clustering algorithm and executes semi‐supervised learning twice. Compared to traditional cluster ensemble approaches, CPSSSCE is characterized by several properties. First, the original data are transformed to lower‐dimensional representations by constraint projection before base clustering. Second, a similarity matrix is constructed using the base clustering results and modified using pairwise constraints. Third, the spectral clustering algorithm is applied to process the similarity matrix to obtain a consensus cluster result. Extensive experiments on standard University of California Irvine Machine Learning Repository (UCI) and Microsoft datasets demonstrated that the CPSSSCE is superior to other cluster ensemble algorithms including a semi‐supervised spectral clustering ensemble.

Day-Ahead Hourly Forecasting of Solar Generation Based on Cluster Analysis and Ensemble Model

Accurate solar generation prediction is of great significance for grid dispatching and operation of photovoltaic power plants. In this paper, we propose a novel solar generation forecasting method based on cluster analysis and ensemble model. Two common ways to improve prediction accuracy are adopted. We first conduct cluster analysis based on solar generation to obtain a weather regime, which improves the computational efficiency and avoids the difficulty in selecting weather variables to participate in the clustering process. Then random forests with different parameters is established for different weather regimes, which is used as component models in the followed ensemble model. Finally, we weighted the predictions from different weather regimes to get the final results. To avoid manual design weights, ridge regression is used to calculate weights for each weather regime automatically. A large number of experiments have been carried out on freely available data sets to verify the performance of the proposed method. The experimental results show that our method predicts solar generation more accurately, which has broad prospects in practical application.

Nonnegative matrix factorization for clustering ensemble based on dark knowledge

Traditional cluster ensemble (CE) methods use labels produced by base learning algorithms to obtain an ensemble result. These base learning algorithms can also obtain other information, such as parameter, covariance, or probability data, which is called dark knowledge. In this paper, we propose a method for integrating dark knowledge, which is usually ignored, into the ensemble learning process. This provides more information about the base clustering. We apply nonnegative matrix factorization (NMF) to the clustering ensemble model based on dark knowledge. First, different base clustering results are obtained by using various clustering configurations, before dark knowledge of every base clustering algorithm is extracted. NMF is then applied to the dark knowledge to obtain integrated results. Experimental results show that the method outperforms other clustering ensemble techniques.

Analysis of Hotel Power Consumption Mode Based on Improved Clustering Ensemble Method

In this paper, a hotel building in Shanghai's Huangpu District was studied to analyze its electricity consumption behavior using data analysis methods. For the problem of poor scalability of single clustering methods, a cluster ensemble model using optimizing clustering methods was proposed in this paper. A comprehensive clustering evaluation index was proposed to evaluate different clustering methods. Then good methods were filtered out and voting method was used to fuse results. The results showed that this method has better effectiveness. The improved cluster ensemble method was used in load curves to extract power patterns, analyze power constitution and characteristics.

Spectral clustering ensemble model based on distance decision

Spectral clustering (SC) is a kind of clustering method which shows superior performance in recent years; however, the method is sensitive to the initial scale parameter. In order to improve the accuracy and stability of clustering, this Letter proposes a novel approach of SC ensemble model based on distance decision. First of all, the model performs several times clustering for data samples by SC, and corresponding joint Laplacian matrix is obtained. Then, a new method of distance decision is proposed, the joint Laplacian matrix is transformed into a cumulative distance matrix using the fuzzy theory. Finally, the distance matrix is introduced into the density peaks (DP) algorithm, and the final results of clustering are obtained by the improved DP algorithm for clustering ensemble. The experimental results show that the clustering ensemble model proposed in this Letter is more effective than other classical clustering ensemble model on the 12 data sets in University of California Irvine machine learning database.

Cluster Ensemble with Link-Based Approach for Botnet Detection

Botnet detection is one of the most imminent tasks for cyber security. Among popular botnet countermeasures, an intrusion detection system is the prominent mechanism. In the past, packet-based intrusion detection systems were popular. However, flow-based intrusion detection systems have been preferred in recent years due to their ability to adapt to modern high-speed networks. A collection of flows from an enterprise network usually contains both botnet traffic and normal traffic. To classify this traffic, supervised machine learning algorithms, i.e., classifications, have been applied and achieved a high accuracy. In an effort to improve the ability of intrusion detection systems against botnets, some studies have suggested partitioning flows into clusters before applying the classifications and this step could significantly reduce the complexity of a flow set. However, the instability of individual clustering algorithms is still a constraint for botnet detection.To overcome this bottleneck, we propose a novel method that combines individual partitions to become a strong learner through the use of a link-based algorithm. Our experiments show that our cluster ensemble model outperforms existing botnet detection mechanisms with a high reliability. We also determine the balance between accuracy and computer resources for botnet detection, and thereby propose a range for the maximum duration time of flows in botnet research.

A new multi-objective cluster ensemble based on modularity maximization

Conventional clustering algorithms utilize only one single criterion that may not conform to diverse shapes of the underlying clusters. But in this paper, we hire two important criteria and propose a new multi-objective cluster ensemble model to empower finding clusters of different types. The first criteria is the well-known sum of squared error. The second criterion is modularity which is originally a measure of evaluating communities in social networks. We maximize modularity as a consensus function of cluster ensemble. In order to further improvement, we also modify Non Dominant Sorting Genetic Algorithm (NSGAII) and propose a specialized crossover operator for it. Experimental results over seven UCI real data sets show that the proposed method significantly outperforms other clustering methods.

A Heterogeneous Cluster Ensemble Model for Improving the Stability of Fuzzy Cluster Analysis

Cluster analysis is an important exploratory tool which reveals underlying structures in data and organizes them in clusters (groups) based on their similarities. The fuzzy approach to the clustering problem involves the concept of partial memberships of the instances in the clusters, increasing the flexibility and enhancing the semantics of the generated clusters. Several fuzzy clustering algorithms have been devised like fuzzy c-means (FCM), Gustafson-Kessel, Gath-Geva, kernel-based FCM etc. Although these algorithms do have a myriad of successful applications, each of them has its stability drawbacks related to several factors including the shape and density of clusters, the presence of noise or outliers and the choices about the algorithm's parameters and cluster center initialization. In this paper we are providing a heterogeneous cluster ensemble approach to improve the stability of fuzzy cluster analysis. The key idea of our methodology is the application of different fuzzy clustering algorithms on the datasets obtaining multiple partitions, which in the later stage will be fused into the final consensus matrix. Finally we have experimentally evaluated and compared the accuracy of this methodology.

Hierarchical cluster ensemble model based on knowledge granulation

Cluster ensemble has been shown to be very effective in unsupervised classification learning by generating a large pool of different clustering solutions and then combining them into a final decision. However, the task of it becomes more difficult due to the inherent complexities among base cluster results, such as uncertainty, vagueness and overlapping. Granular computing is one of the fastest growing information-processing paradigms in the domain of computational intelligence and human-centric systems. As the core part of granular computing, the rough set theory dealing with inexact, uncertain, or vague information, has been widely applied in machine learning and knowledge discovery related areas in recent years. From these perspectives, in this paper, a hierarchical cluster ensemble model based on knowledge granulation is proposed with the attempt to provide a new way to deal with the cluster ensemble problem together with ensemble learning application of the knowledge granulation. A novel rough distance is introduced to measure the dissimilarity between base partitions and the notion of knowledge granulation is improved to measure the agglomeration degree of a given granule. Furthermore, a novel objective function for cluster ensembles is defined and the corresponding inferences are made. A hierarchical cluster ensemble algorithm based on knowledge granulation is designed. Experimental results on real-world data sets demonstrate the effectiveness for better cluster ensemble of the proposed method.