Clustering Ensemble Method Research Articles

Semantic Anomaly Detection in Medical Time Series.

The main goal of this project was to define and evaluate a new unsupervised deep learning approach that can differentiate between normal and anomalous intervals of signals like the electrical activity of the heart (ECG). Denoising autoencoders based on recurrent neural networks with gated recurrent units were used for the semantic encoding of such time frames. A subsequent cluster analysis conducted in the code space served as the decision mechanism labelling samples as anomalies or normal intervals, respectively. The cluster ensemble method called cluster-based similarity partitioning proved itself well suited for this task when used in combination with density-based spatial clustering of applications with noise. The best performing system reached an adjusted Rand index of 0.11 on real-world ECG signals labelled by medical experts. This corresponds to a precision and recall regarding the detection task of around 0.72. The new general approach outperformed several state-of-the-art outlier recognition methods and can be applied to all kinds of (medical) time series data. It can serve as a basis for more specific detectors that work in an unsupervised fashion or that are partially guided by medical experts.

Tri-level Robust Clustering Ensemble with Multiple Graph Learning

Clustering ensemble generates a consensus clustering result by integrating multiple weak base clustering results. Although it often provides more robust results compared with single clustering methods, it still suffers from the robustness problem if it does not treat the unreliability of base results carefully. Conventional clustering ensemble methods often use all data for ensemble, while ignoring the noises or outliers on the data. Although some robust clustering ensemble methods are proposed, which extract the noises on the data, they still characterize the robustness in a single level, and thus they cannot comprehensively handle the complicated robustness problem. In this paper, to address this problem, we propose a novel Tri-level Robust Clustering Ensemble (TRCE) method by transforming the clustering ensemble problem to a multiple graph learning problem. Just as its name implies, the proposed method tackles robustness problem in three levels: base clustering level, graph level and instance level. By considering the robustness problem in a more comprehensive way, the proposed TRCE can achieve a more robust consensus clustering result. Experimental results on benchmark datasets also demonstrate it. Our method often outperforms other state-of-the-art clustering ensemble methods. Even compared with the robust ensemble methods, ours also performs better.

Presenting a Proper Ensemble Clustering (EC) Method Based on Hierarchical Methods and Classical Generative Algorithms

Ensemble Clustering (EC) methods became more popular in recent years. In this methods, some primary clustering algorithms are considered to be as inputs and a single cluster is generated to achieve the best results combined with each other. In this paper, we considered three hierarchical methods, which are single-link, average-link, and complete-link as the primary clustering and the results were combined with each other. This combination was done based on correlation matrix. The basic algorithms were combined as binary and triplicate and the results were evaluated as well. the IMDB film dataset were clustered based on existing features. CH, Silhouette and Dunn Index criteria were used to evaluate the results. These criteria evaluate the clustering quality by calculating intra-cluster and inter-cluster distances. CH index had the highest value when all three basic clusters are combined. our method shows that EC can achieve better results and present clusters with higher robustness and accuracy.

A new ensemble clustering method based on fuzzy cmeans clustering while maintaining diversity in ensemble

A new ensemble clustering method based on fuzzy cmeans clustering while maintaining diversity in ensemble

Ensemble clustering using extended fuzzy k-means for cancer data analysis

Clustering analysis is a significant research topic in discovering cancer using different profiles of gene expression, which is very important to successfully diagnose and treat the cancer decease. Many ensemble clustering methods have been developed to perform clustering using tumor data. Only few of them incorporates a significant number of input clusterings, the optimal number of clusters in each input clustering, and an appropriate ensemble method to combine input clusterings into a final clustering. In this paper, we introduce two new steps in the standard fuzzy k-means algorithm to determine the optimal number of input clusterings, and the optimal number of clusters in each clustering for ensemble clustering. The first one is to incorporate a penalty term for making the algorithm insensitive to the initialization of cluster centroids. The second one is to automate a clustering process for iteratively updating the feature weights. This step addresses the noise values in the dataset. We propose an ensemble clustering method, which combines a set of input clusterings into a final clustering having better overall quality. Experiments on real cancer gene expression profiles illustrate that the proposed algorithm outperformed the well-known clustering algorithms.

Enhancing web search result clustering model based on multiview multirepresentation consensus cluster ensemble (mmcc) approach.

Existing text clustering methods utilize only one representation at a time (single view), whereas multiple views can represent documents. The multiview multirepresentation method enhances clustering quality. Moreover, existing clustering methods that utilize more than one representation at a time (multiview) use representation with the same nature. Hence, using multiple views that represent data in a different representation with clustering methods is reasonable to create a diverse set of candidate clustering solutions. On this basis, an effective dynamic clustering method must consider combining multiple views of data including semantic view, lexical view (word weighting), and topic view as well as the number of clusters. The main goal of this study is to develop a new method that can improve the performance of web search result clustering (WSRC). An enhanced multiview multirepresentation consensus clustering ensemble (MMCC) method is proposed to create a set of diverse candidate solutions and select a high-quality overlapping cluster. The overlapping clusters are obtained from the candidate solutions created by different clustering methods. The framework to develop the proposed MMCC includes numerous stages: (1) acquiring the standard datasets (MORESQUE and Open Directory Project-239), which are used to validate search result clustering algorithms, (2) preprocessing the dataset, (3) applying multiview multirepresentation clustering models, (4) using the radius-based cluster number estimation algorithm, and (5) employing the consensus clustering ensemble method. Results show an improvement in clustering methods when multiview multirepresentation is used. More importantly, the proposed MMCC model improves the overall performance of WSRC compared with all single-view clustering models.

Prediction of disease using fuzzy random forest

The learning approach is basically used for classification of different data into clusters. Basically, semi supervised learning has been used worldwide to classify the labelled as well as unlabeled data. The dataset sometimes may be in mixed features that may consist of both numeric and categorical type of data. In these two types, data may differ in their characteristics. Due to the differences in their characteristics, in order to group these types of mixed data, it is better to use the ensemble clustering method which uses split and merge approach to solve this problem. This research work carried out the original mixed dataset and is categorised into numeric dataset and categorical dataset and clustered using both traditional clustering algorithms and fuzzy clustering algorithms using random subspace approach called as fuzzy random forest (FRF). The resultant clusters are combined using ensemble clustering methods and evaluated by both f-measure and entropy measure. It is found that splitting is more beneficial and applying fuzzy clustering algorithms provides better results than traditional clustering algorithms. The system was tested on Hadoop multi node cluster environment as well as traditional environment. The hybrid genetic algorithm is used for optimisation.

Information Theoretic Weighted Fuzzy Clustering Ensemble

In order to improve performance and robustness of clustering, it is proposed to generate and aggregate a number of primary clusters via clustering ensemble technique. Fuzzy clustering ensemble approaches attempt to improve the performance of fuzzy clustering tasks. However, in these approaches, cluster (or clustering) reliability has not paid much attention to. Ignoring cluster (or clustering) reliability makes these approaches weak in dealing with low-quality base clustering methods. In this paper, we have utilized cluster unreliability estimation and local weighting strategy to propose a new fuzzy clustering ensemble method which has introduced Reliability Based weighted co-association matrix Fuzzy C-Means (RBFCM), Reliability Based Graph Partitioning (RBGP) and Reliability Based Hyper Clustering (RBHC) as three new fuzzy clustering consensus functions. Our fuzzy clustering ensemble approach works based on fuzzy cluster unreliability estimation. Cluster unreliability is estimated according to an entropic criterion using the cluster labels in the entire ensemble. To do so, the new metric is defined to estimate the fuzzy cluster unreliability; then, the reliability value of any cluster is determined using a Reliability Driven Cluster Indicator (RDCI). The time complexities of RBHC and RBGP are linearly proportional with the number of data objects. Performance and robustness of the proposed method are experimentally evaluated for some benchmark datasets. The experimental results demonstrate efficiency and suitability of the proposed method.

Sc-GPE: A Graph Partitioning-Based Cluster Ensemble Method for Single-Cell.

Clustering is an efficient way to analyze single-cell RNA sequencing data. It is commonly used to identify cell types, which can help in understanding cell differentiation processes. However, different clustering results can be obtained from different single-cell clustering methods, sometimes including conflicting conclusions, and biologists will often fail to get the right clustering results and interpret the biological significance. The cluster ensemble strategy can be an effective solution for the problem. As the graph partitioning-based clustering methods are good at clustering single-cell, we developed Sc-GPE, a novel cluster ensemble method combining five single-cell graph partitioning-based clustering methods. The five methods are SNN-cliq, PhenoGraph, SC3, SSNN-Louvain, and MPGS-Louvain. In Sc-GPE, a consensus matrix is constructed based on the five clustering solutions by calculating the probability that the cell pairs are divided into the same cluster. It solved the problem in the hypergraph-based ensemble approach, including the different cluster labels that were assigned in the individual clustering method, and it was difficult to find the corresponding cluster labels across all methods. Then, to distinguish the different importance of each method in a clustering ensemble, a weighted consensus matrix was constructed by designing an importance score strategy. Finally, hierarchical clustering was performed on the weighted consensus matrix to cluster cells. To evaluate the performance, we compared Sc-GPE with the individual clustering methods and the state-of-the-art SAME-clustering on 12 single-cell RNA-seq datasets. The results show that Sc-GPE obtained the best average performance, and achieved the highest NMI and ARI value in five datasets.

Immunocluster: A Computational Tool to Explore the Immune Profile and Cellular Heterogeneity of Hematological Diseases Using Liquid and Imaging Mass, and Flow Cytometry Data

Introduction Our understanding of 'immunome" and its role in the pathogenesis and clinical outcomes of hematological diseases has significantly improved in recent years, which highlights the importance of multidimensional cytometry techniques in investigating immune response both in clinical and research settings. Liquid mass cytometry (LMC), imaging mass cytometry (IMC) and flow cytometry (FC) are powerful techniques for immunophenotyping, biomarker discovery, and patients' immune-monitoring. These techniques provide the ability to profile over 40 markers per cell, resulting in large amounts of data which could be challenging to analyze and interpret. Therefore, we developed ImmunoCluster, an easy-to-use open-source computational pipeline, to explore high dimensionality single-cell cytometry datasets. Case studies Here we describe three examples of the implementation of the ImmunoCluster tool, which is a self-contained R package (accessed via GitHub: https://github.com/kordastilab/ImmunoCluster). Previously published LMC data from 15 leukemia patients 30 and 90 days after bone marrow transplantation (BMT) were used to test ImmunoCluster's ability to reproduce results. Post-BMT 3/15 patients suffered acute graft versus host disease (GvHD). For IMC data a lymph node section from a diffuse large B-cell lymphoma (DLBCL) patient. Finally, FC data from BM of 7 healthy donors (HDs) taken during hip surgery. The pipeline provides tools to allow researchers to follow a workflow which guides them through experimental design, data analyses and interpretation, to publishable graphics identifying differences in phenotype and abundance of cells between conditions (Figure 1A). ImmunoCluster comprises of three core computational stages: Stage 1. Data import and quality control In the experimental design stage, the high dimensional dataset is imported into ImmunoCluster with an associated metadata file, this included timepoints, and response to treatment. All data was stored within a SingleCellExperiment (SCE) object, a data container in which you can store/retrieve information such as metadata and UMAP/tSNE coordinates (Figure 1B). Initial exploratory visualization of the data, e.g. Multidimensional scaling (MDS) plots, and heatmaps showing marker expression for each patient were created and metadata used for annotation. Stage 2. Dimensionality reduction and unsupervised clustering We used three dimensionality reduction tools: MDS, uniform manifold approximation and projection (UMAP), and t-Distributed Stochastic Neighbor Embedding (tSNE). Two clustering algorithms: an ensemble clustering method of FlowSOM and Consensus clustering; and PhenoGraph. The aim of these algorithms were to assign all cells to clusters corresponding to true cell types. Stage 3. Annotation and differential testing Tools exploring cluster marker expression via projection onto UMAP/tSNE, or heatmaps, aided the identification of cell types and phenotypically distinct clusters. Metadata were used to annotate figures, allowing for visualization of the distribution of cell islands between different conditions/timepoints. Tools such as median marker expression, hierarchical clustered heatmaps, and box plots of cell cluster abundance were applied. Statistically significant differences between conditions were identified. Results We successfully replicated the findings from Hartmann et al., 24 cell populations were identified (Figure 2A-B). Significant differences between memory B-cells (FDR p=4.38 x 10-3), naïve B-cells (FDR p=1.35 x 10-2), and naïve CD4+ T-cells (FDR p=3.47 x 10-2) were identified between the GvHD and none (Figure 2C). From the FC HD CD4+ BM population data we were able to identify regulatory T cells (Tregs), including subpopulations, Treg A and Treg B (as low as 0.7% (0.1-2.0) and 0.9% (0.2-2.4), respectively). A marker expression ranking tool was applied to the DLBCL patient IMC data. We identified the majority of Ki-67 high population were proliferating tumor cells (84%), and the Ki-67 low population consisted of a heterogeneous collection of immune cell populations (Figure 3). These case studies show that ImmunoCluster could help clinicians and researchers with varying experience in computational biology to drive their projects from experimental design, wet lab/clinical trial, through to the data analysis process and visualization. Disclosures McLornan: JAZZ PHARMA: Honoraria, Speakers Bureau; NOVARTIS: Honoraria, Speakers Bureau; CELGENE: Honoraria, Speakers Bureau. Harrison:Gilead Sciences: Honoraria, Speakers Bureau; Incyte Corporation: Speakers Bureau; Janssen: Speakers Bureau; Sierra Oncology: Honoraria; Novartis: Honoraria, Research Funding, Speakers Bureau; AOP Orphan Pharmaceuticals: Honoraria; Shire: Honoraria, Speakers Bureau; Promedior: Honoraria; Roche: Honoraria; Celgene: Honoraria, Research Funding, Speakers Bureau; CTI Biopharma Corp: Honoraria, Speakers Bureau. Kordasti:Celgene: Research Funding; Novartis: Research Funding; Alexion: Honoraria.

Clustering Ensemble Model Based on Self-Organizing Map Network.

This paper proposes a clustering ensemble method that introduces cascade structure into the self-organizing map (SOM) to solve the problem of the poor performance of a single clusterer. Cascaded SOM is an extension of classical SOM combined with the cascaded structure. The method combines the outputs of multiple SOM networks in a cascaded manner using them as an input to another SOM network. It also utilizes the characteristic of high-dimensional data insensitivity to changes in the values of a small number of dimensions to achieve the effect of ignoring part of the SOM network error output. Since the initial parameters of the SOM network and the sample training order are randomly generated, the model does not need to provide different training samples for each SOM network to generate a differentiated SOM clusterer. After testing on several classical datasets, the experimental results show that the model can effectively improve the accuracy of pattern recognition by 4%∼10%.

Consensus function based on cluster-wise two level clustering

The ensemble clustering tries to aggregate a number of basic clusterings with the aim of producing a more consistent, robust and well-performing consensus clustering result. The current paper wants to introduce an ensemble clustering method. The proposed method, called consensus function based on two level clustering (CFTLC), introduces a new consensus clustering where it makes a cluster clustering task through applying an average hierarchical clustering on a cluster–cluster similarity matrix obtained by an innovative similarity metric. By applying the average hierarchical clustering algorithm, a set of meta clusters has been attained. Considering each meta cluster as a consensus cluster in the consensus clustering output, it then assigns each data point to a meta cluster through defining an object-cluster similarity. Before doing anything, CFTLC converts the primary partitions into a binary cluster representation where the primary ensemble has been broken into a number of basic binary clusters (BC). CFTLC first combines the basic BCs with the maximum cluster–cluster similarity. This step is iterated as long as a predefined number of meta clusters are ready. At the subsequent step, it assigns each data point to exactly one meta cluster. The proposed method has been experimentally compared with the state of the art clustering algorithms in terms of accuracy and robustness.

Environmental air pollution clustering using enhanced ensemble clustering methodology.

Air pollution these days could cause severe effects on human health. As human health is crumbled with serious respiratory or other lung diseases, it is prominent to study air pollution. One of the ways to address this issue is by applying clustering techniques. The two main important problems that are faced in the clustering algorithm are, firstly, the exact shape of the cluster and the number of clusters that input data can produce. Secondly, choosing an appropriate algorithm for a particular problem is not clearly known. Finally, multiple replications of the same algorithm lead to alternative solutions due to the fact such as random initialization of cluster heads. Ensembling algorithms can handle these problems and overcome bias and variance in the traditional clustering process. An adequate study has not been carried out in the ensembling approach mainly for clustering. In this paper, we use an enhanced ensemble clustering method to cluster the pollution data levels. This study helps to take preventive measures that are needed to control further contamination, reduce the alarming levels, and analyze the results to find healthy and unhealthy regions in a given area. This ensemble technique also explains about uncertain objects that are found in clustering. The distinct advantage of this algorithm is that there is no requirement of prior information about the data. This experiment shows that the implemented ensemble consensus clustering has demonstrated improved performance when compared with basic clustering algorithms.

High-Dimensional Unsupervised Active Learning Method

In this work, a hierarchical ensemble of projected clustering algorithm for high-dimensional data is proposed. The basic concept of the algorithm is based on the active learning method (ALM) which is a fuzzy learning scheme, inspired by some behavioral features of human brain functionality. High-dimensional unsupervised active learning method (HUALM) is a clustering algorithm which blurs the data points as one-dimensional ink drop patterns, in order to summarize the effects of all data points, and then applies a threshold on the resulting vectors. It is based on an ensemble clustering method which performs one-dimensional density partitioning to produce ensemble of clustering solutions. Then, it assigns a unique prime number to the data points that exist in each partition as their labels. Consequently, a combination is performed by multiplying the labels of every data point in order to produce the absolute labels. The data points with identical absolute labels are fallen into the same cluster. The hierarchical property of the algorithm is intended to cluster complex data by zooming in each already formed cluster to find further sub-clusters. The algorithm is verified using several synthetic and real-world datasets. The results show that the proposed method has a promising performance, compared to some well-known high-dimensional data clustering algorithms.

Prediction of students’ procrastination behaviour through their submission behavioural pattern in online learning

Prediction of students' performance has been reported as a vital task which enables educators to take necessary actions to improve students’ learning. Numerous studies have concluded that students with lower procrastination tendencies archive more compared to those with higher procrastination tendencies. In this study, a new method is proposed to predict students’ procrastination tendencies discerned from their submission behavioural patterns in online learning. In this method, feature vectors signifying students’ submission patterns on homework are firstly drafted. Next, an ensemble clustering method is employed to optimally sort students into various categories of procrastination: procrastinator, procrastinator candidate, and non-procrastinator. Lastly, various classification methods are assessed to discern which one best predicts students’ procrastination tendencies. The efficacy of this approach is assessed through the data from a course comprised of 242 students at the University of Tartu in Estonia. Our study found that our method correctly identifies student procrastination from submission pattern data with 97% accuracy, and that the best performing classifier is linear support vector machine. Investigating the effect of different number of features (homework) on performance of clustering and classification methods indicate that finding the optimal number of feature to use in both clustering and classification methods is a vital task as it could potentially affect prediction power of our approach. More specifically, the results show that in our proposed approach, unlike clustering methods that show a better performance with lower number of features, classification methods mostly tend to show a better performance with larger number of features.

A Methodology to Attain Public Transit Origin–Destination Mobility Patterns Using Multi-Layered Mesoscopic Analysis

Knowledge about mobility patterns has become increasingly important to urban development. In this work, public transit origin-destination (OD) mobility patterns are undergoing meso-level analysis in using the advantages of big data and for the creation of a new planning and decision-based tool. An ensemble clustering method is proposed to abstract the common OD pairs by fully considering link-based information, and the nonnegative tensor factorization model is adopted to effectively extract and visualize quantitatively the mobility patterns of OD pairs. This is attained by using multi-layered analysis such as of traffic demand, traffic accessibility and traffic congestion to enable different visual and quantitative mobility patterns. In the case study of Beijing, these patterns were analyzed and discussed by temporal and spatial factors. The results of the various patterns show explicitly when and where to provide remedies to traffic problems, by time and space. This is analogue, to some extent, to detecting and treating black spots of road accidents. The new developed multi-layered mesoscopic analysis could, therefore, be an important tool for improving urban planning, public transit planning, traffic management, and emergency intervention.

Assessment of air quality monitoring networks using an ensemble clustering method in the three major metropolitan areas of Mexico

The spatial representativeness of air quality stations is a crucial factor in monitoring networks for designing and applying adequate air quality control measures. If redundant stations, which duplicate air quality data, want to be avoided in order to optimize and reduce the operational cost of air quality networks, robust methodologies must be applied for identifying redundant stations. Therefore, this study proposes the use of a clustering ensemble method to recognize similar and redundant stations by combining three clustering techniques: principle component analysis, hierarchical clustering, and k-means. The result of the ensemble method is analyzed based on additional information, such as emission sources and the meteorological and topographical conditions of the area of interest. This methodology is applied to the ozone (O3) and particulate matter with an aerodynamic diameter of less than or equal to 10 μm (PM10) time series data, acquired from the air pollutant monitoring systems located in the three main metropolitan areas of Mexico: Mexico City (MCMA), Monterrey (MMA), and Guadalajara (GMA). The findings show that the GMA has a well distributed air quality network with the fewest number of similar stations, as well as the MMA, which presents the same stations clusters for PM10 and O3. In contrast, in the MCMA, a cluster of possible redundant stations is found. Results confirm that the clustering ensemble method represents a reliable tool for identifying similar stations.

Self-Paced Clustering Ensemble.

The clustering ensemble has emerged as an important extension of the classical clustering problem. It provides an elegant framework to integrate multiple weak base clusterings to generate a strong consensus result. Most existing clustering ensemble methods usually exploit all data to learn a consensus clustering result, which does not sufficiently consider the adverse effects caused by some difficult instances. To handle this problem, we propose a novel self-paced clustering ensemble (SPCE) method, which gradually involves instances from easy to difficult ones into the ensemble learning. In our method, we integrate the evaluation of the difficulty of instances and ensemble learning into a unified framework, which can automatically estimate the difficulty of instances and ensemble the base clusterings. To optimize the corresponding objective function, we propose a joint learning algorithm to obtain the final consensus clustering result. Experimental results on benchmark data sets demonstrate the effectiveness of our method.

Identification of air quality redundant stations through a clustering ensemble method

The main purpose of this study is to assess the performance of three air quality monitoring networks of Mexico. Emphasis is placed on an ensemble method to combine the results of the different clustering techniques: Principle Component Analysis, Hierarchical Clustering and k-means. The specific objectives of this paper are: (i) finding similar and redundant stations using the ensemble method and (ii) giving a physical meaning to groups of similar stations by evaluating additional information like emission sources, meteorology and topography of the area of interest. The study was applied on time series data of particulates that have aerodynamic diameters less than or equal to 10 μm (PM10) and ozone (O3), acquired from the air pollutant monitoring systems in the metropolitan areas of Mexico City (MCMA), Monterrey (MMA) and Guadalajara (GMA). These three conurbations are characterized by diverse meteorological and geographical conditions. The findings show that the GMA has a well distributed air quality network with the fewest number of similar stations. The MMA presents the same clusters of stations for PM10 and O3, while in the MCMA a cluster of possible redundant stations is found. Results confirm that the clustering ensemble method is a confidence tool to identify similar stations.

Improved Intelligent Techniques of Ensemble Data Clustering Method Using Bees Swarm Optimization Ensemble Approach

Improved Intelligent Techniques of Ensemble Data Clustering Method Using Bees Swarm Optimization Ensemble Approach