Ensemble Clustering Research Articles

A Survey on Multi-View Clustering.

Clustering is a machine learning paradigm of dividing sample subjects into a number of groups such that subjects in the same groups are more similar to those in other groups. With advances in information acquisition technologies, samples can frequently be viewed from different angles or in different modalities, generating multi-view data. Multi-view clustering, that clusters subjects into subgroups using multi-view data, has attracted more and more attentions. Although MVC methods have been developed rapidly, there has not been enough survey to summarize and analyze the current progress. Therefore, we propose a novel taxonomy of the MVC approaches. Similar to other machine learning methods, we categorize them into generative and discriminative classes. In discriminative class, based on the way of view integration, we split it further into five groups: Common Eigenvector Matrix, Common Coefficient Matrix, Common Indicator Matrix, Direct Combination and Combination After Projection. Furthermore, we relate MVC to other topics: multi-view representation, ensemble clustering, multi-task clustering, multi-view supervised and semi-supervised learning. Several representative real-world applications are elaborated for practitioners. Some benchmark multi-view datasets are introduced and representative MVC algorithms from each group are empirically evaluated to analyze how they perform on benchmark datasets. To promote future development of MVC approaches, we point out several open problems that may require further investigation and thorough examination.

Customer Choice Modelling: A Multi-Level Consensus Clustering Approach

Customer Choice Modeling aims to model the decision-making process of customers, or segments of customers, through their choices and preferences identified by the analysis of their behaviors in one or more specific contexts. Clustering techniques are used in this context to identify patterns in their choices and preferences, to define segments of customers with similar behaviors, and to model how customers of different segments respond to competing products and offers. However, data clustering is an unsupervised learning task by nature, that is the grouping of customers with similar behaviors in clusters must be performed without prior knowledge about the nature and the number of intrinsic groups of data instances, i.e., customers, in the data space. Thus, the choice of both the clustering algorithm used and its parameterization, and of the evaluation method used to assess the relevance of the resulting clusters are central issues. Consensus clustering, or ensemble clustering, aims to solve these issues by combining the results of different clustering algorithms and parameterizations to generate a more robust and relevant final clustering result. We present a Multi-level Consensus Clustering approach combining the results of several clustering algorithmic configurations to generate a hierarchy of consensus clusters in which each cluster represents an agreement between different clustering results. A closed sets based approach is used to identified relevant agreements, and a graphical hierarchical representation of the consensus cluster construction process and their inclusion relationships is provided to the end-user. This approach was developed and experimented in travel industry context with Amadeus SAS. Experiments show how it can provide a better segmentation, and refine the customer segments by identifying relevant sub-segments represented as sub-clusters in the hierarchical representation, for Customer Choice Modeling. The clustering of travelers was able to distinguish relevant segments of customers with similar needs and desires (i.e., customers purchasing tickets according to different criteria, like price, duration of flight, lay-over time, etc.) and at different levels of precision, which is a major issue for improving the personalization of recommendations in flight search queries.

Subtractive Gradient Boost Clustering for Mobile Node Authentication in Internet of Things Aware 5G Networks

The 5G mobile wireless network systems faces a lot of security issues due to the opening of network and its insecurity. The insecure network prone to various attacks and it disrupts secure data communications between legitimate users. Many works have addressed the security problems in 3G and 4G networks in efficient way through authentication and cryptographic techniques. But, the security in 5G networks during data communication was not improved. Subtractive Gradient Boost Clustered Node Authentication (SGBCNA) Method is introduced to perform secure data communication. The subtractive gradient boost clustering technique is applied to authenticate the mobile node as normal nodes and malicious nodes based on the selected features. The designed ensemble clustering model combines the weak learners to make final strong clustering results with minimum loss. Finally, the malicious nodes are eliminated and normal mobile nodes are taken for performing the secured communication in 5G networks. Simulation is carried out on factors such as authentication accuracy, computation overhead and security level with respect to a number of mobile nodes and data packets. The observed outcomes clearly illustrate that the SGBCNA Method efficiently improves node authentication accuracy, security level with minimum overhead than the state-of-the-art-methods.

Characterizing COVID-19 clinical phenotypes and associated comorbidities and complication profiles.

Heterogeneity has been observed in outcomes of hospitalized patients with coronavirus disease 2019 (COVID-19). Identification of clinical phenotypes may facilitate tailored therapy and improve outcomes. The purpose of this study is to identify specific clinical phenotypes across COVID-19 patients and compare admission characteristics and outcomes. This is a retrospective analysis of COVID-19 patients from March 7, 2020 to August 25, 2020 at 14 U.S. hospitals. Ensemble clustering was performed on 33 variables collected within 72 hours of admission. Principal component analysis was performed to visualize variable contributions to clustering. Multinomial regression models were fit to compare patient comorbidities across phenotypes. Multivariable models were fit to estimate associations between phenotype and in-hospital complications and clinical outcomes. The database included 1,022 hospitalized patients with COVID-19. Three clinical phenotypes were identified (I, II, III), with 236 [23.1%] patients in phenotype I, 613 [60%] patients in phenotype II, and 173 [16.9%] patients in phenotype III. Patients with respiratory comorbidities were most commonly phenotype III (p = 0.002), while patients with hematologic, renal, and cardiac (all p<0.001) comorbidities were most commonly phenotype I. Adjusted odds of respiratory, renal, hepatic, metabolic (all p<0.001), and hematological (p = 0.02) complications were highest for phenotype I. Phenotypes I and II were associated with 7.30-fold (HR:7.30, 95% CI:(3.11-17.17), p<0.001) and 2.57-fold (HR:2.57, 95% CI:(1.10-6.00), p = 0.03) increases in hazard of death relative to phenotype III. We identified three clinical COVID-19 phenotypes, reflecting patient populations with different comorbidities, complications, and clinical outcomes. Future research is needed to determine the utility of these phenotypes in clinical practice and trial design.

Donaldson-Thomas transformation of Grassmannian

Let m and n be two integers such that 1<m<n−1. The configuration space Confn×(Pm−1), which is the moduli space of n points in the projective space Pm−1 satisfying certain linearly independent condition, is closely related to the Grassmannian Grm,n, and is birationally equivalent to the cluster Poisson variety XAm−1□An−m−1.A Donaldson-Thomas transformation is a special formal automorphism on a cluster Poisson variety and encodes the Donaldson-Thomas invariants of the moduli space of stability conditions on the associated 3d Calabi-Yau category. The existence of a cluster Donaldson-Thomas transformation is part of Gross-Hacking-Keel-Kontsevich's sufficient condition for the Fock-Goncharov cluster duality on the cluster ensemble. In this paper we construct the cluster Donaldson-Thomas transformation on the cluster Poisson variety XAm−1□An−m−1 and realize it as an isomorphism on the configuration space Confn×(Pm−1), proving a conjecture of Goncharov and Shen. Via our construction we also prove the Fock-Goncharov cluster duality conjecture on the associated cluster ensemble and obtain a periodicity result on the Donaldson-Thomas transformation associated to the Am−1□An−m−1 quiver.

An Inception Convolutional Autoencoder Model for Chinese Healthcare Question Clustering.

Healthcare question answering (HQA) system plays a vital role in encouraging patients to inquire for professional consultation. However, there are some challenging factors in learning and representing the question corpus of HQA datasets, such as high dimensionality, sparseness, noise, nonprofessional expression, etc. To address these issues, we propose an inception convolutional autoencoder model for Chinese healthcare question clustering (ICAHC). First, we select a set of kernels with different sizes using convolutional autoencoder networks to explore both the diversity and quality in the clustering ensemble. Thus, these kernels encourage to capture diverse representations. Second, we design four ensemble operators to merge representations based on whether they are independent, and input them into the encoder using different skip connections. Third, it maps features from the encoder into a lower-dimensional space, followed by clustering. We conduct comparative experiments against other clustering algorithms on a Chinese healthcare dataset. Experimental results show the effectiveness of ICAHC in discovering better clustering solutions. The results can be used in the prediction of patients' conditions and the development of an automatic HQA system.

Enhanced Non-parametric Sequence-based Learning Algorithm for Outlier Detection in the Internet of Things

Although research on outlier detection methods has been an investigation area for long, few of those studies relate to an Internet of Things (IoT) domain. Several critical decisions taken on daily business operations depend on various data collected over time. Therefore, it is mandatory to guarantee its correctness, integrity, and accuracy before any further processing can commence. Outliers are often assumed to be Error by most algorithms in the past, which is always attributed to faulty sensors. Hence, this assumption has been investigated and results show that outliers can be classified into Error and Event types with the support of a Non-parametric sequence-based learning algorithm. The event type outlier is majorly caused by abnormality from sensor readings, which are very important and should not be ignored. However, the non-parametric sequence approach and other existing techniques still find it elusive to detect outliers in the global search space of a large dataset. Therefore, this paper proposes an Enhanced Non-parametric sequence learning algorithm based on Ensemble Clustering Techniques to detect Event and Error outliers in large datasets. Experiments are conducted on six different datasets from the UCL repository, except one collected from a laboratory testbed, to demonstrate the robustness and effectiveness of the proposed approach over the existing techniques. The results show a remarkable performance rate of 96.653% accuracy, 94.284% precision, and 98.112% for Error outlier detection. It also performs better in Event outlier detection with 87.611% accuracy, 71.141% precision and 85.755% specificity with 1291 s execution time.

Presenting a Proper Ensemble Clustering (EC) Method Based on Hierarchical Methods and Classical Generative Algorithms

Ensemble Clustering (EC) methods became more popular in recent years. In this methods, some primary clustering algorithms are considered to be as inputs and a single cluster is generated to achieve the best results combined with each other. In this paper, we considered three hierarchical methods, which are single-link, average-link, and complete-link as the primary clustering and the results were combined with each other. This combination was done based on correlation matrix. The basic algorithms were combined as binary and triplicate and the results were evaluated as well. the IMDB film dataset were clustered based on existing features. CH, Silhouette and Dunn Index criteria were used to evaluate the results. These criteria evaluate the clustering quality by calculating intra-cluster and inter-cluster distances. CH index had the highest value when all three basic clusters are combined. our method shows that EC can achieve better results and present clusters with higher robustness and accuracy.

OceanSODA-ETHZ: a global gridded data set of the surface ocean carbonate system for seasonal to decadal studies of ocean acidification

Abstract. Ocean acidification has profoundly altered the ocean's carbonate chemistry since preindustrial times, with potentially serious consequences for marine life. Yet, no long-term, global observation-based data set exists that allows us to study changes in ocean acidification for all carbonate system parameters over the last few decades. Here, we fill this gap and present a methodologically consistent global data set of all relevant surface ocean parameters, i.e., dissolved inorganic carbon (DIC), total alkalinity (TA), partial pressure of CO2 (pCO2), pH, and the saturation state with respect to mineral CaCO3 (Ω) at a monthly resolution over the period 1985 through 2018 at a spatial resolution of 1∘×1∘. This data set, named OceanSODA-ETHZ, was created by extrapolating in time and space the surface ocean observations of pCO2 (from the Surface Ocean CO2 Atlas, SOCAT) and total alkalinity (TA; from the Global Ocean Data Analysis Project, GLODAP) using the newly developed Geospatial Random Cluster Ensemble Regression (GRaCER) method (code available at https://doi.org/10.5281/zenodo.4455354, Gregor, 2021). This method is based on a two-step (cluster-regression) approach but extends it by considering an ensemble of such cluster regressions, leading to improved robustness. Surface ocean DIC, pH, and Ω were then computed from the globally mapped pCO2 and TA using the thermodynamic equations of the carbonate system. For the open ocean, the cluster-regression method estimates pCO2 and TA with global near-zero biases and root mean squared errors of 12 µatm and 13 µmol kg−1, respectively. Taking into account also the measurement and representation errors, the total uncertainty increases to 14 µatm and 21 µmol kg−1, respectively. We assess the fidelity of the computed parameters by comparing them to direct observations from GLODAP, finding surface ocean pH and DIC global biases of near zero, as well as root mean squared errors of 0.023 and 16 µmol kg−1, respectively. These uncertainties are very comparable to those expected by propagating the total uncertainty from pCO2 and TA through the thermodynamic computations, indicating a robust and conservative assessment of the uncertainties. We illustrate the potential of this new data set by analyzing the climatological mean seasonal cycles of the different parameters of the surface ocean carbonate system, highlighting their commonalities and differences. Further, this data set provides a novel constraint on the global- and basin-scale trends in ocean acidification for all parameters. Concretely, we find for the period 1990 through 2018 global mean trends of 8.6 ± 0.1 µmol kg−1 per decade for DIC, −0.016 ± 0.000 per decade for pH, 16.5 ± 0.1 µatm per decade for pCO2, and −0.07 ± 0.00 per decade for Ω. The OceanSODA-ETHZ data can be downloaded from https://doi.org/10.25921/m5wx-ja34 (Gregor and Gruber, 2020).

Fuzzy Clustering Ensemble Considering Cluster Dependability

Clustering ensemble has been progressively popular in the ongoing years by combining several base clustering methods into a most likely better and increasingly robust one. Nonetheless, fuzzy clustering dependability (durability) has been unnoticed within the majority of the proposed clustering ensemble approach. This makes them weak against low-quality fuzzy base clusters. In spite of a few endeavors made to the clustering methods, it appears that they consider each base-clustering separately without considering its local diversity. In this paper, to compensate for the mentioned weakness a new fuzzy clustering ensemble approach has been proposed using a weighting strategy at fuzzy cluster level. Indeed, each fuzzy cluster has a contribution weight computed based on its reliability (dependability/durability). After computing fuzzy cluster dependability (reliability/durability), dependability based fuzzy cluster-wise weighted matrix (DFCWWM) is computed. As a final point, the final clustering is obtained by applying the FCM traditional clustering algorithm over DFCWWM. The time complexity of the proposed approach is linear in terms of the number of data-points. The proposed approach has been assessed on 15 various standard datasets. The experimental evaluation has indicated that the proposed method has better performance than the state-of-the-art methods.

Classification and Clustering Based Ensemble Techniques for Intrusion Detection Systems: A Survey

A huge amount of data is transmitted through the networks, which allowed the exchange of knowledge and medical expertise, trade and banking facilities, etc. However, due to the huge connections to these networks, the security issue has been floated on the surface. Intrusion Detection System (IDS) plays a significant role to protect computer systems. To compensate these issues, the orientation is to employed machine learning and data mining techniques to design and implement powerful IDSs. Among these techniques is ensemble learning which enables a combination of multiple models to enhance overall performance. This study presents a brief overview of IDSs, discusses the history of ensemble systems, specifies the methods adapted in designed such system, highlights the most important ensemble techniques, demonstrates in detail the main methods that have been adapted in combining ensemble components. Besides, special attention was paid to studies in the period (2009-2020) that focus onto both ensemble classification and clustering when developing IDSs.

Fuzzy Ensemble Clustering Approach to Address Regionalization Uncertainties in Flood Frequency Analysis

AbstractVarious regionalization approaches are in use to delineate watersheds in an area into homogeneous groups/regions for regional frequency analysis of hydrologic extremes (floods, droughts). Each approach differs in the underlying assumptions and strategy for regionalization and yields regions that differ in composition. There is ambiguity in the choice of approaches, as none is established to be universally superior in yielding true regions that are unknown. This article proposes a novel fuzzy ensemble clustering (FEC) approach to deal with uncertainty in the composition of regions obtained using different regionalization approaches. It forms fuzzy meta‐regions by integrating information on similarities in watershed groupings found in an ensemble of regions derived using several regionalization approaches. The FEC approach's potential to form effective regions (than those found in the ensemble) and their utility in regional flood frequency analysis to predict flood quantiles at ungauged sites is demonstrated through Monte Carlo simulation experiments and a case study on peninsular India. An ensemble of regions for use with FEC is formed using the region of influence approach, clustering approach based on Gaussian mixture model (GMM), and a hybrid approach which combines canonical correlation analysis with GMM based clustering. The FEC is shown to be effective in grouping similar watersheds even when the ensemble comprises regions having some improperly/wrongly grouped watersheds.

Linear-time cluster ensembles of large-scale single-cell RNA-seq and multimodal data.

A fundamental task in single-cell RNA-seq (scRNA-seq) analysis is the identification of transcriptionally distinct groups of cells. Numerous methods have been proposed for this problem, with a recent focus on methods for the cluster analysis of ultralarge scRNA-seq data sets produced by droplet-based sequencing technologies. Most existing methods rely on a sampling step to bridge the gap between algorithm scalability and volume of the data. Ignoring large parts of the data, however, often yields inaccurate groupings of cells and risks overlooking rare cell types. We propose method Specter that adopts and extends recent algorithmic advances in (fast) spectral clustering. In contrast to methods that cluster a (random) subsample of the data, we adopt the idea of landmarks that are used to create a sparse representation of the full data from which a spectral embedding can then be computed in linear time. We exploit Specter's speed in a cluster ensemble scheme that achieves a substantial improvement in accuracy over existing methods and identifies rare cell types with high sensitivity. Its linear-time complexity allows Specter to scale to millions of cells and leads to fast computation times in practice. Furthermore, on CITE-seq data that simultaneously measures gene and protein marker expression, we show that Specter is able to use multimodal omics measurements to resolve subtle transcriptomic differences between subpopulations of cells.

WITHDRAWN: An optimized spectral neural classification model based on multiple kernel fork joining clustering for premature treatment in health care industries

The prolonged treatment in diabetics is a big problem need to classify the data for premature treatments. The diabetic’s premature diagnosis issue that arises in the bioinformatics and medicine field is reaching the incorrect time because of prediction accuracy in patient records. For the final diagnosis. Usually, several tests are performed involving the compilation and data classification. This classification process is needed to achieve all the final diagnoses failed in proposing weak leaners as features analyses and weight adjustments in classification methods to investigate performance and accuracy. To resolve the classification accuracy to propose an Optimized Spectral Neural Classification Model based on Multiple Kernel Fork Joining Clustering (SNC-MKFJC) for Premature Treatment in Health Care Industries. To create spectral margins in diabetic records with the support of Clustering ensembles to the selective attributes as feature case closet weightage as spectral features. The kernel creates relative weightage among the features, which is the firm's pattern and input to neurons. By finding a diagnostic solution by analytical patterns, the purpose of this paper is to examine taxonomic analyses by placing decision neurons on relative strong attachment to improve classification accuracy. Data found by research wants to recommend fast and efficient methods for diagnosing diseases that lead to timely treatment of patients in a premature state.

A new ensemble clustering method based on fuzzy cmeans clustering while maintaining diversity in ensemble

A new ensemble clustering method based on fuzzy cmeans clustering while maintaining diversity in ensemble

Ensemble clustering using extended fuzzy k-means for cancer data analysis

Clustering analysis is a significant research topic in discovering cancer using different profiles of gene expression, which is very important to successfully diagnose and treat the cancer decease. Many ensemble clustering methods have been developed to perform clustering using tumor data. Only few of them incorporates a significant number of input clusterings, the optimal number of clusters in each input clustering, and an appropriate ensemble method to combine input clusterings into a final clustering. In this paper, we introduce two new steps in the standard fuzzy k-means algorithm to determine the optimal number of input clusterings, and the optimal number of clusters in each clustering for ensemble clustering. The first one is to incorporate a penalty term for making the algorithm insensitive to the initialization of cluster centroids. The second one is to automate a clustering process for iteratively updating the feature weights. This step addresses the noise values in the dataset. We propose an ensemble clustering method, which combines a set of input clusterings into a final clustering having better overall quality. Experiments on real cancer gene expression profiles illustrate that the proposed algorithm outperformed the well-known clustering algorithms.

Analisis Cluster Pada Data Kategorik dan Numerik dengan Pendekatan Cluster Ensemble

Cluster analysis used to process categorical and numerical data at once is Cluster Ensemble algorithm Based on Mixed Data Clustering (algCEBMDC), which is a cluster algorithm with an ensemble cluster approach. The method used for numerical data is Agglomerative Nesting (AGNES) algorithm and for categorical data is the RObust Clustering using linK (ROCK) algorithm. The best clustering method and the optimum number of clusters in the AGNES algorithm is selected based on the maximum Pseudo-F value and the minimum icdrate value. The optimum number of clusters in the ROCK algorithm is selected using the minimum value of ratio . The purpose of this study was to make a group of 179 Puskesmas in East Kalimantan on December 2017. Based on the results of the analysis, obtained 5 optimum cluster for numerical clustering with the AGNES algorithm and 2 optimum cluster for categorical clustering data with the ROCK algorithm. Final cluster for mixed data clustering obtained 2 optimum cluster at a threshold of 0.2 and 0.3 with value of ratio is . The first cluster consists of 83 Puskesmas and cluster two of 96 Puskesmas.

Enhancing web search result clustering model based on multiview multirepresentation consensus cluster ensemble (mmcc) approach.

Existing text clustering methods utilize only one representation at a time (single view), whereas multiple views can represent documents. The multiview multirepresentation method enhances clustering quality. Moreover, existing clustering methods that utilize more than one representation at a time (multiview) use representation with the same nature. Hence, using multiple views that represent data in a different representation with clustering methods is reasonable to create a diverse set of candidate clustering solutions. On this basis, an effective dynamic clustering method must consider combining multiple views of data including semantic view, lexical view (word weighting), and topic view as well as the number of clusters. The main goal of this study is to develop a new method that can improve the performance of web search result clustering (WSRC). An enhanced multiview multirepresentation consensus clustering ensemble (MMCC) method is proposed to create a set of diverse candidate solutions and select a high-quality overlapping cluster. The overlapping clusters are obtained from the candidate solutions created by different clustering methods. The framework to develop the proposed MMCC includes numerous stages: (1) acquiring the standard datasets (MORESQUE and Open Directory Project-239), which are used to validate search result clustering algorithms, (2) preprocessing the dataset, (3) applying multiview multirepresentation clustering models, (4) using the radius-based cluster number estimation algorithm, and (5) employing the consensus clustering ensemble method. Results show an improvement in clustering methods when multiview multirepresentation is used. More importantly, the proposed MMCC model improves the overall performance of WSRC compared with all single-view clustering models.

A novel clustering ensemble model based on granular computing

Clustering ensemble is one of the popular methods in the field of data mining for discovering hidden patterns in unlabeled datasets. Researches have shown that selecting base clustering results with certain differences and high quality to participate in the fusion process can improve the quality of the final result. However, the existing inherent characteristics of uncertainty, ambiguity, and overlap of the base clustering results make the selection of the base clustering members more difficult. The accuracy of the final results is easily disturbed by low-quality base clustering members. From the perspective of granular computing, a novel clustering ensemble model is proposed. The similarity among ensemble members is measured by granularity distance, so the quality of the base clustering results is ensured meanwhile the difference among them is enlarged, which is beneficial to improve the accuracy of the final result. According to the dividing ability of knowledge granularity, the method of elements generation for the co-association matrix is optimized and improved. The results obtained from the improved sample similarity measurement are more consistent with the structure of the real data. Compared with the traditional single clustering algorithm and some popular clustering ensemble methods, experiments show that the proposed model improves the quality of the final clustering result and has good expandability.

A clustering and ensemble based classifier for data stream classification

In the era of data mining, the research industry has great attention to data stream mining as well as it has a great impact on a wide range of applications like networking, telecommunication, education, banking, weather forecasting, a stock market, and so on. Because of these data stream mining having more attention from researchers. The handling of concept drifting data streams is one of the major issues and challenges in the data stream mining field. In the presence of the concept drift, the performance of the learning algorithm always degrades. In this paper, a hybrid method has been proposed which are the combination of an ensemble, and grid and density-based clustering methods. The proposed method is tested on both synthetic as well as real data. The proposed method works well in the presence of concept drift and performance is measured in terms of time, accuracy, and memory. As compared with the state-of-art algorithms, the proposed method performed well and gave better accuracy using synthetic datasets like 88.29%, 71.34%, and 75.39% for Hyperplane, RBF, and LED respectively and for real datasets 86.17%, 86.28%, 95.15%, and 99.83% for Adult, Census-Income, KDDCup99%–10%, and Covertype respectively.