Ensemble Clustering Research Articles

Differential diagnosis of bone marrow failure syndromes guided by machine learning

AbstractThe choice to postpone treatment while awaiting genetic testing can result in significant delay in definitive therapies in patients with severe pancytopenia. Conversely, the misdiagnosis of inherited bone marrow failure (BMF) can expose patients to ineffectual and expensive therapies, toxic transplant conditioning regimens, and inappropriate use of an affected family member as a stem cell donor. To predict the likelihood of patients having acquired or inherited BMF, we developed a 2-step data-driven machine-learning model using 25 clinical and laboratory variables typically recorded at the initial clinical encounter. For model development, patients were labeled as having acquired or inherited BMF depending on their genomic data. Data sets were unbiasedly clustered, and an ensemble model was trained with cases from the largest cluster of a training cohort (n = 359) and validated with an independent cohort (n = 127). Cluster A, the largest group, was mostly immune or inherited aplastic anemia, whereas cluster B comprised underrepresented BMF phenotypes and was not included in the next step of data modeling because of a small sample size. The ensemble cluster A–specific model was accurate (89%) to predict BMF etiology, correctly predicting inherited and likely immune BMF in 79% and 92% of cases, respectively. Our model represents a practical guide for BMF diagnosis and highlights the importance of clinical and laboratory variables in the initial evaluation, particularly telomere length. Our tool can be potentially used by general hematologists and health care providers not specialized in BMF, and in under-resourced centers, to prioritize patients for genetic testing or for expeditious treatment.

Fraction of stars in clusters for the LEGUS dwarf galaxies

ABSTRACT We study the young star cluster populations in 23 dwarf and irregular galaxies observed by the Hubble Space Telescope (HST) Legacy ExtraGalactic Ultraviolet Survey (LEGUS), and examine relationships between the ensemble properties of the cluster populations and those of their host galaxies: star formation rate (SFR) density (ΣSFR). A strength of this analysis is the availability of SFRs measured from temporally resolved star formation histories that provide the means to match cluster and host galaxy properties on several time-scales (1–10, 1–100, and 10–100 Myr). Nevertheless, studies of this kind are challenging for dwarf galaxies due to the small numbers of clusters in each system. We mitigate these issues by combining the clusters across different galaxies with similar ΣSFR properties. We find good agreement with a well-established relationship ($M_{V}^{\mathrm{ brightest}}$–SFR), but find no significant correlations between ΣSFR and the slopes of the cluster luminosity function, mass function, nor the age distribution. We also find no significant trend between the fraction of stars in bound clusters at different age ranges (Γ1–10, Γ10–100, and Γ1–100) and ΣSFR of the host galaxy. Our data show a decrease in Γ over time (from 1–10 to 10–100 Myr) suggesting early cluster dissolution, though the presence of unbound clusters in the youngest time bin makes it difficult to quantify the degree of dissolution. While our data do not exhibit strong correlations between ΣSFR and ensemble cluster properties, we cannot rule out that a weak trend might exist given the relatively large uncertainties due to low number statistics and the limited ΣSFR range probed.

Semi-Supervised Clustering Ensemble Based on Cluster Consensus Selection

Ensemble clustering emerged as an important extension of classical clustering problems and is one of the most recent advances in unsupervised learning. Its purpose is to combine the results obtained using different algorithms by a consensus function so that the final solution is more favorable than the individual clustering algorithms. In this study, we propose a semi-supervised clustering ensemble framework using cluster consensus selection, which tries to improve the accuracy of clustering results. In general, there are two types of semi-supervised clustering algorithms, including constraint-based and metric-based. Here, the proposed ensemble clustering algorithm is equipped with a semi-supervised clustering mechanism based on pairwise constraints. Since the complexity of consensus functions scales with the number of clustering methods, processing big data for ensemble clustering is sometimes slow or impossible. Usually, all primary clusters from all clustering methods are used in the consensus function. However, the merit of clusters from different methods can be considered to improve the consensus quality. Accordingly, we propose a cluster consensus selection approach that selects a subset of meriting primary clusters to participate in the final consensus. Here, Normalized Mutual Information (NMI) is developed to measure the merit of clusters. Meanwhile, reducing the number of primary clusters in the consensus function can enable big data clustering. The proposed algorithm is very computationally efficient and provides linear complexity in clustering. Experimental results show the effectiveness of the proposed algorithm in terms of different performance metrics such as NMI, ARI and CPCC.

An ensemble hierarchical clustering algorithm based on merits at cluster and partition levels

Ensemble clustering has emerged as a combination of several basic clustering algorithms to achieve high quality final clustering. However, this technique is challenging due to the complexities in primary clusters such as overlapping, vagueness, instability and uncertainty. Typically, ensemble clustering uses all the primary clusters into partitions for consensus, where the merits of a cluster or a partition can be considered to improve the quality of the consensus. In general, the robustness of a partition may be poorly measured, while having some high-quality clusters. Inspired by the evaluation of cluster and partition, this paper proposes an ensemble hierarchical clustering algorithm based on the cluster consensus selection approach. Here, the selection of a subset of primary clusters from partitions based on their merit level is emphasized. Merit level is defined using the development of Normalized Mutual Information measure. Clusters of basic clustering algorithms that satisfy the predefined threshold of this measure are selected to participate in the final consensus. In addition, the consensus of the selected primary clusters to create the final clusters is performed based on the clusters clustering technique. In this technique, the selected primary clusters are re-clustered to create hyper-clusters. Finally, the final clusters are formed by assigning instances to hyper-clusters with the highest similarity. Here, an innovative criterion based on merit and cluster size for defining similarity is presented. The performance of the proposed algorithm has been proven by extensive experiments on real-world datasets from the UCI repository compared to state-of-the-art algorithms such as CPDM, ENMI, IDEA, CFTLC and SSCEN.

Semi-supervised clustering of quaternion time series: Application to gait analysis in multiple sclerosis using motion sensor data.

Recent approaches in gait analysis involve the use of wearable motion sensors to extract spatio-temporal parameters that characterize multiple aspects of an individual's gait. In particular, the medical community could largely benefit from this type of devices as they could provide the clinicians with a valuable tool for assessing gait impairment. Motion sensor data are however complex and there is an urgent unmet need to develop sound statistical methods for analyzing such data and extracting clinically relevant information. In this article, we measure gait by following the hip rotation over time and the resulting statistical unit is a time series of unit quaternions. We explore the possibility to form groups of patients with similar walking impairment by taking into account their walking data and their global decease severity with semi-supervised clustering. We generalize a compromise-based method named hclustcompro to unit quaternion time series by combining it with the proper dissimilarity quaternion dynamic time warping. We apply this method on patients diagnosed with multiple sclerosis to form groups of patients with similar walking deficiencies while accounting for the clinical assessment of their overall disability. We also compare the compromise-based clustering approach with the method mergeTrees that falls into a sub-class of ensemble clustering named collaborative clustering. The results provide a first proof of both the interest of using wearable motion sensors for assessing gait impairment and the use of prior knowledge to guide the clustering process. It also demonstrates that compromise-based clustering is a more appropriate approach in this context.

Ensemble Clustering in GPS Velocities: A Case Study of Turkey

Block modeling is an effective way to understand Earth’s crustal deformation. However, the choice of block boundaries and the number of blocks affect the model results. Therefore, the subjectivity of this analysis should be avoided. Clustering analysis can be used to define the blocks of GPS (Global Positioning System) velocity fields without a priori information. Unfortunately, clustering methods also have unique solutions and differ with various algorithms. Ensemble methods could be an answer to enhance the clustering results for GPS velocities. In this study, we use ensemble clustering to identify block boundaries before block modeling without a priori information about the data. The ensemble clustering method is used for the first time in the clustering of GPS velocities and the case of Turkey is discussed. The published horizontal GPS velocities were first clustered with five different clustering methods and the optimum classes were determined using ensemble clustering methods. It is proven that the Meta-CLustering Algorithm can be used in terms of ensemble clustering for this region.

Big data analysis using a parallel ensemble clustering architecture and an unsupervised feature selection approach

Ensemble clustering is known as a challenging research direction in data mining. The results of several individual clustering methods are combined to produce higher quality final clusters. This study introduces a parallel hierarchical clustering approach based on the divide-and-conquer strategy, which is an attempt to realize faster and more efficient ensemble clustering. Here, we propose a cluster consensus selection approach that selects a subset of meriting primary clusters to participate in the final consensus. Considering the sample-cluster and cluster–cluster similarity on the selected primary clusters, we form the final clusters based on the clusters clustering technique as a consensus function. In addition, the proposed scheme is equipped with an unsupervised feature selection approach to remove features that do not contribute significantly to clustering. Extensive evaluations have been performed on datasets of different dimensions from the University of California Irvine (UCI) machine learning repository. The simulation results guarantee the efficiency of the proposed scheme and improves the average performance between 6% and 24% compared to the state-of-the-art clustering methods.

An ensemble clustering method based on consistency cluster consensus approach and MapReduce model

AbstractEnsemble clustering is an efficient unsupervised learning technique that has attracted a lot of attention. The purpose of this technique is to aggregate the results of several basic clustering algorithms in order to create a better clustering. This is not only possible, but has been developed with many techniques in recent years. However, there are still challenges such as similarity measure, agreement function and how to implement clustering algorithms. To address these challenges, this article proposes an ensemble clustering algorithm based on the consistency cluster consensus approach and the MapReduce model. In addition, the proposed algorithm is equipped with a new membership similarity measure. The consistency cluster consensus is defined as a new agreement function for the consensus of the results of the basic clustering methods. Besides, the proposed similarity measure consists of two factors: one is cluster similarity and the another is membership similarity. The process of the proposed ensemble clustering method is summarized in four steps. In the first step, partitions are generated by applying a number of individual clustering algorithms to the data. The second step is to convert the primary clusters from the partitions to a binary representation. The third step is to identify the candidate primary clusters for the consensus, where a subset with highest similarity between the clusters is emphasized. In the fourth step, the consistency cluster consensus is performed through a new agreement function. In addition, we use the MapReduce model to implement basic clustering algorithms to reduce computational complexity. Our method is evaluated on several real‐world datasets and their performance was compared with other clustering ensemble methods, such as TWCE and RDPC‐DSS. Experiments show that our method is more efficient and accurate and is 7% to 15% better than the best method compared.

Finite Mixtures of ERGMs for Modeling Ensembles of Networks

Ensembles of networks arise in many scientific fields, but there are relatively few statistical tools for inferring their generative processes, particularly in the presence of both dyadic dependence and cross-graph heterogeneity. To address this gap, we propose characterizing network ensembles via finite mixtures of exponential family random graph models (ERGMs), a class of parametric statistical models that has been successful in explicitly modeling the complex stochastic processes that govern the structure of edges in a network. Our proposed modeling framework can also be used for applications such as model-based clustering of ensembles of networks and density estimation for complex graph distributions. We develop a joint approach to estimate the number of mixture components and identify cluster-specific parameters simultaneously as well as to obtain an identified model under the Bayesian paradigm. Specifically, we develop a Metropolis-within-Gibbs algorithm to perform Bayesian inference, and estimate the number of mixture components using a strategy of deliberate overfitting with sparse priors that removes excess components during MCMC. As the true ERGM likelihood is generally intractable for model specifications with dyadic dependence terms, we consider two tractable approximations (pseudolikelihood and adjusted pseudolikelihood) to facilitate efficient statistical inference. We run simulation studies to compare the performance of these two approximations with respect to multiple metrics, showing conditions under which both are useful. We demonstrate the utility of the proposed approach using an ensemble of political co-voting networks among U.S. Senators and an ensemble of brain functional connectivity networks.

Model and Method for Constructing a Heterogeneous Cluster Ensemble

Model and Method for Constructing a Heterogeneous Cluster Ensemble

Analysis of Power Load Characteristics Based on Adaptive Ensemble Clustering Algorithm

With the growing number of power users and more flexible power consumption, how to cluster the power load curves efficiently becomes particularly important. In order to improve the accuracy, effectiveness and rapidity of load clustering, an adaptive integrated fuzzy clustering algorithm was proposed in this paper. Firstly, the initial clustering centroids were obtained by the improved DPC algorithm; then, under the dual guarantee of Euclidean distance and membership, FCM algorithm was used to cluster load curves; finally, the optimal number of clusters was obtained by the improved adaptive method in the iterative process; thus, the autonomous and accurate clustering of power load curve was realized. The proposed algorithm, DPC algorithm and FCM algorithm were applied to the load clustering experiment of power system respectively. The comparative analysis shows that this algorithm is effective and superior for load curve clustering.

Industrial park electric power load pattern recognition: An ensemble clustering-based framework

Electric power load pattern recognition from various accumulated load data is performed for energy efficiency improvement, power system operation support, and demand side management. Industrial park is a typical energy consumption scenario, in which there are many different types of energy users. In this study, a framework based on ensemble clustering is proposed for electric power load pattern recognition in industrial parks. The proposed framework can not only accelerate the ensemble clustering process but also improve the clustering performance. Specifically, we focus on the electric power load pattern recognition of business office industrial parks with air conditioners as the main energy-consuming device. The hourly air conditioning load data and total electric power load data of eight buildings in an industrial park in Suzhou, China, from January 1, 2019, to December 31, 2019, were used in the experiment. The results showed the effectiveness of the proposed framework in recognizing the electric power load patterns in industrial parks. The identified electric power load patterns could be potentially used to support more efficient and targeted industrial park energy management decision-making.

GeoWaVe: geometric median clustering with weighted voting for ensemble clustering of cytometry data.

Clustering is an unsupervised method for identifying structure in unlabelled data. In the context of cytometry, it is typically used to categorize cells into subpopulations of similar phenotypes. However, clustering is greatly dependent on hyperparameters and the data to which it is applied as each algorithm makes different assumptions and generates a different 'view' of the dataset. As such, the choice of clustering algorithm can significantly influence results, and there is often not one preferred method but different insights to be obtained from different methods. To overcome these limitations, consensus approaches are needed that directly address the effect of competing algorithms. To the best of our knowledge, consensus clustering algorithms designed specifically for the analysis of cytometry data are lacking. We present a novel ensemble clustering methodology based on geometric median clustering with weighted voting (GeoWaVe). Compared to graph ensemble clustering methods that have gained popularity in single-cell RNA sequencing analysis, GeoWaVe performed favourably on different sets of high-dimensional mass and flow cytometry data. Our findings provide proof of concept for the power of consensus methods to make the analysis, visualization and interpretation of cytometry data more robust and reproducible. The wide availability of ensemble clustering methods is likely to have a profound impact on our understanding of cellular responses, clinical conditions and therapeutic and diagnostic options. GeoWaVe is available as part of the CytoCluster package https://github.com/burtonrj/CytoCluster and published on the Python Package Index https://pypi.org/project/cytocluster. Benchmarking data described are available from https://doi.org/10.5281/zenodo.7134723. Supplementary data are available at Bioinformatics online.

Examining unsupervised ensemble learning using spectroscopy data of organic compounds.

One solution to the challenge of choosing an appropriate clustering algorithm is to combine different clusterings into a single consensus clusteringresult, known as cluster ensemble (CE). This ensemble learning strategy can provide more robust and stable solutions across different domains and datasets. Unfortunately, not all clusterings in the ensemble contribute to the final data partition. Cluster ensemble selection (CES) aims at selecting a subset from a large library of clustering solutions to form a smaller cluster ensemble that performs as well as or better than the set of all available clustering solutions. In this paper, we investigate four CES methods for the categorization of structurally distinct organic compounds using high-dimensional IR and Raman spectroscopy data. Single quality selection (SQI) forms a subset of the ensemble by selecting the highest quality ensemble members. The Single Quality Selection (SQI) method is used with various quality indices to select subsets by including the highest quality ensemble members. The Bagging method, usually applied in supervised learning,ranksensemble members by calculating the normalized mutual information (NMI) between ensemble members and consensus solutions generated from a randomly sampled subset of the full ensemble. The hierarchical cluster and select method (HCAS-SQI) uses the diversity matrix of ensemble members to select a diverse set of ensemble members with the highest quality. Furthermore, a combining strategy can be used to combine subsets selected using multiple quality indices (HCAS-MQI) for the refinement of clustering solutions in the ensemble. The IR + Raman hybrid ensemble library is created by merging two complementary "views" of the organic compounds. This inherently more diverse library gives the best full ensemble consensus results. Overall, the Bagging method is recommended because it provides the most robust results that are better than or comparable to the full ensemble consensus solutions.

DCE-IVI: Density-based clustering ensemble by selecting internal validity index

As each clustering algorithm cannot efficiently partition datasets with arbitrary shapes, the thought of clustering ensemble is proposed to consistently integrate clustering results to obtain better division. Most of ensemble research employs a single algorithm with different parameters to clustering. And this can be easily integrated, however it is hardly to divide complex datasets. Other available methods integrate different algorithms, it can divide datasets from different aspects, but fail to take outliers into account, which produces negative effects on the partition results. In order to solve these problems, we clustering datasets with three different density-based algorithms. The innovation of this paper is described as: (1) by setting dynamic thresholds, lower frequency evidence in the co-association matrix is gradually deleted to obtain multiple reconstructed matrices; (2) these reconstructed matrices are analyzed by hierarchical clustering to obtain basic clustering results; (3) an internal validity index is designed by the compactness within clusters and the correlation between clusters, which is used to select the final clustering result. By this innovation, the clustering effect is significantly improved. Finally, a series of experiments are designed, and the results verify the improvement and effectiveness of the proposed technique (DCE-IVI).

Massive Spatial Well Clustering Based on Conventional Well Log Feature Extraction for Fast Formation Heterogeneity Characterization

Abstract Recent enhancements in computational capacity provide an opportunity for harnessing the enormous amount of reservoir data already acquired and extracting useful information for hydrocarbon exploration, development, and production. This article reports a three-step clustering technique to determine well groups based on subsurface geological heterogeneity using feature extraction, hierarchical ensemble clustering, and spatial mapping. The first step of the presented methodology is to group the wells into different clusters based on the formation rock composition and property features extracted from well logs using the expectation maximization algorithm. The one-dimensional (1D) stacking pattern of each well log curve is expressed through a two-dimensional (2D) transformation scheme. Thus, the clustering can capture the vertical stacking patterns of well logs, which is essential for reservoir heterogeneity characterization. This base clustering process generated a feature matrix which is further grouped through the hierarchical ensemble clustering in a latent space of well logs in the second step. Through the ensemble clustering, different clustering proposals obtained from the base clustering are integrated corroboratively to reflect a comprehensive feature of all studied logs. In the third step, the spatial clustering is performed based on the ensemble results, considering the spatial distances between well locations in the target area. The results of the 2D spatial map may provide insights into the sedimentary depositional environment in terms of the lateral geological heterogeneity features. Therefore, the proposed clustering technique can present a fast geological modeling method to integrate geological heterogeneity features presented in multiple well logs, which is not yet fully utilized in traditional geomodeling approaches. The results can also support further reservoir studies, such as petrophysical modeling, reservoir modeling, and fluid flow simulation studies.

Two-stage semi-supervised clustering ensemble framework based on constraint weight

Two-stage semi-supervised clustering ensemble framework based on constraint weight

A new approach based on game theory to reflect meta-cluster dependencies into VoIP attack detection using ensemble clustering

A new approach based on game theory to reflect meta-cluster dependencies into VoIP attack detection using ensemble clustering

Deep Belief Neural Network (DBNN)-Based Categorization of Uncertain Data Streams

In the data mining era, the research field is paying attention to data stream mining, which offers a substantial influence on a variety of applications such as networking, wireless communications, education, economics, weather prediction, financial sector, and so on. Moreover, processing of this uncertain data stream faces two major challenges, which are computational difficulty and long processing time of data. Thus, to overcome this, this work proposes a technique that employs a deep belief neural network to categorize uncertain data streams. Initially, this work utilized a hybrid method that combines ensemble, grid, and density-dependent clustering approaches to acquire the local optimum value in uncertain data streams. Furthermore, for classification, a deep belief neural network (DBNN) has been used. As a result of mining, target semantics or chunks will be obtained from the classified data. The suggested technique performs well, and its effectiveness has been assessed in terms of time and accuracy. Thus, the proposed method outperforms the existing techniques.

Atomistic insights into the binding of SARS-CoV-2 spike receptor binding domain with the human ACE2 receptor: The importance of residue 493

SARS-CoV-2 is a coronavirus that has created a global pandemic. The virus contains a spike protein which has been shown to bind to the ACE2 receptor on the surface of human cells. Vaccines have been developed that recognize elements of the SARS-CoV-2 spike protein and they have been successful in preventing infection. Recently, the Omicron variant of the SARS-CoV-2 virus was reported and quickly became a variant of concern due to its transmissibility. This variant contained an unusually large number (32) of point mutations, of which 15 of those mutations are in the receptor binding domain of the spike protein. While several computational and experimental investigations comparing the binding of the Omicron and wild type RBD to the human ACE2 receptor have been conducted, many of these report contradictory findings. In order to assess the differential binding ability, we conducted 2 μs of classical molecular dynamics (cMD) simulation to estimate the binding affinities and behaviors. Based upon MM-GBSA binding affinity, per-residue energy decomposition analysis, center of mass distance measurements, ensemble clustering, pairwise residue decomposition and hydrogen bonding analysis, our results suggest that a single point mutation is responsible for the enhanced binding of the Omicron mutant relative to the WT. While the 15-point mutations in the receptor binding domain contribute positively and negatively to the affinity of the spike protein for the human ACE2 receptor, it is the point mutation Q493R that confers enhanced binding while the Q493K mutation results in similar binding. The MM-GBSA binding estimations over a 2 μs trajectory, suggest that the wild type binds to ACE2 with a value of −29.69 kcal/mol while the Q493K and Q493R Omicron mutants bind with energy values of −26.67 and −34.56 kcal/mol, respectively. These values are significantly different, given the error estimates associated with the MM-GBSA method. In general, while some mutations increase binding, more mutations diminish binding, leading to an overall similar picture of binding for Q493K and enhanced binding for Q493R.