Ensemble Clustering Research Articles

Large-scale gene expression data clustering through incremental ensemble approach

Abstract DNA microarray technology monitors gene activity in real-time in living organisms. It creates a large amount of data that helps scientists learn about how genes work. Clustering this data helps understand gene interactions and uncover important biological processes. However, the traditional clustering techniques have difficulties due to the enormous dimensionality of gene expression data and the intricacy of biological networks. Although ensemble clustering is a viable strategy, such high-dimensional data may not lend itself well to traditional approaches. This study introduces a novel technique for gene expression data clustering called incremental ensemble clustering for gene expression data (IECG). There are two steps in the IECG. A technique for grouping gene expression data into windows is presented in the first step, producing a tree of clusters. This procedure is carried out again for succeeding windows that have distinct feature sets. The base clusterings of two consecutive windows are ensembled using a new goal function to form a new clustering solution. By repeating this step-by-step method for further windows, reliable patterns that are beneficial for medical applications can be extracted. The results from both biological and non-biological data demonstrate that the proposed algorithm outperformed the state-of-the-art algorithms. Additionally, the running time of the proposed algorithm has been examined.

Toden-E: Topology-Based and Density-Based Ensembled Clustering for the Development of Super-PAG in Functional Genomics using PAG Network and LLM.

The integrative analysis of gene sets, networks, and pathways is pivotal for deciphering omics data in translational biomedical research. To significantly increase gene coverage and enhance the utility of pathways, annotated gene lists, and gene signatures from diverse sources, we introduced pathways, annotated gene lists, and gene signatures (PAGs) enriched with metadata to represent biological functions. Furthermore, we established PAG-PAG networks by leveraging gene member similarity and gene regulations. However, in practice, high similarity in functional descriptions or gene membership often leads to redundant PAGs, hindering the interpretation from a fuzzy enriched PAG list. In this study, we developed todenE (topology-based and density-based ensemble) clustering, pioneering in integrating topology-based and density-based clustering methods to detect PAG communities leveraging the PAG network and Large Language Models (LLM). In computational genomics annotation, the genes can be grouped/clustered through the gene relationships and gene functions via guilt by association. Similarly, PAGs can be grouped into higher-level clusters, forming concise functional representations called Super-PAGs. TodenE captures PAG-PAG similarity and encapsulates functional information through LLM, in characterizing network-based functional Super-PAGs. In synthetic data, we introduced a metric called the Disparity Index (DI), measuring the connectivity of gene neighbors to gauge clusterability. We compared multiple clustering algorithms to identify the best method for generating performance-driven clusters. In non-simulated data (Gene Ontology), by leveraging transfer learning and LLM, we formed a language-based similarity embedding. TodenE utilizes this embedding together with the topology-based embedding to generate putative Super-PAGs with superior performance in semantic and gene member inclusiveness.

Ensemble clustering by block diagonal representation

Ensemble clustering by block diagonal representation

A Clustering Ensemble Method for Drug Safety Signal Detection in Post-Marketing Surveillance.

Post-marketing surveillance refers to the process of monitoring the safety of drugs once they reach the market, after the successful completion of clinical trials. In this work, we investigate a computational approach using data mining tools to detect safety signals from post-market safety databases, or in other words, to identify adverse events (AEs) with disproportionately high reporting rates compared to other AEs, associated with a particular drug or a drug class. Essentially, we view this as a problem of cluster analysis-based anomaly detection on post-market safety data, where the goal is to 'unsupervisedly' detect the anomalous or the signal AEs. Our findings demonstrate the potential of using a clustering ensemble method to detect drug safety signals. It employs multiple clustering or anomaly detection algorithms, followed by a performance comparison of the candidate algorithms based on a collection of appropriate measures of goodness of clustering results. The method is general enough to include any number of clustering or anomaly detection algorithms and goodness measures, and performs better than any of the candidate algorithms in identifying the signal AEs. Extensive simulation studies illustrate that the ensemble method detects the AE signals from synthetic post-market safety datasets pretty accurately across the different scenarios explored. Based on the cases reported to the FDA Adverse Event Reporting System (FAERS) between 2013 and 2022, we further demonstrate that the ensemble method successfully identifies and confirms most of the adverse events that are known to occur most frequently in reaction to antiepileptic drugs and -lactam antibiotics.

A clustering ensemble algorithm for handling deep embeddings using cluster confidence

Abstract Clustering ensemble, which aims to learn a robust consensus clustering from multiple weak base clusterings, has achieved promising performance on various applications. With the development of big data, the scale and complexity of data is constantly increasing. However, most existing clustering ensemble methods typically employ shallow clustering algorithms to generate base clusterings. When confronted with high-dimensional complex data, these shallow algorithms fail to fully utilize the intricate features present in the latent data space. As a result, the quality and diversity of the generated base clusterings are insufficient, thus affecting the subsequent ensemble performance. To address this issue, we propose a novel clustering ensemble algorithm for handling deep embeddings using cluster confidence (CEDECC) to improve the robustness and performance. Instead of simply combining deep clustering with clustering ensembles, we take into consideration that the performance of existing deep clustering methods heavily relies on the quality of low-dimensional embeddings generated during the pre-training stage. The quality of embeddings is unstable due to the influence of different initialization parameters. In CEDECC, specifically, we first construct a cluster confidence measure to evaluate the quality of low-dimensional embeddings. Typically, high-quality low-dimensional embeddings yield accurate clustering results with the same model parameters. Then, we utilize multiple high-quality embeddings to generate the base partitions. In the ensemble strategy phase, we consider the cluster-wise diversity and propose a novel ensemble cluster estimation to improve the overall consensus performance of the model. Extensive experiments on three benchmark datasets and four real-world biological datasets have demonstrated that the proposed CEDECC consistently outperforms the state-of-the-art clustering ensemble methods.

Identifying at-risk patients for congenital heart disease using integrated predictive models and fuzzy clustering analysis: A cross-sectional study

Congenital heart disease (CHD) remains a significant global health concern, affecting approximately 1 % of newborns worldwide. While its accurate causes often remain elusive, a combination of genetic and environmental factors is implicated. In this cross-sectional study, we propose a comprehensive prediction framework leveraging Machine Learning (ML) and Multi-Attribute Decision Making (MADM) techniques to enhance CHD diagnostics and forecasting. Our framework integrates supervised and unsupervised learning methodologies to remove data noise and address imbalanced datasets effectively. Through the utilization of imbalance ensemble methods and clustering algorithms such as K-means, we enhance predictive accuracy, particularly in non-clinical datasets where imbalances are prevalent. Our results demonstrate an improvement of 8 % in recall compared to existing literature, showcasing the efficacy of our approach. Moreover, our framework identifies clusters of patients at the highest risk using MADM techniques, providing insights into susceptibility to CHD. Fuzzy clustering techniques further assess the degree of risk for individuals within each cluster, enabling personalized risk evaluation. Importantly, our analysis reveals that unhealthy lifestyle factors, annual per capita income, nutrition, and folic acid supplementation emerge as crucial predictors of CHD occurrences. Additionally, environmental risk factors and maternal illnesses significantly contribute to the predictive model. These findings underscore the multifactorial nature of CHD development, emphasizing the importance of considering socioeconomic and lifestyle factors alongside medical variables in CHD risk assessment and prevention strategies. Our proposed framework offers a promising avenue for early identification and intervention, potentially mitigating the burden of CHD on affected individuals and healthcare systems globally.

Strategies for designing machine learning models in renewable energy with insufficient data

In the area of renewable energy including wind, solar, and electricity, Machine Learning has established a far-reaching conceptual and algorithmic design framework. However, the design of Machine Learning architectures typically relies on substantial amounts of data. The data acquisition process often involves complex tasks of gathering, cleaning, and labeling data. This challenge is exacerbated in the energy sector due to concerns related to data security and the uniqueness of development initiatives, leading to a potential scarcity of available data. Quite commonly, researchers encounter a vast array of innovative projects that have not been studied in the past and therefore only limited data become available. Therefore, there is an urgent and highly motivated need to develop machine learning strategies that can construct reliable models in the presence of limited data. In this study, three machine learning models tailored for small-scale data scenarios are developed and analyzed. The first model, a fuzzy cluster ensemble approach, focuses on utilizing small training samples treated as the core of clustering. By applying clustering algorithms, one derives membership functions reflecting the structure of data that facilitate accurate predictions. The second model, a kernel integration framework, combines kernel functions with limited training data to calculate activation levels, enabling accurate prediction outputs. For both models, the final predictions are generated by applying weighted aggregations, where the weights are derived from the respective membership grades and activation levels of the data. Additionally, a neural network contrastive learning model is introduced, which enhances the training process by employing pairwise comparisons among data points. This method effectively increases the amount of useable training data, improving the model's ability to make accurate predictions. Together, these approaches provide a robust solution for making reliable predictions even in data-scarce scenarios within renewable energy research. Moreover, the predictive performance of these models is quantified using the Root Mean Square Error criterion. This study also assesses the performance of the developed models by forming a granular characterization of results delivered by the designed models. A comprehensive suite of experimental studies is included, accompanied by an exhaustive comparative analysis. Compared to conventional machine learning models, especially neural networks, the designed models offer reliable results even with limited training data, showing an improvement of around 20 %, thus laying the groundwork for digitization processes in the realm of sustainable energy.

An ensemble approach to downscale climatological variables via screening of clustered ANNs

ABSTRACT The statistical downscaling of global circulation models presents a significant challenge in selecting appropriate input variables from a vast pool of predictors. To address the issue, we developed ensemble approach based on the Combining Multiple Clusters via Similarity Graph (COMUSA), which integrates k-means and self-organizing maps (SOMs) methods with the mutual information (MI)-random sampling approach. This innovative feature extraction technique demonstrated a 21% improvement in the classification efficacy of large-scale climatic variables. When comparing feature extraction methods, the combination of MI-random sampling and ensemble clustering yielded more accurate results than SOM clustering alone. The most efficient artificial neural networks (ANNs)-based downscaling model was employed to project near- and mid-future precipitation and temperature (2025–2035, 2035–2045), revealing varied outcomes under different scenarios (SSP3-7.0 and SSP5-8.5). Under SSP3-7.0 and SSP5-8.5, annual mean precipitation values are projected to decrease by 2–3 and by 4–5%. Also, projected annual mean temperature values indicate an increase in 21-27 and 29-35% under SSP3-7.0 and SSP5-8.5 scenarios. Integrating COMUSA ensemble clustering with MI-random sampling enhances the estimation accuracy of the ANN downscaling model, contributing to accurate projections of future precipitation and temperature values.

Auto-weighted Graph Reconstruction for efficient ensemble clustering

Ensemble clustering (EC) is an important field in data clustering, which aims to combine multiple base clustering results of a given dataset into a single consensus result. Previous studies have shown that one of the effective methods is to convert the base clustering results into a graph partitioning problem. Existing spectral ensemble clustering methods typically obtain a spectral embedding matrix for the graph and then perform k-means, separating the graph construction and clustering tasks into two distinct stages. To address these challenges, we propose a novel and efficient ensemble clustering method named Auto-weighted Graph Reconstruction for efficient ensemble clustering. This method integrates weighted base clusters, consensus distance graph construction, and clustering into a unified framework. A consensus distance graph is constructed, from which a stable and discrete label matrix is derived using spectral clustering. An effective optimization algorithm is then employed to address the resulting problem. Finally, the experimental results on eight real-world datasets verify the effectiveness and superiority of the proposed method.

ESCHR: a hyperparameter-randomized ensemble approach for robust clustering across diverse datasets

Clustering is widely used for single-cell analysis, but current methods are limited in accuracy, robustness, ease of use, and interpretability. To address these limitations, we developed an ensemble clustering method that outperforms other methods at hard clustering without the need for hyperparameter tuning. It also performs soft clustering to characterize continuum-like regions and quantify clustering uncertainty, demonstrated here by mapping the connectivity and intermediate transitions between MNIST handwritten digits and between hypothalamic tanycyte subpopulations. This hyperparameter-randomized ensemble approach improves the accuracy, robustness, ease of use, and interpretability of single-cell clustering, and may prove useful in other fields as well.

Active Clustering Ensemble With Self-Paced Learning.

A clustering ensemble provides an elegant framework to learn a consensus result from multiple prespecified clustering partitions. Though conventional clustering ensemble methods achieve promising performance in various applications, we observe that they may usually be misled by some unreliable instances due to the absence of labels. To tackle this issue, we propose a novel active clustering ensemble method, which selects the uncertain or unreliable data for querying the annotations in the process of the ensemble. To fulfill this idea, we seamlessly integrate the active clustering ensemble method into a self-paced learning framework, leading to a novel self-paced active clustering ensemble (SPACE) method. The proposed SPACE can jointly select unreliable data to label via automatically evaluating their difficulty and applying easy data to ensemble the clusterings. In this way, these two tasks can be boosted by each other, with the aim to achieve better clustering performance. The experimental results on benchmark datasets demonstrate the significant effectiveness of our method. The codes of this article are released in https://Doctor-Nobody.github.io/codes/space.zip.

Optimisation of multiple clustering based undersampling using artificial bee colony: Application to improved detection of obfuscated patterns without adversarial training

Attack detection is one of the main features required in modern defence systems. Despite the ongoing research, it remains challenging for a typical mechanism like network-based intrusion detection system (NIDS) to catch up with evolving adversarial attacks. They specifically aim to confuse a machine-learning based predictor. Without the knowledge of adversarial patterns, the best approach is generalising signatures learned from a dataset of legitimate connections and known intrusions. This work focuses on analysing non-payload traffics so that the resulting techniques can be exploited to a range of network-based applications. It investigates a novel means to deal with the problem of imbalanced classes. An optimised undersampling method is introduced to select a subset of majority-class representatives initially created through an ensemble clustering procedure. A weighted combination of criteria representing distributions within and between classes is proposed as the objective function for a global optimisation using the artificial bee colony (ABC). This approach usually outperforms its baselines and other state-of-the-art undersampling models, with ABC being more effective using the global best strategy than a random selection of solutions or an iterative greedy search. The paper also details the parameter analysis offering a heuristic guide for potential taking up of the proposed techniques.

Leveraging ensemble clustering for privacy-preserving data fusion: Analysis of big social-media data in tourism

Discovering knowledge from social media becomes a trend in many domains such as tourism, where users' feedback and rating are the basis of recommendation systems. In this context, cluster analysis has been a major tool to disclose user groups by which the process of collaborative filtering can better determine a personalised suggestion. Matching this to the curse of big data is a challenge with previous studies either implementing conventional techniques on a distributed system or making use of data sampling. Specific to ensemble clustering, only a few aim to obtain both scalability and privacy preserving that are significant to handling social data. This paper presents a new bi-level framework of ensemble clustering in which an instance-segment based analysis is adopted to ensure data privacy and reduce the complexity of clustering the whole dataset. Unlike existing studies, instead of drawing a single clustering from each segment, multiple clusterings are selected to better represent instances therein. Based on published tourism datasets and different experimental settings, the new approach usually outperforms its baselines whilst being competitive to related methods found in the literature. Additional case studies on simulated big datasets and noisy variations are reported and discussed in addition to the analysis of algorithmic parameters.

Spectral ensemble clustering with doubly stochastic co-association matrix

Although co-association (CA) matrix-based spectral ensemble clustering (SEC) has achieved significant success, it faces two persistent challenges: (1) CA matrix-based SEC may result in imbalanced cluster outcomes, and (2) SEC encounters difficulties due to the high time and space complexity resulting from the quadratic scaling of the CA matrix with the number of data points. To tackle these issues, we propose a doubly stochastic co-association (DSCA) matrix construction method. We have theoretically demonstrated that DSCA matrix-based SEC can achieve a more balanced cluster outcome compared to CA matrix-based SEC. Leveraging the DSCA matrix, we introduce two novel methods: Fast SEC (FSEC) and Approximate SEC (ASEC). FSEC efficiently solves the normalized cut on the DSCA matrix as it is symmetric and doubly stochastic. In ASEC, we employ random samples rather than the entire big dataset for ensemble clustering. After obtaining the ensemble clustering result of benchmark random sample, the approximate ensemble clustering result for all data points is derived using our newly proposed probability nearest neighbors (PNN) algorithm. Experimental results on both real-world and synthetic datasets are evaluated in terms of accuracy (ACC), normalized mutual information (NMI), purity, F1-score, and adjusted rand index (ARI) and confirm the scalability and effectiveness of the proposed FSEC and ASEC.

ELIMS: Ensemble Learning-Based Spatial Segmentation of Mass Spectrometry Imaging to Explore Metabolic Heterogeneity.

Spatial segmentation is an essential processing method for image analysis aiming to identify the characteristic suborgans or microregions from mass spectrometry imaging (MSI) data, which is critical for understanding the spatial heterogeneity of biological information and function and the underlying molecular signatures. Due to the intrinsic characteristics of MSI data including spectral nonlinearity, high-dimensionality, and large data size, the common segmentation methods lack the capability for capturing the accurate microregions associated with biological functions. Here we proposed an ensemble learning-based spatial segmentation strategy, named eLIMS, that combines a randomized unified manifold approximation and projection (r-UMAP) dimensionality reduction module for extracting significant features and an ensemble pixel clustering module for aggregating the clustering maps from r-UMAP. Three MSI datasets are used to evaluate the performance of eLIMS, including mouse fetus, human adenocarcinoma, and mouse brain. Experimental results demonstrate that the proposed method has potential in partitioning the heterogeneous tissues into several subregions associated with anatomical structure, i.e., the suborgans of the brain region in mouse fetus data are identified as dorsal pallium, midbrain, and brainstem. Furthermore, it effectively discovers critical microregions related to physiological and pathological variations offering new insight into metabolic heterogeneity.

Ensemble Clustering via Co-Association Matrix Self-Enhancement.

Ensemble clustering integrates a set of base clustering results to generate a stronger one. Existing methods usually rely on a co-association (CA) matrix that measures how many times two samples are grouped into the same cluster according to the base clusterings to achieve ensemble clustering. However, when the constructed CA matrix is of low quality, the performance will degrade. In this article, we propose a simple, yet effective CA matrix self-enhancement framework that can improve the CA matrix to achieve better clustering performance. Specifically, we first extract the high-confidence (HC) information from the base clusterings to form a sparse HC matrix. By propagating the highly reliable information of the HC matrix to the CA matrix and complementing the HC matrix according to the CA matrix simultaneously, the proposed method generates an enhanced CA matrix for better clustering. Technically, the proposed model is formulated as a symmetric constrained convex optimization problem, which is efficiently solved by an alternating iterative algorithm with convergence and global optimum theoretically guaranteed. Extensive experimental comparisons with 12 state-of-the-art methods on ten benchmark datasets substantiate the effectiveness, flexibility, and efficiency of the proposed model in ensemble clustering. The codes and datasets can be downloaded at https://github.com/Siritao/EC-CMS.

Argo data anomaly detection algorithm based on selective ensemble of fuzzy clustering

Abstract Argo profiling float data is a crucial data source for fundamental research and predictive forecasting operations in oceanography and environmental science. However, compiling and organizing such datasets demands considerable time and human resources. Therefore, the quest for effective methods of detecting anomalies in Argo data is of paramount importance. In this regard, we propose three improvement strategies within the stacking ensemble framework: preserving the original training set, weighting base model outputs, and combining the two former methods. The aim is to explore implicit relationships within the data, enhance model prediction diversity, and improve Accuracy. Additionally, in the selection of base models, to address the challenge of conventional clustering-based ensemble algorithms in achieving high levels of both diversity and accuracy among base learners, we introduce a selective ensemble method based on C-means clustering. This method selects base learners for the ensemble based on weighted scores derived from membership and performance evaluation metrics. Both of these enhancement approaches demonstrate effective application and improved detection performance when applied to Argo data.

Selective multiple kernel fuzzy clustering with locality preserved ensemble

Multiple kernel fuzzy clustering (MKFC) has demonstrated promising performance in capturing the non-linear relationships within data. However, its effectiveness relies heavily on the appropriate selection of the fuzzification coefficient and kernel functions. To address this challenge, this paper proposes a novel clustering ensemble approach for improving the robustness and accuracy of the MKFC algorithm. The proposed method employs a multi-objective evolutionary optimization approach to heuristically select the optimal fuzzification and kernel coefficients. By employing the selected coefficients, the MKFC algorithm generates a diverse set of accurate candidate clustering results. Subsequently, a locality preserved ensemble mechanism is introduced to derive the final partition matrix, ensuring that all candidate clusterings within the Pareto non-dominated set contribute to the final consensus matrix. Moreover, this mechanism incorporates the knowledge about the locality of the dataset via a graph regularization term, thereby further enhancing the clustering performance. Comprehensive experiments conducted on widely adopted benchmark datasets demonstrate the superiority of the proposed method over the state-of-the-art approaches.

The Reality of the Situation: A Survey of Situated Analytics.

The advent of low-cost, accessible, and high-performance augmented reality (AR) has shed light on a situated form of analytics where in-situ visualizations embedded in the real world can facilitate sensemaking based on the user's physical location. In this work, we identify prior literature in this emerging field with a focus on situated analytics. After collecting 47 relevant situated analytics systems, we classify them using a taxonomy of three dimensions: situating triggers, view situatedness, and data depiction. We then identify four archetypical patterns in our classification using an ensemble cluster analysis. We also assess the level which these systems support the sensemaking process. Finally, we discuss insights and design guidelines that we learned from our analysis.

Radial Basis Function Neural Network with ensemble clustering for modeling mathematics achievement in Indonesia based on cognitive and non-cognitive factors

Mathematics achievement could be influenced by cognitive and non-cognitive factors. The potential variable of cognitive factor is metacognition, whereas non-cognitive factors include Economic, Social, and Cultural Status (ESCS), resilience, life satisfaction, happiness, pride, fear, sadness, and gender. Those variables involve numerical and categorical data. For this reason, this study aims to apply the Radial Basis Function Neural Network (RBFNN) model with ensemble clustering to model the relation between cognitive and non-cognitive aspects and mathematics achievement. The RBFNN is a soft computing approach based on the neural network model and has been shown as an effective model and free of assumption. The ensemble clustering is a process in RBFNN modeling to capture the independent variables involving the numerical and categorical data. It employs K-means clustering for the numerical data and K-modes for categorical data and combines the results of those two methods. The data used in this study are published by PISA (Program for International Student Assessment) 2018. The results show that the RBFNN with ensemble clustering deliver good performance in modeling the students’ mathematics achievement based on the cognitive and non-cognitive factors in terms of prediction accuracy. Other than RBFNN model, the use of cognitive and non-cognitive factors involving in this study also contributes to the high accuracy prediction. This further emphasizes that these factors are good predictors of mathematic achievement. Additionally, we suggest the silhouette cluster validation in the clustering process, since it leads to the number of hidden neurons of the best RBFNN model.