Clustering Performance Research Articles

A mini-review of single-cell Hi-C embedding methods

Single-cell Hi-C (scHi-C) techniques have significantly advanced our understanding of the 3D genome organization, providing crucial insights into the spatial genome architecture within individual nuclei. Numerous computational and statistical methods have been developed to analyze scHi-C data, with embedding methods playing a key role. Embedding reduces the dimensionality of complex scHi-C contact maps, making it easier to extract biologically meaningful patterns. These methods not only enhance cell clustering based on chromatin structures but also facilitate visualization and other downstream analyses. Most scHi-C embedding methods incorporate strategies such as normalization and imputation to address the inherent sparsity of scHi-C data, thereby further improving data quality and interpretability. In this review, we systematically examine the existing methods designed for scHi-C embedding, outlining their methodologies and discussing their capabilities in handling normalization and imputation. Additionally, we present a comprehensive benchmarking analysis to compare both embedding techniques and their clustering performances. This review serves as a practical guide for researchers seeking to select suitable scHi-C embedding tools, ultimately contributing to the understanding of the 3D organization of the genome.

Identifying Landslide Hotspots Using Unsupervised Clustering: A Case Study

Landslides pose significant threats to life, property, and infrastructure. This study explores applying unsupervised learning techniques to identify and understand landslide-prone areas. We analyzed topographic data by employing K-Means, Hierarchical Clustering, Spectral Clustering, Mean Shift Clustering, and DBSCAN to uncover hidden patterns in landslide occurrence. Evaluation metrics, including the Silhouette Score, Davies-Bouldin Index, and Calinski-Harabasz Index, were used to assess the performance of these algorithms. Hierarchical Clustering achieved the highest Silhouette Score of 0.635, indicating excellent cluster separation. However, Mean Shift Clustering outperformed the other methods with a superior Davies-Bouldin Index of 0.603 and the highest Calinski-Harabasz Index of 4121.75, demonstrating the best overall clustering performance. DBSCAN also performed well, with a Silhouette Score of 0.610 and 12 noise points identified. These findings contribute to a deeper understanding of landslide spatial distribution and can inform the development of effective early warning systems and mitigation strategies.

Green Fruit‐Stem Pairing and Clustering for Machine Vision System in Robotic Thinning of Apples

ABSTRACTApples are one of the most highly‐valued specialty crops in the United States. Recent labor shortages have made crop production difficult for fruit growers, including the task of green fruit thinning. Current methods including hand, chemical, and mechanical thinning impose tradeoffs between selectivity, cost, tree damage, and speed. A robotic green fruit thinning system could potentially selectively thin fruit in a quick, cost‐effective, and non‐damaging manner. The machine vision system would be a critical component for robotic thinning, and would not only need to perform green fruit detection/segmentation, but also fruit‐stem pairing and clustering to facilitate proper decision‐making for thinning. A neural network‐based fruit and stem pairing algorithm was devised and evaluated; an LSTM‐based clustering algorithm was devised and compared to the density‐based clustering algorithm, OPTICS. The algorithms were evaluated on an image data set consisting of GoldRush, Fuji, and Golden Delicious cultivars at the green fruit stage, with evaluations on overall performance, cultivar‐wise performance, cluster size‐specific performance, and feature importance. For fruit and stem pairing, the neural network‐based pairing algorithm achieved an AP of 81.4% on all fruits and stems, and that reached 90.6% when only fruits and stems with labeled angles were considered. For green fruit clustering, the LSTM‐based clustering achieved a clustering success rate of 68.4%, whereas the OPTICS algorithm obtained 50.9%. The algorithms will be further implemented in a pipeline of a future green fruit thinning vision system, as well as integrate the use of point clouds and other 3D orchard information to improve pairing and clustering performance.

Euler Kernel Mapping for Hyperspectral Image Clustering via Self-Paced Learning

Clustering, as a classical unsupervised artificial intelligence technology, is commonly employed for hyperspectral image clustering tasks. However, most existing clustering methods designed for remote sensing tasks aim to solve a non-convex objective function, which can be optimized iteratively, beginning with random initializations. Consequently, during the learning phase of the clustering model, it may easily fall into bad local optimal solutions and finally hurt the clustering performance. Additionally, prevailing approaches often exhibit limitations in capturing the intricate structures inherent in hyperspectral images and are very sensitive to noise and outliers that widely exist in remote sensing data. To address these issues, we proposed a novel Euler kernel mapping for hyperspectral image clustering via self-paced learning (EKM-SPL). EKM-SPL first employs self-paced learning to learn the clustering model in a meaningful order by progressing samples from easy to complex, which can help to remove bad local optimal solutions. Secondly, a probabilistic soft weighting scheme is employed to measure complexity across the data sample, which makes the optimization process more reasonable. Thirdly, in order to more accurately characterize the intricate structure of hyperspectral images, Euler kernel mapping is used to convert the original data into a reproduced kernel Hilbert space, where the nonlinearly inseparable clusters may become linearly separable. Moreover, we innovatively integrate the coordinate descent technique into the optimization algorithm to circumvent the computational inefficiencies and information loss typically associated with conventional kernel methods. Extensive experiments conducted on classic benchmark hyperspectral image datasets illustrate the effectiveness and superiority of our proposed model.

A Dirichlet Process Model for Directional-linear Data with Application to Bloodstain Pattern Analysis

Directional data require specialized models because of the non-Euclidean nature of their domain. When a directional variable is observed jointly with linear variables, modeling their dependence adds an additional layer of complexity. A Bayesian nonparametric approach is introduced to analyze directional-linear data. Firstly, the projected normal distribution is extended to model the joint distribution of linear variables and a directional variable with arbitrary dimension projected from a higher-dimensional augmented multivariate normal distribution. The new distribution is called the semi-projected normal distribution (SPN) and can be used as the mixture distribution in a Dirichlet process model to obtain a more flexible class of models for directional-linear data. Then, a conditional inverse-Wishart distribution is proposed as part of the prior distribution to address an identifiability issue inherited from the projected normal and preserve conjugacy with the SPN. The SPN mixture model shows superior performance in clustering on synthetic data compared to the semi-wrapped Gaussian model. The experiments show the ability of the SPN mixture model to characterize bloodstain patterns. A hierarchical Dirichlet process model with the SPN distribution is built to estimate the likelihood of bloodstain patterns under a posited causal mechanism for use in a likelihood ratio approach to the analysis of forensic bloodstain pattern evidence.

Deep Incomplete Multi-view Clustering via Multi-level Imputation and Contrastive Alignment

Deep incomplete multi-view clustering (DIMVC) aims to enhance clustering performance by capturing consistent information from incomplete multiple views using deep models. Most existing DIMVC methods typically employ imputation-based strategies to handle missing views before clustering. However, they often assume complete data availability across all views, overlook potential low-quality views, and perform imputation at a single data level, leading to challenges in accurately inferring missing data. To address these issues, we propose a novel imputation-based approach called Multi-level Imputation and Contrastive Alignment (MICA) to simultaneously improve imputation quality and boost clustering performance. Specifically, MICA employs an individual deep model for each view, which unifies view feature learning and cluster assignment prediction. It leverages the learned features from available instances to construct an adaptive cross-view graph for reliable view selection. Guided by these reliable views, MICA performs multi-level (feature-level, data-level, and reconstruction-level) imputation to preserve topological structures across levels and ensure accurate missing feature inference. The complete features are then used for discriminative cluster assignment learning. Additionally, an instance- and cluster-level contrastive alignment is conducted on the cluster assignments to further enhance semantic consistency across views. Experimental results show the effectiveness and superior performance of the proposed MICA method.

A Novel k-Means Framework via Constrained Relaxation and Spectral Rotation.

Owing to its simplicity, the traditional k - means (Lloyd heuristic) clustering method plays a vital role in a variety of machine-learning applications. Disappointingly, the Lloyd heuristic is prone to local minima. In this article, we propose k - mRSR, which converts the sum-of-squared error (SSE) (Lloyd) into a combinatorial optimization problem and incorporates a relaxed trace maximization term and an improved spectral rotation term. The main advantage of k - mRSR is that it only needs to solve the membership matrix instead of computing the cluster centers in each iteration. Furthermore, we present a nonredundant coordinate descent method that brings the discrete solution infinitely close to the scaled partition matrix. Two novel findings from the experiments are that k - mRSR can further decrease (increase) the objective function values of the k - means obtained by Lloyd (CD), while Lloyd (CD) cannot decrease (increase) the objective function obtained by k - mRSR. In addition, the results of extensive experiments on 15 datasets indicate that k - mRSR outperforms both Lloyd and CD in terms of the objective function value and outperforms other state-of-the-art methods in terms of clustering performance.

Structured multi-view k-means clustering

K-means is a very efficient clustering method and many multi-view k-means clustering methods have been proposed for multi-view clustering during the past decade. However, since k-means have trouble uncovering clusters of varying sizes and densities, these methods suffer from the same performance issues as k-means. Improving the clustering performance of multi-view k-means has become a challenging problem. In this paper, we propose a new multi-view k-means clustering method that is able to uncover clusters in arbitrary sizes and densities. The new method simultaneously performs three tasks, i.e., sparse connection probability matrices learning, prototypes aligning, and cluster structure learning. We evaluate the proposed new method by 5 benchmark datasets and compare it with 11 multi-view clustering methods. The experimental results on both synthetic and real-world experiments show the superiority of our proposed method.

Viewpoint-Based Collaborative Feature-Weighted Multi-View Intuitionistic Fuzzy Clustering Using Neighborhood Information

This paper presents an intuitionistic fuzzy c-means-based clustering algorithm for multi-view clustering, addressing key challenges such as noise sensitivity, outlier influence, and the distinct importance of views, features, and samples. Our proposed approach incorporates view weights, feature weights, sample weights, and neighborhood information into a novel objective function. Additionally, we introduce an effective initial cluster center selection strategy that enhances clustering robustness. The efficiency of the proposed method is evaluated using various clustering criteria (AR, NMI, RI, FMI, and JI). Moreover, the effect of each module of the algorithm on the general clustering performance is examined exclusively. Experimental results on various benchmark multi-view datasets demonstrate that our algorithm outperforms state-of-the-art methods in terms of clustering accuracy and stability.

Improved indoor localization using k‐medoids and k‐nearest neighbour algorithms with context similarity coefficient‐based fingerprint similarity metric

AbstractFingerprint database clustering and localization using k‐medoids and k‐nearest neighbour (k‐NN) algorithms respectively typically use distance‐based fingerprint similarity metrics, with their performances dependent on the type of distance metric used. This paper proposes employing a pattern‐based metric, the context similarity coefficient (CSC), for both algorithms instead of traditional distance‐based metrics. The CSC accounts for fingerprint behaviour and the non‐linear relationships among fingerprints during the similarity measurement. The performance of both algorithms with the CSC as the similarity metric is evaluated on four publicly available fingerprint databases, using position root mean square error (RMSE) and silhouette score as performance metrics. These results are compared to those of the same algorithms using five distance‐based metrics: Euclidean, square Euclidean, Manhattan, cosine, and Chebyshev distances. The k‐medoids algorithm with CSC shows moderate clustering performance compared to the five distance‐based metrics considered. However, when combined with the k‐NN algorithm also using CSC, it achieves the highest localization accuracy, with at least a 29% improvement in position RMSE across all four databases. The results indicate that while k‐medoids with CSC may not create well‐separated clusters, combining it with the k‐NN algorithm with CSC as its similarity metric significantly enhances localization accuracy compared to distance‐based metrics.

Artificial Algae Algorithm for Clustering of Benchmark Datasets

The best solution found for a problem under specific circumstances is called optimization. Algorithms for optimization can make the best use of the information at their disposal. Numerous optimization algorithms have been created thus far by researchers, and most of these algorithms are based on the characteristics of naturally occurring biological organisms. Optimization algorithms have proven to be highly effective in numerous fields, including finance, engineering, and medical. Apart from these applications, they have also been employed in data mining techniques including clustering and classification. In many different domains, the clustering method is widely applied. Finding the optimum cluster centers is the most crucial step in the clustering process. In this study, the Artificial Algae Algorithm (AAA) is used to perform the clustering procedure by using 12 datasets that were taken from the UCI Machine Learning Repository. For every dataset, the squared distance values between the cluster centers and the data were computed in order to assess the effectiveness of AAA. The study evaluated AAA's performance against that of the ALO, DEA, MFO, PSO, TSA, and WOA algorithms. The clustering performance of AAA on benchmark datasets was measured to be better than the performance of other algorithms.

Enhancing computational thinking in children through logical puzzles: A machine learning approach for early coding education

This recent growth in early education in coding evidences the rising need for the computing-thinking-skills development in children. Though most of the programming curricula are based upon syntax and other basic concepts of coding, computational thinking encompasses broader cognitive skills, such as problem-solving, abstraction, and logical reasoning. This research therefore intends to fill a gap in research and will investigate the ways pre-coding games employ logical puzzles as an enhancement in developing computational thinking skills in children aged 7-10 years and how these affect the acquisition of programming knowledge. In all, there will be two groups assigned: one will be exposed to logical-oriented puzzle training, Group A, and the other will be a control group, Group B, which receives only regular standard lessons in programming introduction. Pre- and post-measures of computational thinking and basic programming skills will be obtained from the participants. The machine learning algorithms, such as Random Forest for prediction in terms of improvement in subject performance in computational thinking and K-Means clustering to identify patterns in their development of skills, will be implemented in their pure or integrated forms to analyze data. Logistic Regression will be executed in order to model the odds of improved performance in programming given an intervention of exposure to puzzles. The research study will, therefore, apply these algorithms in comparing the performance of the two groups and reveal the aspects of computational thinking to which the greatest influence in a positive way through puzzle activities is exerted. It is expected that children exposed to logical puzzles may show significant improvements in both computational thinking and programming skills. These findings should be useful to educators in their effort to show how logical puzzles can be integrated into the curricula of early coding education to build foundational skills and thereby contribute to a refinement of best practices in STEM education.

Clustering by detecting skeletal structure and identifying density fluctuation

Clustering is one of the most important techniques for unsupervised learning, it tries to divide points into different clusters without any priori knowledge of data. Therefore, several criterions for clustering algorithm are as follows: 1. Handling clusters with arbitrary shape and various density; 2. Finding cluster centers automatically; 3. Low parameter sensitivity and computational complexity. In this context, a novel algorithm namely clustering by detecting skeletal structure and identifying density fluctuation (CSSDF) was presented. In CSSDF, an efficient strategy based on density and local information of neighborhood is firstly proposed to detect the skeletal structure, which can collect the local information and identify the rough distribution of data. With the identified distribution information, a method takes expanded neighborhood and density fluctuation into consideration is proposed to further collect global information of data, which can assign all skeleton points and find cluster centers. To sum up, CSSDF can not only discover the underlying structure of data regardless of its’ distribution, but also ensure the correct assignment of all skeleton points and thus lead to a satisfying clustering performance. In addition, the computational complexity of the proposed approach is O(nlogn), which makes it possible to deal with some large clustering problem.

Augmented ELBO regularization for enhanced clustering in variational autoencoders

With significant advances in deep neural networks, various new algorithms have emerged that effectively model latent structures within data, surpassing traditional clustering methods. Each data point is expected to belong to a single cluster in a typical clustering algorithm. However, clustering based on variational autoencoders (VAEs) represents the expectation of the overall clusters, denoted as c=1,…,K in the KL divergence term. Consequently, the latent embedding z can be learned to exist across multiple clusters with relatively balanced probabilities, rather than being strongly associated with a specific cluster. This study introduces an additional regularizer to encourage the latent embedding z to have a strong affiliation with specific clusters. We introduce optimization methods to maximize the ELBO that includes the newly added regularization term and explore methods to eliminate computationally challenging terms. The positive impact of this regularization on clustering accuracy was verified by examining the variance of the final cluster probabilities. Furthermore, an enhancement in the clustering performance was observed when regularization was introduced.

Optimization study of diesel engine emission prediction based on neural network model cluster

This study aims to reduce the testing volume and cost of engine bench tests. By combining neural network models and ensemble learning algorithms, a model cluster prediction method is proposed to predict engine emissions. The core of this method is to use a large number of neural network models for prediction, taking the average of the prediction results as the final prediction result, thereby reducing prediction errors and improving accuracy. The findings reveal that this cluster-based approach significantly outperforms a single optimal model in forecasting steady-state emissions of NOx, HC, and CO in diesel engines. Notably, the performance of the model cluster is highly dependent on both the quality and the number of constituent sub-models, with enhanced predictive capabilities observed when higher-quality sub-models are included. Additionally, the study identifies a stabilization in predictive accuracy as the number of models increases, particularly within the range of 50 to 100 models, and achieving robust stability beyond 100 models. The research also underscores the importance of the training dataset size on the effectiveness of neural network modeling. A reduction in the training dataset from 28% to 18% leads to a decline in the coefficient of determination ( R2) for NOx emissions prediction in a single neural network model from 0.6954 to 0.6539, a 5.97% decrease. For the model cluster method, the R2 similarly drops from 0.8633 to 0.8154, a reduction of 5.55%. Notably, the neural network model suffers from distortion in predicting the peak position of NOx emissions. After supplementing specific operating condition training data in a targeted manner, the model training amount was increased from 18% to 25%. The model cluster method showed a good improvement in the prediction of NOx emissions, with the prediction coefficient increasing from 0.8154 to 0.8997, with an improvement rate of 10.34%. Through planning of training data, the study demonstrates that employing just 25% of experimental data for model clustering can effectively predict the engine emission trends for the remaining 75% of data in the target conditions. This approach offers a substantial reduction in experimental requirements while maintaining high prediction accuracy.

Mcadet: A feature selection method for fine-resolution single-cell RNA-seq data based on multiple correspondence analysis and community detection.

Single-cell RNA sequencing (scRNA-seq) data analysis faces numerous challenges, including high sparsity, a high-dimensional feature space, and biological noise. These challenges hinder downstream analysis, necessitating the use of feature selection methods to identify informative genes, and reduce data dimensionality. However, existing methods for selecting highly variable genes (HVGs) exhibit limited overlap and inconsistent clustering performance across benchmark datasets. Moreover, these methods often struggle to accurately select HVGs from fine-resolution scRNA-seq datasets and minority cell types, which are more difficult to distinguish, raising concerns about the reliability of their results. To overcome these limitations, we propose a novel feature selection framework for scRNA-seq data called Mcadet. Mcadet integrates Multiple Correspondence Analysis (MCA), graph-based community detection, and a novel statistical testing approach. To assess the effectiveness of Mcadet, we conducted extensive evaluations using both simulated and real-world data, employing unbiased metrics for comparison. Our results demonstrate the superior performance of Mcadet in the selection of HVGs in scenarios involving fine-resolution scRNA-seq datasets and datasets containing minority cell populations. Overall, we demonstrate that Mcadet enhances the reliability of selected HVGs, although the impact of HVG selection on various downstream analyses varies and needs to be further investigated.

Wavelet-Entropy Enhanced Clustering: A Comprehensive Analysis of Drought Patterns in the Southern Plains, USA

Abstract Droughts may exhibit spatiotemporal heterogeneity at regional scale. Effective drought assessment and management necessitates identifying homogeneous areas. However, previous studies often simplified clustering analysis by focusing only on a single variable. In this study, we present a novel drought risk map for the Southern Plains (SP) region of the United States by integrating the wavelet-entropy approach with k-means clustering algorithm to capture spatio-temporal patterns of drought-related variables across various resolutions while eliminating redundant information. We considered multiple drought indicators and indices including gridded precipitation (P), potential evapotranspiration (PET), normalized difference vegetation index (NDVI), and Standardized Precipitation Evapotranspiration Index (SPEI) as well as geographical coordinates and topography map. Through evaluating five different combinations of input datasets, we selected the one demonstrating optimal results based on Davies-Bouldin and Calinski-Harabasz criteria. In addition to P, PET, and NDVI, including the coordinates and elevation as secondary variables significantly enhanced the clustering performance. Using these variables, the region was subdivided into 21 clusters. The Pearson’s correlation coefficients for the SPEI between centroid members and corresponding cells within clusters averaged between 0.84 to 0.94. Comparison with an existing cluster map (DRA) for the region revealed that our proposed cluster map showed higher variability between clusters for P, PET, and NDVI, confirming the robustness of the clustering results for drought conditions in the SP. The new clustering framework is expected to provide valuable insights for understanding and addressing drought dynamics in the SP region.

Adding Highly Variable Genes to Spatially Variable Genes Can Improve Cell Type Clustering Performance in Spatial Transcriptomics Data.

Spatial transcriptomics has allowed researchers to analyze transcriptome data in its tissue sample's spatial context. Various methods have been developed for detecting spatially variable genes (SV genes), whose gene expression over the tissue space shows strong spatial autocorrelation. Such genes are often used to define clusters in cells or spots downstream. However, highly variable (HV) genes, whose quantitative gene expressions show significant variation from cell to cell, are conventionally used in clustering analyses. In this report, we investigate whether adding highly variable genes to spatially variable genes can improve the cell type clustering performance in spatial transcriptomics data. We tested the clustering performance of HV genes, SV genes, and the union of both gene sets (concatenation) on over 50 real spatial transcriptomics datasets across multiple platforms, using a variety of spatial and non-spatial metrics. Our results show that combining HV genes and SV genes can improve overall cell-type clustering performance.

A comparative analysis of unsupervised machine-learning methods in PSG-related phenotyping.

Obstructive sleep apnea is a heterogeneous sleep disorder with varying phenotypes. Several studies have already performed cluster analyses to discover various obstructive sleep apnea phenotypic clusters. However, the selection of the clustering method might affect the outputs. Consequently, it is unclear whether similar obstructive sleep apnea clusters can be reproduced using different clustering methods. In this study, we applied four well-known clustering methods: Agglomerative Hierarchical Clustering; K-means; Fuzzy C-means; and Gaussian Mixture Model to a population of 865 suspected obstructive sleep apnea patients. By creating five clusters with each method, we examined the effect of clustering methods on forming obstructive sleep apnea clusters and the differences in their physiological characteristics. We utilized a visualization technique to indicate the cluster formations, Cohen's kappa statistics to find the similarity and agreement between clustering methods, and performance evaluation to compare the clustering performance. As a result, two out of five clusters were distinctly different with all four methods, while three other clusters exhibited overlapping features across all methods. In terms of agreement, Fuzzy C-means and K-means had the strongest (κ = 0.87), and Agglomerative hierarchical clustering and Gaussian Mixture Model had the weakest agreement (κ = 0.51) between each other. The K-means showed the best clustering performance, followed by the Fuzzy C-means in most evaluation criteria. Moreover, Fuzzy C-means showed the greatest potential in handling overlapping clusters compared with other methods. In conclusion, we revealed a direct impact of clustering method selection on the formation and physiological characteristics of obstructive sleep apnea clusters. In addition, we highlighted the capability of soft clustering methods, particularly Fuzzy C-means, in the application of obstructive sleep apnea phenotyping.

Joint channel allocation and transmit power control for underlay EH‐CRNs: A clustering‐based multi‐agent DDPG approach

AbstractTo address the concerns of energy supply and spectrum scarcity for wireless devices, energy harvesting cognitive radio networks have been proposed. To improve spectrum utilization, secondary users (SUs) access the licensed spectrum in underlay mode, which may cause severe interference to primary users and SUs. The focus is on the underlay energy harvesting cognitive radio networks with multiple pairs of SUs, and formulate the long‐term secondary throughput maximization problem as a mixed‐integer non‐linear programming problem. As traditional approaches could hardly solve the mixed‐integer non‐linear programming problem well, a centralized deep deterministic policy gradient (C‐DDPG) approach is proposed that achieves satisfactory throughput performance. To reduce the computational complexity of C‐DDPG, we further propose a clustering‐based multi‐agent DDPG (CMA‐DDPG) approach that combines the advantages of the centralized deep reinforcement learning approach and the distributed deep reinforcement learning approach. In the CMA‐DDPG, a novel interference‐based clustering algorithm is proposed, which partitions the SUs that cause severe mutual interference into one cluster, and the sizes of state space and action space are smaller than those in C‐DDPG. Numerical results validate the superiority of the proposed approaches in terms of the throughput and outage probability, and validate the clustering performance of the interference‐based clustering algorithm in terms of the outage probability of the secondary network.