Improve Clustering Performance Research Articles

Multiple kernel clustering with local kernel reconstruction and global heat diffusion

Multiple Kernel Clustering (MKC) is an effective approach for revealing nonlinear cluster structures in candidate kernels. However, existing MKC methods still face two key challenges. Firstly, the pairwise affinity in these methods is primarily determined by kernel similarity, disregarding the correlations among highly similar neighbors and resulting in redundant weight assignments and reduced clustering discriminability. Secondly, the direct utilization of affinity matrices overlooks high-order connections and introduces noise due to independent row-wise solving. To address these issues, we propose a novel local MKC method called LKRGDF. We begin by exploring affinity using the Local Kernel Reconstruction (LKR) model, reducing redundancy and enhancing clustering discriminability. Furthermore, we exploit the affinities with the Global heat kernel Diffusion (GD) procedure to capture long-range connections smoothly. The GD process acts as a low pass filter, focusing on small eigenvalues corresponding to top clusters. Finally, we integrate these smooth affinities within an auto-weighted Multiple Graph Fusion (MGF) framework to obtain a consensus graph. By assembling LKR, GD, and MGF in a sequential pipeline, our approach achieves the exploration and exploitation of local structures, gradually improving clustering performance while ensuring computational efficiency without the need for iterative steps. Extensive experiments on ten datasets demonstrate the superiority of our algorithm in terms of effectiveness and efficiency compared to state-of-the-art methods. The code for our method is publicly available at https://github.com/YanChenSCU/LKRGDF-2023.git.

Collaborative Decision-Reinforced Self-Supervision for Attributed Graph Clustering.

Attributed graph clustering aims to partition nodes of a graph structure into different groups. Recent works usually use variational graph autoencoder (VGAE) to make the node representations obey a specific distribution. Although they have shown promising results, how to introduce supervised information to guide the representation learning of graph nodes and improve clustering performance is still an open problem. In this article, we propose a Collaborative Decision-Reinforced Self-Supervision (CDRS) method to solve the problem, in which a pseudo node classification task collaborates with the clustering task to enhance the representation learning of graph nodes. First, a transformation module is used to enable end-to-end training of existing methods based on VGAE. Second, the pseudo node classification task is introduced into the network through multitask learning to make classification decisions for graph nodes. The graph nodes that have consistent decisions on clustering and pseudo node classification are added to a pseudo-label set, which can provide fruitful self-supervision for subsequent training. This pseudo-label set is gradually augmented during training, thus reinforcing the generalization capability of the network. Finally, we investigate different sorting strategies to further improve the quality of the pseudo-label set. Extensive experiments on multiple datasets show that the proposed method achieves outstanding performance compared with state-of-the-art methods. Our code is available at https://github.com/Jillian555/TNNLS_CDRS.

ScGCC: Graph Contrastive Clustering With Neighborhood Augmentations for scRNA-Seq Data Analysis.

Single-cell RNA sequencing (scRNA-seq) has rapidly emerged as a powerful technique for analyzing cellular heterogeneity at the individual cell level. In the analysis of scRNA-seq data, cell clustering is a critical step in downstream analysis, as it enables the identification of cell types and the discovery of novel cell subtypes. However, the characteristics of scRNA-seq data, such as high dimensionality and sparsity, dropout events and batch effects, present significant computational challenges for clustering analysis. In this study, we propose scGCC, a novel graph self-supervised contrastive learning model, to address the challenges faced in scRNA-seq data analysis. scGCC comprises two main components: a representation learning module and a clustering module. The scRNA-seq data is first fed into a representation learning module for training, which is then used for data classification through a clustering module. scGCC can learn low-dimensional denoised embeddings, which is advantageous for our clustering task. We introduce Graph Attention Networks (GAT) for cell representation learning, which enables better feature extraction and improved clustering accuracy. Additionally, we propose five data augmentation methods to improve clustering performance by increasing data diversity and reducing overfitting. These methods enhance the robustness of clustering results. Our experimental study on 14 real-world datasets has demonstrated that our model achieves extraordinary accuracy and robustness. We also perform downstream tasks, including batch effect removal, trajectory inference, and marker genes analysis, to verify the biological effectiveness of our model.

Optimizing environmental monitoring in IoT: integrating DBSCAN with genetic algorithms for enhanced clustering

In our study, we introduce an advanced clustering method designed for IoT-based environmental monitoring. We’ve combined two powerful techniques, Density-Based Spatial Clustering of Applications with Noise (DBSCAN) and Genetic Algorithms (GA), to create a specialized approach called EC-GAD (Enhanced-Clustering using Genetic Algorithms and DBSCAN). This integrated system model relies on DBSCAN, a robust clustering algorithm capable of handling irregular shapes and varying data densities, to group sensor nodes based on their physical proximity. To improve clustering performance, we’ve harnessed Genetic Algorithms to optimize the parameters of DBSCAN. Through a repetitive process involving selection, crossover, and mutation, GA refines parameter settings based on the quality of environmental clustering as assessed by fitness metrics. Our approach is tailored specifically for IoT deployments in environmental monitoring, which involve collecting data from sensor nodes and integrating DBSCAN and GA. We’ve paid special attention to choosing an appropriate distance metric and fine-tuning DBSCAN parameters such as epsilon (ε) and minPts to match the unique needs of environmental monitoring applications. Furthermore, we’ve taken energy efficiency into account by implementing energy-aware node selection and optimizing cluster formation to minimize energy consumption.

A feature-weighted suppressed possibilistic fuzzy c-means clustering algorithm and its application on color image segmentation

The possibilistic fuzzy c-means clustering (PFCM) algorithm is a hybridization of possibilistic c-means clustering (PCM) and fuzzy c-means clustering (FCM) algorithms. However, there are two main problems in PFCM. One is that the Euclidean distance employed in PFCM always disregards the imbalance among sample features because it treats all features of data equally, which easily causes misclassification for feature-imbalanced multidimensional data. The other is that PFCM always produces significant center deviations and overlapping centers for multiclass datasets with strong noise injection, due to the difficulty of PFCM in the membership-weight parameter setting and the lack of between-class relationships in possibilistic memberships. Therefore, a feature-weighted suppressed possibilistic fuzzy c-means clustering (FW-S-PFCM) algorithm is presented by introducing a feature-weighted method and “suppressed competitive learning” strategy into the PFCM algorithm in this paper. First, the FW-S-PFCM algorithm introduces a feature-weight matrix into the objective function that can automatically assign feature-weight values to different features and different clusters according to the distribution of samples, thus overcoming the influence of feature imbalance and improving clustering effects for noisy multidimensional datasets. Second, combined with the feature-weight matrix, a “suppressed competitive learning” strategy is designed to resolve the center-overlapping problem in noisy multiclass dataset clustering. Specifically, partial crucial points of each class near the center are selected according to a cluster core generated by a cross-section of a threshold on the possibilistic membership surface. Third, their possibilistic memberships participate in the suppressed learning process to overcome the lack of between-class relationships. Last, a segmentation algorithm for noisy color images based on FW-S-PFCM is proposed combined with the feature-weight method and noise-identification ability of possibilistic memberships. Experiments on synthetic data, UCI data and color image segmentation demonstrate that the proposed FW-S-PFCM algorithm can overcome the partial center-overlapping problem and improve clustering performance on complex datasets with feature imbalance and strong noise injection. The proposed algorithm can also reduce the iteration number, sensitivity to membership weights, and initializations of PFCM.

Multi-View Clustering With Graph Learning for scRNA-Seq Data.

Advances in single-cell biotechnologies have generated the single-cell RNA sequencing (scRNA-seq) of gene expression profiles at cell levels, providing an opportunity to study cellular distribution. Although significant efforts developed in their analysis, many problems remain in studying cell types distribution because of the heterogeneity, high dimensionality, and noise of scRNA-seq. In this study, a multi-view clustering with graph learning algorithm (MCGL) for scRNA-seq data is proposed, which consists of multi-view learning, graph learning, and cell type clustering. In order to avoid a single feature space of scRNA-seq being inadequate to comprehensively characterize the functions of cells, MCGL constructs the multiple feature spaces and utilizes multi-view learning to comprehensively characterize scRNA-seq data from different perspectives. MCGL adaptively learns the similarity graphs of cells that overcome the dependence on fixed similarity, transforming scRNA-seq analysis into the analysis of multi-view clustering. MCGL decomposes the networks of cells into view-specific and common networks in multi-view learning, which better characterizes the topological relationship of cells. MCGL simultaneously utilizes multiple types of cell-cell networks and fully exploits the connection relationship between cells through the complementarity between networks to improve clustering performance. The graph learning, graph factorization, and cell-type clustering processes are accomplished simultaneously under one optimization framework. The performance of the MCGL algorithm is validated with ten scRNA-seq datasets from different scales, and experimental results imply that the proposed algorithm significantly outperforms fourteen state-of-the-art scRNA-seq algorithms.

Knowledge-aware progressive clustering for social image

Social image data refer to the annotated image with tags in social media, in which the tags are always labeled by users. Integrating the visual and textual information of social image can obtain accurate and comprehensive feature and improve clustering performance. However, the heterogeneous gap between tags and images makes it difficult to reasonably organize the social images. In addition, the tags are often sparse and incomplete due to personal preference and cognition differences of users. To solve these problems, we propose a novel knowledge-aware progressive clustering (KAPC) method, which employs human knowledge to guide the cross-modal clustering of social images. Firstly, we design a dual-similarity semantic expansion strategy to complement the sparse tags with human knowledge, which constructs a more complete semantic similarity matrix for tags through knowledge graphs. Secondly, we define an objective function based on information theory to bridge the heterogeneous gap, which align inter-modal cluster distribution to explore the correlation between visual and textual information. Finally, a progressive iteration method is designed to make the two modalities guide each other and obtain better performance of social image clustering. Extensive experiments on four social image datasets verify the effectiveness of the proposed KAPC method.

Decomposed deep multi-view subspace clustering with self-labeling supervision

Most deep multi-view subspace clustering (DMVSC) methods usually employ pipeline optimization by learning the self-expression with a deep model first and then applying spectral clustering to group multi-view data. Subsequently, end-to-end DMVSC methods have been proposed by integrating these two steps into a unified optimization framework. However, the pipeline methods may suffer from misaligned clustering accumulation due to the noise or outlying entries, and the end-to-end methods often constitute a relatively complex parameter optimization. In this paper, we propose a novel method named decomposed deep multi-view subspace clustering with self-labeling supervision (D2MVSC) that runs with a decomposed optimization strategy by three-stage training. Specifically, multi-scale features are extracted by autoencoder in the pre-training stage. According to the discriminative contribution of each view, consensus self-expression is learned from these features by adaptive fusion and structure supervision to generate high-quality pseudo-labels in the fine-tuning stage. Finally, the pseudo-labels are used to retrain the model in a self-labeling supervision manner for robust clustering. Exciting, the self-labeling supervision can be used as an add-on module for other DMVSC methods to improve clustering performance. Extensive experiments on six datasets verify the effectiveness and superiority of our method over other state-of-the-art methods.

NonPC: Non-parametric clustering algorithm with adaptive noise detecting

Graph-based clustering performs efficiently for identifying clusters in local and nonlinear data Patterns. The existing methods face the problem of parameter selection, such as the setting of k of the k-nearest neighbor graph and the threshold in noise detection. In this paper, a non-parametric clustering algorithm (NonPC) is proposed to tackle those inherent limitations and improve clustering performance. The weighted natural neighbor graph (wNaNG) is developed to represent the given data without any prior knowledge. What is more, the proposed NonPC method adaptively detects noise data in an unsupervised way based on some attributes extracted from wNaNG. The algorithm works without preliminary parameter settings while automatically identifying clusters with unbalanced densities, arbitrary shapes, and noises. To assess the advantages of the NonPC algorithm, extensive experiments have been conducted compared with some classic and recent clustering methods. The results demonstrate that the proposed NonPC algorithm significantly outperforms the state-of-the-art and well-known algorithms in Adjusted Rand index, Normalized Mutual Information, and Fowlkes-Mallows index aspects.

ScFseCluster: a feature selection-enhanced clustering for single-cell RNA-seq data.

Single-cell RNA sequencing (scRNA-seq) enables researchers to reveal previously unknown cell heterogeneity and functional diversity, which is impossible with bulk RNA sequencing. Clustering approaches are widely used for analyzing scRNA-seq data and identifying cell types and states. In the past few years, various advanced computational strategies emerged. However, the low generalization and high computational cost are the main bottlenecks of existing methods. In this study, we established a novel computational framework, scFseCluster, for scRNA-seq clustering analysis. scFseCluster incorporates a metaheuristic algorithm (Feature Selection based on Quantum Squirrel Search Algorithm) to extract the optimal gene set, which largely guarantees the performance of cell clustering. We conducted simulation experiments in several aspects to verify the performance of the proposed approach. scFseCluster performed very well on eight benchmark scRNA-seq datasets because of the optimal gene sets obtained using the Feature Selection based on Quantum Squirrel Search Algorithm. The comparative study demonstrated the significant advantages of scFseCluster over seven State-of-the-Art algorithms. In addition, our analysis shows that feature selection on high-variable genes can significantly improve clustering performance. In conclusion, our study demonstrates that scFseCluster is a highly versatile tool for enhancing scRNA-seq data clustering analysis.

Robust Multi-View Clustering Through Partition Integration on Stiefel Manifold

Multi-view clustering aims at integrating information from different views to improve clustering performance. Recent methods integrate multiple view-specific partition matrices to seek a consensus one and have demonstrated promising clustering performance in various applications. However, the clustering performance of such methods heavily relies on the consensus partition matrix estimated by the arithmetic mean in Euclidean space and thus is highly susceptible to noise corruption. To this end, this paper proposes to learn a consensus partition matrix through the geometric mean on the manifold to achieve robust clustering. Specifically, the multiple view-specific partition matrices can be regarded as points residing in the Stiefel manifold and enable a manifold-based integration. Consequently, the view-specific partition matrices are integrated by estimating a consensus partition matrix as the center point on the Stiefel manifold. Such a partition integration boils down to the Fréchet mean problem on a manifold, which is solved by the intrinsic manifold-based optimization and proves effective in providing a more robust estimation against noise. Experimental results on seven benchmark datasets demonstrate the effectiveness and noise-robustness of our proposed method in comparison to eight competitive methods.

Adversarial Multiview Clustering Networks With Adaptive Fusion.

The existing deep multiview clustering (MVC) methods are mainly based on autoencoder networks, which seek common latent variables to reconstruct the original input of each view individually. However, due to the view-specific reconstruction loss, it is challenging to extract consistent latent representations over multiple views for clustering. To address this challenge, we propose adversarial MVC (AMvC) networks in this article. The proposed AMvC generates each view's samples conditioning on the fused latent representations among different views to encourage a more consistent clustering structure. Specifically, multiview encoders are used to extract latent descriptions from all the views, and the corresponding generators are used to generate the reconstructed samples. The discriminative networks and the mean squared loss are jointly utilized for training the multiview encoders and generators to balance the distinctness and consistency of each view's latent representation. Moreover, an adaptive fusion layer is developed to obtain a shared latent representation, on which a clustering loss and the l1,2 -norm constraint are further imposed to improve clustering performance and distinguish the latent space. Experimental results on video, image, and text datasets demonstrate that the effectiveness of our AMvC is over several state-of-the-art deep MVC methods.

Hypergraph-Clustering Method Based on an Improved Apriori Algorithm

With the complexity and variability of data structures and dimensions, traditional clustering algorithms face various challenges. The integration of network science and clustering has become a popular field of exploration. One of the main challenges is how to handle large-scale and complex high-dimensional data effectively. Hypergraphs can accurately represent multidimensional heterogeneous data, making them important for improving clustering performance. In this paper, we propose a hypergraph-clustering method dubbed the “high-dimensional data clustering method” based on hypergraph partitioning using an improved Apriori algorithm (HDHPA). First, the method constructs a hypergraph based on the improved Apriori association rule algorithm, where frequent itemsets existing in high-dimensional data are treated as hyperedges. Then, different frequent itemsets are mined in parallel to obtain hyperedges with corresponding ranks, avoiding the generation of redundant rules and improving mining efficiency. Next, we use the dense subgraph partition (DSP) algorithm to divide the hypergraph into multiple subclusters. Finally, we merge the subclusters through dense sub-hypergraphs to obtain the clustering results. The advantage of this method lies in its use of the hypergraph model to discretize the association between data in space, which further enhances the effectiveness and accuracy of clustering. We comprehensively compare the proposed HDHPA method with several advanced hypergraph-clustering methods using seven different types of high-dimensional datasets and then compare their running times. The results show that the clustering evaluation index values of the HDHPA method are generally superior to all other methods. The maximum ARI value can reach 0.834, an increase of 42%, and the average running time is lower than other methods. All in all, HDHPA exhibits an excellent comparable performance on multiple real networks. The research results of this paper provide an effective solution for processing and analyzing large-scale network datasets and are also conducive to broadening the application range of clustering techniques.

Clustering Analysis of Wind Turbine Alarm Sequences Based on Domain Knowledge-Fused Word2vec

The alarm data contain abundant fault information related to almost all components of the wind turbine. Reasonable analysis and utilization of alarm data can assist wind farm maintenance personnel in quickly identifying the types of turbine faults, reducing operation and maintenance costs. This paper proposes a clustering analysis method that groups similar alarm sequences with the same fault type. Firstly, the alarm data are preprocessed, where alarm sequences are segmented, and redundant alarms are removed. Then, a domain knowledge-fused Word2vec (DK-Wrod2vec) method is introduced to transform non-numeric alarm codes into numeric vector representations. Finally, new distance metrics are incorporated into the K-means clustering algorithm to improve clustering performance. The performance of the proposed clustering method is assessed by applying it to labeled alarm sequences. The results demonstrate that the clustering performance is the best when using DK-Word2vec and the word rotator’s distance compared with other methods. Additionally, with the optimal parameter combination, the fault types of unlabeled alarm sequences are also analyzed.

Late Fusion Multiple Kernel Clustering With Proxy Graph Refinement.

Multiple kernel clustering (MKC) optimally utilizes a group of pre-specified base kernels to improve clustering performance. Among existing MKC algorithms, the recently proposed late fusion MKC methods demonstrate promising clustering performance in various applications and enjoy considerable computational acceleration. However, we observe that the kernel partition learning and late fusion processes are separated from each other in the existing mechanism, which may lead to suboptimal solutions and adversely affect the clustering performance. In this article, we propose a novel late fusion multiple kernel clustering with proxy graph refinement (LFMKC-PGR) framework to address these issues. First, we theoretically revisit the connection between late fusion kernel base partition and traditional spectral embedding. Based on this observation, we construct a proxy self-expressive graph from kernel base partitions. The proxy graph in return refines the individual kernel partitions and also captures partition relations in graph structure rather than simple linear transformation. We also provide theoretical connections and considerations between the proposed framework and the multiple kernel subspace clustering. An alternate algorithm with proved convergence is then developed to solve the resultant optimization problem. After that, extensive experiments are conducted on 12 multi-kernel benchmark datasets, and the results demonstrate the effectiveness of our proposed algorithm. The code of the proposed algorithm is publicly available at https://github.com/wangsiwei2010/graphlatefusion_MKC.

An efficient semi-supervised balanced cut with hard pairwise constraints and partial labels

With a small scale of prior information, including pairwise constraints and partial labels, semi-supervised clustering algorithms can improve clustering performance by imposing certain restrictions on the clustering results, and have accordingly attracted a lot of attention in recent years. Moreover, a large number of graph-based semi-supervised clustering algorithms have been proposed, since graphs are naturally suited to pairwise constraints. However, most graph-based approaches involve a two-step process and are not scalable to large-scale datasets due to their high time complexity. Additionally, they also encounter a serious problem of constraint violation. To deal with this issue, we propose a new non-parametric model to directly obtain the indicator matrix by minimizing the cut of the graph with a regularization term that corresponds to the balance of the clusters. To simultaneously support the two forms of prior information, our approach restricts the indicator matrix during optimization, while also ensuring that the clusters obtained by our approach do not violate any constraints. The computational complexity of the proposed approach is nearly linear to the data size n. Experiments show that the algorithm proposed in this paper achieves significant improvements in both clustering performance and efficiency.

ScSemiAAE: a semi-supervised clustering model for single-cell RNA-seq data

BackgroundSingle-cell RNA sequencing (scRNA-seq) strives to capture cellular diversity with higher resolution than bulk RNA sequencing. Clustering analysis is critical to transcriptome research as it allows for further identification and discovery of new cell types. Unsupervised clustering cannot integrate prior knowledge where relevant information is widely available. Purely unsupervised clustering algorithms may not yield biologically interpretable clusters when confronted with the high dimensionality of scRNA-seq data and frequent dropout events, which makes identification of cell types more challenging.ResultsWe propose scSemiAAE, a semi-supervised clustering model for scRNA sequence analysis using deep generative neural networks. Specifically, scSemiAAE carefully designs a ZINB adversarial autoencoder-based architecture that inherently integrates adversarial training and semi-supervised modules in the latent space. In a series of experiments on scRNA-seq datasets spanning thousands to tens of thousands of cells, scSemiAAE can significantly improve clustering performance compared to dozens of unsupervised and semi-supervised algorithms, promoting clustering and interpretability of downstream analyses.ConclusionscSemiAAE is a Python-based algorithm implemented on the VSCode platform that provides efficient visualization, clustering, and cell type assignment for scRNA-seq data. The tool is available from https://github.com/WHang98/scSemiAAE.

SimpleMKKM: Simple Multiple Kernel K-Means.

We propose a simple yet effective multiple kernel clustering algorithm, termed simple multiple kernel k-means (SimpleMKKM). It extends the widely used supervised kernel alignment criterion to multi-kernel clustering. Our criterion is given by an intractable minimization-maximization problem in the kernel coefficient and clustering partition matrix. To optimize it, we equivalently rewrite the minimization-maximization formulation as a minimization of an optimal value function, prove its differenentiablity, and design a reduced gradient descent algorithm to decrease it. Furthermore, we prove that the resultant solution of SimpleMKKM is the global optimum. We theoretically analyze the performance of SimpleMKKM in terms of its clustering generalization error. After that, we develop extensive experiments to investigate the proposed SimpleMKKM from the perspectives of clustering accuracy, advantage on the formulation and optimization, variation of the learned consensus clustering matrix with iterations, clustering performance with varied number of samples and base kernels, analysis of the learned kernel weight, the running time and the global convergence. The experimental study demonstrates the effectiveness of the proposed SimpleMKKM by considerably and consistently outperforming state of the art multiple kernel clustering alternatives. In addition, the ablation study shows that the improved clustering performance is contributed by both the novel formulation and new optimization. Our work provides a more effective approach to integrate multi-view data for clustering, and this could trigger novel research on multiple kernel clustering. The source code and data for SimpleMKKM are available at https://github.com/xinwangliu/SimpleMKKMcodes/.

Learning Distributed Representations and Deep Embedded Clustering of Texts

Instructors face significant time and effort constraints when grading students’ assessments on a large scale. Clustering similar assessments is a unique and effective technique that has the potential to significantly reduce the workload of instructors in online and large-scale learning environments. By grouping together similar assessments, marking one assessment in a cluster can be scaled to other similar assessments, allowing for a more efficient and streamlined grading process. To address this issue, this paper focuses on text assessments and proposes a method for reducing the workload of instructors by clustering similar assessments. The proposed method involves the use of distributed representation to transform texts into vectors, and contrastive learning to improve the representation that distinguishes the differences among similar texts. The paper presents a general framework for clustering similar texts that includes label representation, K-means, and self-organization map algorithms, with the objective of improving clustering performance using Accuracy (ACC) and Normalized Mutual Information (NMI) metrics. The proposed framework is evaluated experimentally using two real datasets. The results show that self-organization maps and K-means algorithms with Pre-trained language models outperform label representation algorithms for different datasets.

Multi-view clustering via matrix factorization assisted k-means

Matrix factorization based multi-view clustering algorithms has attracted much attention in recent years due to the strong interpretability and efficient implementation. In general, these approaches firstly compute the coefficient matrices of each data views, and learn a consensus matrix simultaneously. By applying the classical clustering techniques, such as k-means, on the consensus matrix, the final partition can be easily obtained. However, most of previous models work in a “step-by-step” manner, which cannot perform multi-view matrix factorization and clustering label generation simultaneously, leading to degenerated performance. In this paper, we propose a novel “one-pass” method, which integrates matrix factorization and k-means into a unified framework, named multi-view clustering via matrix factorization assisted k-means (MFK). In MFK, the generation of cluster indicator matrix and coefficient matrix learning can boost each other, leading to final improved clustering performance. Furthermore, we adopt a graph Laplacian regularization on the indicator matrix in order to capture the intrinsic geometric structure of original data. An alternating optimization strategy is designed to solve the resultant optimization problem and extensive experiments on six publicly datasets are conducted to demonstrate the superiority and effectiveness of the proposed MFK model.