Enhance Clustering Performance Research Articles

MICCF: A Mutual Information Constrained Clustering Framework for Learning Clustering-Oriented Feature Representations

Deep clustering is a crucial task in machine learning and data mining that focuses on acquiring feature representations conducive to clustering. Previous research relies on self-supervised representation learning for general feature representations, such features may not be optimally suited for downstream clustering tasks. In this paper, we introduce MICCF, a framework designed to bridge this gap and enhance clustering performance. MICCF enhances feature representations by combining mutual information at different levels and employs an auxiliary alignment mutual information module for learning clustering-oriented features. To be specific, we propose a dual mutual information constraints module, incorporating minimal mutual information constraints at the feature level and maximal mutual information constraints at the instance level. This reduction in feature redundancy encourages the neural network to extract more discriminative features, while maximization ensures more unbiased and robust representations. To obtain clustering-oriented representations, the auxiliary alignment mutual information module utilizes pseudo-labels to maximize mutual information through a multi-classifier network, aligning features with the clustering task. The main network and the auxiliary one work in synergy to jointly optimize feature representations that are well-suited for the clustering task. We validate the effectiveness of our method through extensive experiments on six benchmark datasets. The results indicate that our method performs well in most scenarios, particularly on fine-grained datasets, where our approach effectively distinguishes subtle differences between closely related categories. Notably, our approach achieved a remarkable accuracy of 96.4% on the ImageNet-10 dataset, surpassing other comparison methods. The code is available at https://github.com/Li-Hyn/MICCF.git .

Robust tensor ring-based graph completion for incomplete multi-view clustering

Incomplete multi-view clustering (IMVC) aims to enhance clustering performance by leveraging complementary information from multi-view data, even in the presence of missing instances. This is challenging due to the interference caused by these missing data points. Current IMVC algorithms generally adopt two main strategies: either disregarding the missing instances and focusing on observable data for clustering, or complementing the missing instances within each view to improve the clustering results. However, effectively leveraging the latent information within these missing instances remains a challenge. In response, we propose a novel IMVC framework that complements the tensor consisting of similarity matrices learned from the available instances to enhance the relationships among samples from different views. In addition, we integrate tensor ring completion and M-estimator-based methods into the IMVC approach. This promotes the integration of inter-view information and mitigates errors introduced by missing instances in the model, respectively. Furthermore, we develop two efficient half-quadratic (HQ) based iterative algorithms with soft/hard threshold or multivariate generalization of minimax-concave (GMC) penalty to regularize the low-rank property. Comprehensive evaluations on seven benchmark datasets demonstrate that our method outperforms state-of-the-art IMVC approaches across various metrics.

One-step incremental multi-view spectral clustering based on graph linkage learning

Most traditional multi-view spectral clustering algorithms involve two separate steps: solving the spectral embedding matrix and clustering, which may introduce errors during the clustering process. Moreover, in practical applications, the number of available views may increase over time, and the approach of re-fusing all views in each computation would result in elevated computational costs. In this paper, we propose a novel model: One-step Incremental Multi-view Spectral Clustering based on Graph Linkage learning (OIMvSC). OIMvSC only needs to store the consensus spectral embedding matrix and the consensus label matrix of all previous views and combines them with the spectral embedding matrix of the newly available view to solve the fused consensus spectral embedding matrix and consensus label matrix. To further enhance clustering performance, OIMvSC introduces graph linkage learning, which reduces erroneous connections between clusters while preserving correct connections within clusters. A convergent iterative algorithm for solving OIMvSC is proposed. Experimental results demonstrate that OIMvSC exhibits excellent clustering performance.

A New Hybrid Optimization Algorithm to Solve the Clustering Problem

: This paper presents a novel hybrid technique, merging the K-means algorithm with Genetic Algorithm (GA), aiming to enhance clustering performance. This approach leverages the strengths of both algorithms, enabling improved cluster generation by overcoming individual algorithmic limitations. The GA-KM algorithm is introduced to aid K-means in avoiding local optima, with GA exhibiting proficiency in determining optimal cluster initialization and parameter optimization. The focus is on developing a GA-based algorithm for generating high-quality clusters efficiently. Notably, the research explores the application of this hybrid approach to address issues in the educational domain, specifically for out-of-school children. The fitness function in GA is tailored to the problem area, emphasizing the need for an appropriate system to study and address school children's problems. The research proposes a hybrid algorithm (KM-GA-NM-PSO) that amalgamates the best features of existing algorithms, thereby overcoming individual limitations and promising superior results. This hybridization is expected to yield high-quality clusters with minimal function evaluations, outperforming other methods by producing clusters with small standard deviations on selected datasets. The proposed approach, combining KM, GA, NM, and PSO algorithms, demonstrates improved data clustering quality and algorithmic efficiency.

Error-robust multi-view subspace clustering with nonconvex low-rank tensor approximation and hyper-Laplacian graph embedding

Tensor based subspace clustering, a method aimed at partitioning multi-view data into distinct clusters through the tensor low-rank representation, has gained widespread popularity. Despite the effectiveness of tensor singular value decomposition (t-SVD) based nuclear norm in extracting high-dimensional information from multiple views, certain limitations persist. Firstly, while the commonly used l2,1 norm enhances clustering robustness, it remains susceptible to the impact of high-intensity noises. Secondly, the t-SVD based nuclear norm provides a biased estimation of tensor rank. Thirdly, existing methods either fail to preserve manifold structures from the original space or only extract binary relationships between data points. To address these challenges and enhance clustering performance in both high-intensity noisy and real-world conditions, an error-robust multi-view subspace clustering with nonconvex low-rank tensor approximation and hyper-Laplacian graph embedding (EMSC-NLTHG) method is proposed. Specifically, Cauchy loss function which reduces sensitivity to larger noises and outliers is introduced. Based on the Cauchy loss function, we develop the Cauchy pseudo norm to represent the reconstruction error in clustering to enhance the robustness. Moreover, rather than uniformly weighting all singular values in the t-SVD nuclear norm, a nonconvex tensor nuclear norm is adopted to add weights adaptively and approximate the true tensor rank. Additionally, hypergraph is embedded in the representation matrices to preserve local manifold structures and extract the complex multivariate relationships between data points. Extensive experiments demonstrate that the proposed EMSC-NLTHG method consistently outperforms state-of-the-art techniques by approximately 18% to 40% on datasets corrupted by high-intensity noises and around 1% to 27% on eight real-world datasets across six popular evaluation metrics.

Parallel power load abnormalities detection using fast density peak clustering with a hybrid canopy-K-means algorithm

Parallel power loads anomalies are processed by a fast-density peak clustering technique that capitalizes on the hybrid strengths of Canopy and K-means algorithms all within Apache Mahout’s distributed machine-learning environment. The study taps into Apache Hadoop’s robust tools for data storage and processing, including HDFS and MapReduce, to effectively manage and analyze big data challenges. The preprocessing phase utilizes Canopy clustering to expedite the initial partitioning of data points, which are subsequently refined by K-means to enhance clustering performance. Experimental results confirm that incorporating the Canopy as an initial step markedly reduces the computational effort to process the vast quantity of parallel power load abnormalities. The Canopy clustering approach, enabled by distributed machine learning through Apache Mahout, is utilized as a preprocessing step within the K-means clustering technique. The hybrid algorithm was implemented to minimise the length of time needed to address the massive scale of the detected parallel power load abnormalities. Data vectors are generated based on the time needed, sequential and parallel candidate feature data are obtained, and the data rate is combined. After classifying the time set using the canopy with the K-means algorithm and the vector representation weighted by factors, the clustering impact is assessed using purity, precision, recall, and F value. The results showed that using canopy as a preprocessing step cut the time it proceeds to deal with the significant number of power load abnormalities found in parallel using a fast density peak dataset and the time it proceeds for the k-means algorithm to run. Additionally, tests demonstrate that combining canopy and the K-means algorithm to analyze data performs consistently and dependably on the Hadoop platform and has a clustering result that offers a scalable and effective solution for power system monitoring.

ScMAE: a masked autoencoder for single-cell RNA-seq clustering.

Single-cell RNA sequencing has emerged as a powerful technology for studying gene expression at the individual cell level. Clustering individual cells into distinct subpopulations is fundamental in scRNA-seq data analysis, facilitating the identification of cell types and exploration of cellular heterogeneity. Despite the recent development of many deep learning-based single-cell clustering methods, few have effectively exploited the correlations among genes, resulting in suboptimal clustering outcomes. Here, we propose a novel masked autoencoder-based method, scMAE, for cell clustering. scMAE perturbs gene expression and employs a masked autoencoder to reconstruct the original data, learning robust and informative cell representations. The masked autoencoder introduces a masking predictor, which captures relationships among genes by predicting whether gene expression values are masked. By integrating this masking mechanism, scMAE effectively captures latent structures and dependencies in the data, enhancing clustering performance. We conducted extensive comparative experiments using various clustering evaluation metrics on 15 scRNA-seq datasets from different sequencing platforms. Experimental results indicate that scMAE outperforms other state-of-the-art methods on these datasets. In addition, scMAE accurately identifies rare cell types, which are challenging to detect due to their low abundance. Furthermore, biological analyses confirm the biological significance of the identified cell subpopulations. The source code of scMAE is available at: https://zenodo.org/records/10465991.

FEED: a feature selection method based on gene expression decomposition for single cell clustering.

Single-cell clustering is a critical step in biological downstream analysis. The clustering performance could be effectively improved by extracting cell-type-specific genes. The state-of-the-art feature selection methods usually calculate the importance of a single gene without considering the information contained in the gene expression distribution. Moreover, these methods ignore the intrinsic expression patterns of genes and heterogeneity within groups of different mean expression levels. In this work, we present a Feature sElection method based on gene Expression Decomposition (FEED) of scRNA-seq data, which selects informative genes to enhance clustering performance. First, the expression levels of genes are decomposed into multiple Gaussian components. Then, a novel gene correlation calculation method is proposed to measure the relationship between genes from the perspective of distribution. Finally, a permutation-based approach is proposed to determine the threshold of gene importance to obtain marker gene subsets. Compared with state-of-the-art feature selection methods, applying FEED on various scRNA-seq datasets including large datasets followed by different common clustering algorithms results in significant improvements in the accuracy of cell-type identification. The source codes for FEED are freely available at https://github.com/genemine/FEED.

Multi-view clustering via efficient representation learning with anchors

Multi-view spectral clustering has gained considerable attention due to its potential to enhance clustering performance. Although many methods have shown promising results, they often suffer from high time complexity and are not suitable for large-scale datasets. On the other hand, anchor-based methods are well-known for their efficiency. These methods typically learn the similarity relationship between instances and anchors and then convert it into the similarity relationship between instances, involving a considerable number of calculations. To address this issue, we propose a novel method called Multi-view clustering via Efficient Representation LearnIng with aNchors (MERLIN) in this paper. Instead of learning the instance–instance relationship, MERLIN approaches the clustering problem from the perspective of representation learning. Specifically, MERLIN selects the same anchors for different views and utilizes these anchors to learn a consensus representation that integrates information from all views. Additionally, MERLIN adaptively learns weights for different views to fully exploit the complementary information among multiple views. In comparison with seven state-of-the-art baseline methods across five datasets, MERLIN demonstrates both efficiency and effectiveness in handling multi-view datasets and is suitable for handling large-scale datasets.

A Novel Longitudinal Phenotype-Genotype Association Study Based on Deep Feature Extraction and Hypergraph Models for Alzheimer's Disease.

Traditional image genetics primarily uses linear models to investigate the relationship between brain image data and genetic data for Alzheimer's disease (AD) and does not take into account the dynamic changes in brain phenotype and connectivity data across time between different brain areas. In this work, we proposed a novel method that combined Deep Subspace reconstruction with Hypergraph-Based Temporally-constrained Group Sparse Canonical Correlation Analysis (DS-HBTGSCCA) to discover the deep association between longitudinal phenotypes and genotypes. The proposed method made full use of dynamic high-order correlation between brain regions. In this method, the deep subspace reconstruction technique was applied to retrieve the nonlinear properties of the original data, and hypergraphs were used to mine the high-order correlation between two types of rebuilt data. The molecular biological analysis of the experimental findings demonstrated that our algorithm was capable of extracting more valuable time series correlation from the real data obtained by the AD neuroimaging program and finding AD biomarkers across multiple time points. Additionally, we used regression analysis to verify the close relationship between the extracted top brain areas and top genes and found the deep subspace reconstruction approach with a multi-layer neural network was helpful in enhancing clustering performance.

Neural clustering based on implicit maximum likelihood

Clustering is one of the most fundamental unsupervised learning tasks with numerous applications in various fields. Clustering methods based on neural networks, called deep clustering methods, leverage the representational power of neural networks to enhance clustering performance. ClusterGan constitutes a generative deep clustering method that exploits generative adversarial networks (GANs) to perform clustering. However, it inherits some deficiencies of GANs, such as mode collapse, vanishing gradients and training instability. In order to tackle those deficiencies, the generative approach of implicit maximum likelihood estimation (IMLE) has been recently proposed. In this paper, we present a clustering method based on generative neural networks, called neural implicit maximum likelihood clustering, which adopts ideas from both ClusterGAN and IMLE. The proposed method has been compared with ClusterGAN and other neural clustering methods on both synthetic and real datasets, demonstrating promising results.

Self‐supervised image clustering from multiple incomplete views via constrastive complementary generation

AbstractIncomplete Multi‐View Clustering aims to enhance clustering performance by using data from multiple modalities. Despite the fact that several approaches for studying this issue have been proposed, the following drawbacks still persist: (1) It is difficult to learn latent representations that account for complementarity yet consistency without using label information; (2) and thus fails to take full advantage of the hidden information in incomplete data results in suboptimal clustering performance when complete data is scarce. In this study, Contrastive Incomplete Multi‐View Image Clustering with Generative Adversarial Networks (CIMIC‐GAN), which uses Generative Adversarial Network (GAN) to fill in incomplete data and uses double contrastive learning to learn consistency on complete and incomplete data is proposed. More specifically, considering diversity and complementary information among multiple modalities, we incorporate autoencoding representation of complete and incomplete data into double contrastive learning to achieve learning consistency. Integrating GANs into the autoencoding process can not only take full advantage of new features of incomplete data, but also better generalise the model in the presence of high data missing rates. Experiments conducted on four extensively used data sets show that CIMIC‐GAN outperforms state‐of‐the‐art incomplete multi‐View clustering methods.

Incomplete multi-view subspace clustering based on missing-sample recovering and structural information learning

The incomplete multi-view clustering (IMVC) focuses on exploring hidden information from missing samples and complementary information from multiple views to enhance clustering performance. Most of existing methods for IMVC only attempt to obtain a shared consistent representation or graph from the non-missing samples based on matrix factorization. However, these methods rely too much on view alignment information, neglecting the specific-view structural information of different views and the hidden information of missing samples. To address the above problems, a novel method, called incomplete multi-view subspace clustering based on missing-sample recovering and structural information learning (MSR_SIL), is proposed in this paper. Overall, MSR_SIL incorporates the missing-sample recovering in feature space and self-expressiveness coefficient learning in representation space into a unified framework. Specifically, a reconstruction term is introduced to recover missing samples with the help of non-missing ones so that incomplete multi-view is completed. Next, the self-expressiveness coefficient of the recovered complete multi-view is learned. Then, each self-expressiveness coefficient is decomposed into a consistent part to capture the similarity information of view-paired and a specific part to hold the unique information of each view. Especially, to retain the global manifold structure of the multi-view data, Schatten-p norm is applied. Finally, the clustering results can be obtained on the basis of the consistent and specific representation. Experimental results on eleven well-known benchmark multi-view datasets demonstrate that the proposed MSR_SIL yields significant improvements on clustering performance.

Query-Augmented Active Metric Learning

In this article, we propose an active metric learning method for clustering with pairwise constraints. The proposed method actively queries the label of informative instance pairs, while estimating underlying metrics by incorporating unlabeled instance pairs, which leads to a more accurate and efficient clustering process. In particular, we augment the queried constraints by generating more pairwise labels to provide additional information in learning a metric to enhance clustering performance. Furthermore, we increase the robustness of metric learning by updating the learned metric sequentially and penalizing the irrelevant features adaptively. In addition, we propose a novel active query strategy that evaluates the information gain of instance pairs more accurately by incorporating the neighborhood structure, which improves clustering efficiency without extra labeling cost. In theory, we provide a tighter error bound of the proposed metric learning method using augmented queries compared with methods using existing constraints only. Furthermore, we also investigate the improvement using the active query strategy instead of random selection. Numerical studies on simulation settings and real datasets indicate that the proposed method is especially advantageous when the signal-to-noise ratio between significant features and irrelevant features is low.

Multiview Common Subspace Clustering via Coupled Low Rank Representation

Multi-view subspace clustering (MVSC) finds a shared structure in latent low-dimensional subspaces of multi-view data to enhance clustering performance. Nonetheless, we observe that most existing MVSC methods neglect the diversity in multi-view data by considering only the common knowledge to find a shared structure either directly or by merging different similarity matrices learned for each view. In the presence of noise, this predefined shared structure becomes a biased representation of the different views. Thus, in this article, we propose a MVSC method based on coupled low-rank representation to address the above limitation. Our method first obtains a low-rank representation for each view, constrained to be a linear combination of the view-specific representation and the shared representation by simultaneously encouraging the sparsity of view-specific one. Then, it uses the k -block diagonal regularizer to learn a manifold recovery matrix for each view through respective low-rank matrices to recover more manifold structures from them. In this way, the proposed method can find an ideal similarity matrix by approximating clustering projection matrices obtained from the recovery structures. Hence, this similarity matrix denotes our clustering structure with exactly k connected components by applying a rank constraint on the similarity matrix’s relaxed Laplacian matrix to avoid spectral post-processing of the low-dimensional embedding matrix. The core of our idea is such that we introduce dynamic approximation into the low-rank representation to allow the clustering structure and the shared representation to guide each other to learn cleaner low-rank matrices that would lead to a better clustering structure. Therefore, our approach is notably different from existing methods in which the local manifold structure of data is captured in advance. Extensive experiments on six benchmark datasets show that our method outperforms 10 similar state-of-the-art compared methods in six evaluation metrics.

Fuzzy Density Peaks Clustering

As an exemplar-based clustering method, the well-known density peaks clustering (DPC) heavily depends on the computation of kernel-based density peaks, which incurs two issues: first, whether kernel-based density can facilitate a large variety of data well, including cases where ambiguity and uncertainty of the assignment of the data points to their clusters may exist, and second, whether the concept of density peaks can be interpreted and manipulated from the perspective of soft partitions (e.g., fuzzy partitions) to achieve enhanced clustering performance. In this article, in order to provide flexible adaptability for tackling ambiguity and uncertainty in clustering, a new concept of fuzzy peaks is proposed to express the density of a data point as the fuzzy-operator-based coupling of the fuzzy distances between a data point and its neighbors. As a fuzzy variant of DPC, a novel fuzzy density peaks clustering (FDPC) method FDPC based on fuzzy operators (especially S-norm operators) is accordingly devised along with the same algorithmic framework of DPC. With an appropriate choice of a fuzzy operator with its associated tunable parameter for a clustering task, FDPC can indeed inherit the advantage of fuzzy partitions and simultaneously provide flexibility in enhancing clustering performance. The experimental results on both synthetic and real data sets demonstrate that the proposed method outperforms or at least remains comparable to the comparative methods in clustering performance by choosing appropriate parameters in most cases.

An Active Learning Algorithm Based on Shannon Entropy for Constraint-Based Clustering

Pairwise constraints could enhance clustering performance in constraint-based clustering problems, especially when these pairwise constraints are informative. In this paper, a novel active learning pairwise constraint formulation algorithm would be constructed with aim to formulate informative pairwise constraints efficiently and economically. This algorithm consists of three phases: Selecting, Exploring and Consolidating. In Selecting phase, some type of unsupervised clustering algorithm is used to obtain an informative data set in terms of Shannon entropy. In Exploring phase, some type of farthest-first strategy is used to construct a series of query with aim to construct clustering skeleton set structure and informative pairwise constraints are also collected meanwhile based on the informative data set. If the number of skeleton sets equals the number of clusters, the new algorithm gets into third phase Consolidating; otherwise, it would finish. In Consolidating phase, non-skeleton points included in the informative data set are used to construct a series of query with skeleton set representative points constructed in Exploring phase. And some type of priority principle is proposed to help collect more must-link pairwise constraints. Treat the well-known MPCK-means (metric pairwise constrained K-means) as the underlying constraint-based semi-supervised clustering algorithm and data experiment comparison between this new algorithm and its counterparts would be done. Experiment outcome shows that significant improvement of this new algorithm.

Self-paced and auto-weighted multi-view clustering

Multi-view clustering (MVC) methods are effective approaches to enhance clustering performance by exploiting complementary information from multiple views. One main disadvantage of most existing MVC methods is that the corresponding optimization problems are non-convex and thus local optimal solutions are usually obtained. To alleviate this issue, we propose a novel multi-view clustering method equipped with self-paced learning, which first learns the MVC model with easy examples and then progressively considers complex ones from each view. In addition, a soft weighting scheme of self-paced learning is designed to further reduce the negative impact from outliers and noises. Furthermore, to consider the importance of different views, we develop an auto-weighted technique to automatically assign weights to views. The proposed model only needs the number of clusters as input and can be easily solved by an alternating optimization paradigm. Experimental results on various benchmark data sets demonstrate the effectiveness of the proposed model.

Adaptive Sample-level Graph Combination for Partial Multiview Clustering.

Multiview clustering explores complementary information among distinct views to enhance clustering performance under the assumption that all samples have complete information in all available views. However, this assumption does not hold in many real applications, where the information of some samples in one or more views may be missing, leading to partial multiview clustering problems. In this case, significant performance degeneration is usually observed. A collection of partial multiview clustering algorithms has been proposed to address this issue and most treat all different views equally during clustering. In fact, because different views provide features collected from different angles/feature spaces, they might play different roles in the clustering process. With the diversity of different views considered, in this study, a novel adaptive method is proposed for partial multiview clustering by automatically adjusting the contributions of different views. The samples are divided into complete and incomplete sets, while a joint learning mechanism is established to facilitate the connection between them and thereby improve clustering performance. More specifically, the method is characterized by a joint optimization model comprising two terms. The first term mines the underlying cluster structure from both complete and incomplete samples by adaptively updating their importance in all available views. The second term is designed to group all data with the aid of the cluster structure modeled in the first term. These two terms seamlessly integrate the complementary information among multiple views and enhance the performance of partial multiview clustering. Experimental results on real-world datasets illustrate the effectiveness and efficiency of our proposed method.

Learning Transferable Subspace for Human Motion Segmentation

Temporal data clustering is a challenging task. Existing methods usually explore data self-representation strategy, which may hinder the clustering performance in insufficient or corrupted data scenarios. In real-world applications, we are easily accessible to a large amount of related labeled data. To this end, we propose a novel transferable subspace clustering approach by exploring useful information from relevant source data to enhance clustering performance in target temporal data. We manage to transform the original data into a shared low-dimensional and distinctive feature space by jointly seeking an effective domain-invariant projection. In this way, the well-labeled source knowledge can help obtain a more discriminative target representation. Moreover, a graph regularizer is designed to incorporate temporal information to preserve more sequence knowledge into the learned representation. Extensive experiments based on three human motion datasets illustrate that our approach is able to outperform state-of-the-art temporal data clustering methods.