Improve Clustering Performance Research Articles

Infinite Mixtures of Infinite Factor Analysers

Factor-analytic Gaussian mixture models are often employed as a model-based approach to clustering high-dimensional data. Typically, the numbers of clusters and latent factors must be specified in advance of model fitting, and remain fixed. The pair which optimises some model selection criterion is then chosen. For computational reasons, models in which the number of latent factors differ across clusters are rarely considered. Here the infinite mixture of infinite factor analysers (IMIFA) model is introduced. IMIFA employs a Pitman-Yor process prior to facilitate automatic inference of the number of clusters using the stick-breaking construction and a slice sampler. Furthermore, IMIFA employs multiplicative gamma process shrinkage priors to allow cluster-specific numbers of factors, automatically inferred via an adaptive Gibbs sampler. IMIFA is presented as the flagship of a family of factor-analytic mixture models, providing flexible approaches to clustering high-dimensional data. Applications to a benchmark data set, metabolomic spectral data, and a manifold learning handwritten digit example illustrate the IMIFA model and its advantageous features. These include obviating the need for model selection criteria, reducing the computational burden associated with the search of the model space, improving clustering performance by allowing cluster-specific numbers of factors, and quantifying uncertainty in the numbers of clusters and cluster-specific factors.

Multiple clustering and selecting algorithms with combining strategy for selective clustering ensemble

Clustering ensemble can overcome the instability of clustering and improve clustering performance. With the rapid development of clustering ensemble, we find that not all clustering solutions are effective in their final result. In this paper, we focus on selection strategy in selective clustering ensemble. We propose a multiple clustering and selecting approach (MCAS), which is based on different original clustering solutions. Furthermore, we present two combining strategies, direct combining and clustering combining, to combine the solutions selected by MCAS. These combining strategies combine results of MCAS and get a more refined subset of solutions, compared with traditional selective clustering ensemble algorithms and single clustering and selecting algorithms. Experimental results on UCI machine learning datasets show that the algorithm that uses multiple clustering and selecting algorithms with combining strategy performs well on most datasets and outperforms most selective clustering ensemble algorithms.

Adaptive Fusion of Heterogeneous Manifolds for Subspace Clustering.

Multiview clustering (MVC) has recently received great interest due to its pleasing efficacy in combining the abundant and complementary information to improve clustering performance, which overcomes the drawbacks of view limitation existed in the standard single-view clustering. However, the existing MVC methods are mostly designed for vectorial data from linear spaces and, thus, are not suitable for multiple dimensional data with intrinsic nonlinear manifold structures, e.g., videos or image sets. Some works have introduced manifolds' representation methods of data into MVC and obtained considerable improvements, but how to fuse multiple manifolds efficiently for clustering is still a challenging problem. Particularly, for heterogeneous manifolds, it is an entirely new problem. In this article, we propose to represent the complicated multiviews' data as heterogeneous manifolds and a fusion framework of heterogeneous manifolds for clustering. Different from the empirical weighting methods, an adaptive fusion strategy is designed to weight the importance of different manifolds in a data-driven manner. In addition, the low-rank representation is generalized onto the fused heterogeneous manifolds to explore the low-dimensional subspace structures embedded in data for clustering. We assessed the proposed method on several public data sets, including human action video, facial image, and traffic scenario video. The experimental results show that our method obviously outperforms a number of state-of-the-art clustering methods.

Face clustering via learning a sparsity preserving low-rank graph

Face clustering aims to group the face images without any label information into clusters, and has recently attracted considerable attention in machine learning and data mining. Many graph based clustering methods have been proposed and among which sparse representation (SR) and low-rank representation (LRR) are two representative methods for affinity graph construction. The clustering result may be inaccurate if the affinity graph is constructed with low quality. In this paper, we propose a novel face clustering method via learning a sparsity preserving low-rank graph (LSPLRG), where the initial affinity graph is derived on the sparse coefficients without any a priori graph or similarity matrix. In addition, an adaptive weighted matrix is imposed on the data reconstruction errors to enhance the role of important features, while a constraint on the representation matrix is to reduce the redundant features. By integrating the local distance regularization term into LRR, LSPLRG could exploit the global and local structures of data simultaneously. These appealing properties allow LSPLRG to well capture the intrinsic structure of data, and thus has potential to improve clustering performance. Experiments conducted on several face image databases demonstrate the effectiveness and robustness of LSPLRG compared with several state-of-the-art subspace clustering methods.

Simultaneously learning feature-wise weights and local structures for multi-view subspace clustering

Multi-view clustering integrates multiple feature sets, which usually have a complementary relationship and can reveal distinct insights of data from different angles, to improve clustering performance. It remains challenging to productively utilize complementary information across multiple views since there is always noise in real data, and their features are highly redundant. Moreover, most existing multi-view clustering approaches only aimed at exploring the consistency of all views, but overlooked the local structure of each view. However, it is necessary to take the local structure of each view into consideration, because individual views generally present different geometric structures while admitting the same cluster structure. To ease the above issues, in this paper, a novel multi-view subspace clustering method is established by concurrently assigning weights for different features and capturing local information of data in view-specific self-representation feature spaces. In particular, a common clustering assignment regularization is adopted to explore the consistency among multiple views. An alternating iteration algorithm based on the augmented Lagrangian multiplier is also developed for optimizing the associated objective. Experiments conducted on diverse multi-view datasets manifest that the proposed method achieves state-of-the-art performance. We provide the Matlab code on https://github.com/Ekin102003/JFLMSC.

Spectral clustering via ensemble deep autoencoder learning (SC-EDAE)

Several works have studied clustering strategies that combine classical clustering algorithms and deep learning methods. These strategies generally improve clustering performance, however deep autoencoder setting issues impede the robustness of these approaches. To alleviate the impact of hyperparameters setting, we propose a model which combines spectral clustering and deep autoencoder strengths in an ensemble framework. Our proposal does not require any pretraining and includes the three following steps: generating various deep embeddings from the original data, constructing a sparse and low-dimensional ensemble affinity matrix based on anchors strategy and applying spectral clustering to obtain the common space shared by multiple deep representations. While the anchors strategy ensures an efficient merging of the encodings, the fusion of various deep representations enables to mitigate the deep networks setting issues. Experiments on various benchmark datasets demonstrate the potential and robustness of our approach compared to state-of-the-art deep clustering methods.

A Novel Cosine Similarity Like Data Clustering Method for Effective Data Classification in Data Mining

In data mining ample techniques use distance based measures for data clustering. Improving clustering performance is the fundamental goal in cluster domain related tasks. Many techniques are available for clustering numerical data as well as categorical data. Clustering is an unsupervised learning technique and objects are grouped or clustered based on similarity among the objects. A new cluster similarity finding measure, which is cosine like cluster similarity measure (CLCSM), is proposed in this paper. The proposed cluster similarity measure is used for data classification. Extensive experiments are conducted by taking UCI machine learning datasets. The experimental results have shown that the proposed cosinelike cluster similarity measure is superior to many of the existing cluster similarity measures for data classification.

Multi-View Spectral Clustering with Optimal Neighborhood Laplacian Matrix

Multi-view spectral clustering aims to group data into different categories by optimally exploring complementary information from multiple Laplacian matrices. However, existing methods usually linearly combine a group of pre-specified first-order Laplacian matrices to construct an optimal Laplacian matrix, which may result in limited representation capability and insufficient information exploitation. In this paper, we propose a novel optimal neighborhood multi-view spectral clustering (ONMSC) algorithm to address these issues. Specifically, the proposed algorithm generates an optimal Laplacian matrix by searching the neighborhood of both the linear combination of the first-order and high-order base Laplacian matrices simultaneously. This design enhances the representative capacity of the optimal Laplacian and better utilizes the hidden high-order connection information, leading to improved clustering performance. An efficient algorithm with proved convergence is designed to solve the resultant optimization problem. Extensive experimental results on 9 datasets demonstrate the superiority of our algorithm against state-of-the-art methods, which verifies the effectiveness and advantages of the proposed ONMSC.

A New Burrows Wheeler Transform Markov Distance

Prior work inspired by compression algorithms has described how the Burrows Wheeler Transform can be used to create a distance measure for bioinformatics problems. We describe issues with this approach that were not widely known, and introduce our new Burrows Wheeler Markov Distance (BWMD) as an alternative. The BWMD avoids the shortcomings of earlier efforts, and allows us to tackle problems in variable length DNA sequence clustering. BWMD is also more adaptable to other domains, which we demonstrate on malware classification tasks. Unlike other compression-based distance metrics known to us, BWMD works by embedding sequences into a fixed-length feature vector. This allows us to provide significantly improved clustering performance on larger malware corpora, a weakness of prior methods.

Multi-view projected clustering with graph learning

Graph based multi-view learning is well known due to its effectiveness and good clustering performance. However, most existing methods directly construct graph from original high-dimensional data which always contain redundancy, noise and outlying entries in real applications, resulting in unreliable and inaccurate graph. Moreover, they do not effectively select some useful features which are important for graph learning and clustering. To solve these limits, we propose a novel model that combines dimensionality reduction, manifold structure learning and feature selection into a framework. We map high-dimensional data into low-dimensional space to reduce the complexity of the algorithm and reduce the effect of noise and redundance. Therefore, we can adaptively learn a more accurate graph. Further more, ℓ21-norm regularization is adopted to adaptively select some important features which help improve clustering performance. Finally, an efficiently algorithm is proposed to solve the optimal solution. Extensive experimental results on some benchmark datasets demonstrate the superiority of the proposed method.

Spectral representation learning for one-step spectral rotation clustering

Conventional spectral clustering is generally divided into two main steps, 1) constructing a reliable spectral representation by similarity matrix or Laplacian matrix; 2) performing K-means clustering on the spectral representation. In this paper, we propose a novel spectral clustering algorithm to improve clustering performance by separately improving these two steps in a same learning framework. Specifically, we first utilize two dimensionality reduction methods to learn the robust low-dimensional spectral representation, and a hypergraph structure to make the spectral representation keep the high-order relation of original data. Furthermore, a spectral rotation clustering method is embedded into the spectral representation learning model to conduct one-step clustering, which effectively reduces the deviation of clustering and obtains a reliable clustering performance. Besides, an effective optimization algorithm is further proposed to solve the objective problem to have a fast convergence. Experimental analysis on real data sets showed that our proposed clustering method outperformed the classical and state-of-the-art spectral clustering methods in terms of four frequently-used clustering metrics.

Weighted Laplacian Method and Its Theoretical Applications

Recently, a class of multilevel graph clustering (graph partitioning) algorithms have been extensively studied due to their practical utility. Although these newly proposed algorithms work well, there is still not many theoretical guarantees for the quality of partition in these existing methods due to their intrinsic heuristic properties to a great extent. In this paper, we propose a novel weighted Laplacian method for the multilevel graph clustering problem with more powerful theoretical background to further improve clustering performance mainly in terms of accuracy. Since our algorithm inherits the virtues of spectral methods, it possesses a friendly optimization property since it can produce the global optimal solution of a relaxation to the weighted cut on the coarsened graph in the middle stage. Meanwhile, the multilevel strategy can make it possible for our method to produce final clustering results with reasonable time range. Additionally, the weighted graph Laplacian is also suitable for doubly-weighted graph, which will endow our algorithm with a potential wide range of applications. The experimental results verify that our weighted Laplacian methods is indeed superior over existing algorithms in terms of clustering accuracy while also maintains comparable clustering speed.

A neighborhood search based cat swarm optimization algorithm for clustering problems

Clustering is an unsupervised technique that groups the similar data objects into a single subset using a distance function. It is also used to find the optimal set of clusters in a given dataset and each cluster consists of homogenous data objects. In present work, an algorithm based on cat swarm optimization (CSO) is adopted for finding the optimal set of cluster centers for allocating the data objects. Further, some improvements are also incorporated in CSO algorithm for improving clustering performance. These modifications are described as an improved solution search equation to improve convergence rate and an accelerated velocity equation for balancing exploration and exploitation processes of CSO algorithm. Moreover, a neighborhood-based search strategy is introduced to handle local optima problem. The performance of proposed algorithm is tested on eight real-life datasets and compared with well-known clustering algorithms. The simulation results showed that proposed algorithm provides quality results in comparison to existing clustering algorithms.

Model-Based Clustering with Measurement or Estimation Errors.

Model-based clustering with finite mixture models has become a widely used clustering method. One of the recent implementations is MCLUST. When objects to be clustered are summary statistics, such as regression coefficient estimates, they are naturally associated with estimation errors, whose covariance matrices can often be calculated exactly or approximated using asymptotic theory. This article proposes an extension to Gaussian finite mixture modeling—called MCLUST-ME—that properly accounts for the estimation errors. More specifically, we assume that the distribution of each observation consists of an underlying true component distribution and an independent measurement error distribution. Under this assumption, each unique value of estimation error covariance corresponds to its own classification boundary, which consequently results in a different grouping from MCLUST. Through simulation and application to an RNA-Seq data set, we discovered that under certain circumstances, explicitly, modeling estimation errors, improves clustering performance or provides new insights into the data, compared with when errors are simply ignored, whereas the degree of improvement depends on factors such as the distribution of error covariance matrices.

Clustering of Brain Function Network Based on Attribute and Structural Information and Its Application in Brain Diseases.

At present, the diagnosis of brain disease is mainly based on the self-reported symptoms and clinical signs of the patient, which can easily lead to psychiatrists' bias. The purpose of this study is to develop a brain network clustering model to accurately identify brain diseases based on resting state functional magnetic resonance imaging (fMRI) in the absence of clinical information. We use cosine similarity and sub-network kernels to measure attribute similarity and structure similarity, respectively. By integrating the structure similarity and attribute similarity into one matrix, spectral clustering is used to achieve brain network clustering. Finally, we evaluate this method on three diseases: Alzheimer's disease, Bipolar disorder patients, and Schizophrenia. The performance of methods is evaluated by measuring clustering consistency. Clustering consistency is similar to clustering accuracy, which is used to evaluate the consistency between the clustering labels and clinical diagnostic labels of the subjects. The experimental results show that our proposed method can significantly improve clustering performance, with a consistency of 60.6% for Alzheimer's disease, with a consistency of 100% for Schizophrenia, with a consistency of 100% for Bipolar disorder patients.

Nonnegative self-representation with a fixed rank constraint for subspace clustering

A number of approaches to graph-based subspace clustering, which assumes that the clustered data points were drawn from an unknown union of multiple subspaces, have been proposed in recent years. Despite their successes in computer vision and data mining, most neglect to simultaneously consider global and local information, which may improve clustering performance. On the other hand, the number of connected components reflected by the learned affinity matrix is commonly inconsistent with the true number of clusters. To this end, we propose an adaptive affinity matrix learning method, nonnegative self-representation with a fixed rank constraint (NSFRC), in which the nonnegative self-representation and an adaptive distance regularization jointly uncover the intrinsic structure of data. In particular, a fixed rank constraint as a prior is imposed on the Laplacian matrix associated with the data representation coefficients to urge the true number of clusters to exactly equal the number of connected components in the learned affinity matrix. Also, we derive an efficient iterative algorithm based on an augmented Lagrangian multiplier to optimize NSFRC. Extensive experiments conducted on real-world benchmark datasets demonstrate the superior performance of the proposed method over some state-of-the-art approaches.

Cosine kernel based density peaks clustering algorithm

Density peaks clustering (DPC) determines the density peaks according to density-distance, and local density computation significantly impacts the clustering performance of the DPC algorithm. Following this lead, a revised DPC algorithm based on cosine kernel is proposed and examined in this paper. The cosine kernel function uses local information of datasets to define the local density, which not only finds the position difference of different samples within the cutoff distance, but also balances the influence of centre points and boundary points of clusters on local density of samples. Theoretical analysis and experimental verification are included to demonstrate the proposed algorithm's improvement in clustering performance and computational time over the DPC algorithm.

Efficient and Effective Regularized Incomplete Multi-View Clustering.

Incomplete multi-view clustering (IMVC) optimally combines multiple pre-specified incomplete views to improve clustering performance. Among various excellent solutions, the recently proposed multiple kernel k-means with incomplete kernels (MKKM-IK) forms a benchmark, which redefines IMVC as a joint optimization problem where the clustering and kernel matrix imputation tasks are alternately performed until convergence. Though demonstrating promising performance in various applications, we observe that the manner of kernel matrix imputation in MKKM-IK would incur intensive computational and storage complexities, over-complicated optimization and limitedly improved clustering performance. In this paper, we first propose an Efficient and Effective Incomplete Multi-view Clustering (EE-IMVC) algorithm to address these issues. Instead of completing the incomplete kernel matrices, EE-IMVC proposes to impute each incomplete base matrix generated by incomplete views with a learned consensus clustering matrix. Moreover, we further improve this algorithm by incorporating prior knowledge to regularize the learned consensus clustering matrix. Two three-step iterative algorithms are carefully developed to solve the resultant optimization problems with linear computational complexity, and their convergence is theoretically proven. After that, we theoretically study the generalization bound of the proposed algorithms. Furthermore, we conduct comprehensive experiments to study the proposed algorithms in terms of clustering accuracy, evolution of the learned consensus clustering matrix and the convergence. As indicated, our algorithms deliver their effectiveness by significantly and consistently outperforming some state-of-the-art ones.

Adaptive Anchor-Based Partial Multiview Clustering

Partial multiview clustering, which aims to effectively merge multiple prespecified incomplete views to improve clustering performance, is a research hotspot and difficulty in the field of machine learning. Guo et al . proposed a partial multiview clustering method (APMC) based on anchor graph, which uses a Gauss kernel function to solve the similarity matrix. The Gaussian kernel function is sensitive to parameter $\sigma $ , and it is difficult to find the optimal value only by stepwise adjustment in practical applications. This undoubtedly affects the practicality of the APMC algorithm. To address this issue, an adaptive partial multiview clustering method based on anchor graph (AAPMC) is developed in this paper, which proposes an adaptive neighbor assignment strategy and utilizes it to improve the anchor-based similarity matrix computation of each view. The method proposed in this paper uses an adaptive method to solve the similarity matrix, eliminating the tediousness of parameter adjustment. In addition, the non-iterative method based on anchors is used to solve the optimal solution with low time complexity and is suitable for large-scale datasets. In short, our method is simple and effective, and it is easier to implement in practice. Extensive experiments show that our model can not only effectively solve the parameter setting problem, but also performs better than the state-of-the-art partial multiview clustering methods.

Multi-View Spectral Clustering with High-Order Optimal Neighborhood Laplacian Matrix

Multi-view spectral clustering can effectively reveal the intrinsic cluster structure among data by performing clustering on the learned optimal embedding across views. Though demonstrating promising performance in various applications, most of existing methods usually linearly combine a group of pre-specified first-order Laplacian matrices to construct the optimal Laplacian matrix, which may result in limited representation capability and insufficient information exploitation. Also, storing and implementing complex operations on the {$n\times n}$ Laplacian matrices incurs intensive storage and computation complexity. To address these issues, this paper first proposes a multi-view spectral clustering algorithm that learns a high-order optimal neighborhood Laplacian matrix, and then extends it to the late fusion version for accurate and efficient multi-view clustering. Specifically, our proposed algorithm generates the optimal Laplacian matrix by searching the neighborhood of the linear combination of both the first-order and high-order base Laplacian matrices simultaneously. By this way, the representative capacity of the learned optimal Laplacian matrix is enhanced, which is helpful to better utilize the hidden high-order connection information among data, leading to improved clustering performance. We design an efficient algorithm with proved convergence to solve the resultant optimization problem. Extensive experimental results on nine datasets demonstrate the superiority of the proposed algorithm