Basic Partitions Research Articles

Multiview ensemble clustering of hypergraph p-Laplacian regularization with weighting and denoising

Multiview clustering has gained attention for its ability to incorporate complementary information from multiple sources of data, leading to better clustering results. However, these methods don't sufficiently mine high-dimensional information of non-linear subspace or ignore important high-level information between basic partitions (BPs), which are obtained via single-view clustering that could help to overcome differences between heterogeneous feature spaces and defined in Eq. (4). Additionally, existing multiview ensemble clustering methods neglect the noise arising from the generation phase. In light of this, we first propose a novel multiview subspace clustering method with hypergraph p-Laplacian regularization and denoising. Specifically, to utilize the high-dimensional information, a hypergraph p-Laplacian regularized term is added to the model along with low-rank subspace learning to capture the different hierarchical structures in the data. A denoising algorithm based on KMeans and K-nearest neighbor is used to minimize the noise of similarity matrix. Then utilizing this approach as a backbone, a multiview ensemble clustering of hypergraph p-Laplacian regularization with weighting and denoising method is proposed. A hybrid strategy and a novel global weighting ensemble strategy is further proposed to extract high-level information across all the BPs. By integrating hypergraph p-Laplacian operator, low-rank subspace learning, denoising, and weighting ensemble strategy in a unified framework, this ensemble approach adequately learns latent structures and complementary information. Experimental results on multiple datasets demonstrate the efficacy of this approach.

Double High-Order Correlation Preserved Robust Multi-View Ensemble Clustering

Ensemble clustering (EC), utilizing multiple basic partitions (BPs) to yield a robust consensus clustering, has shown promising clustering performance. Nevertheless, most current algorithms suffer from two challenging hurdles: (1) a surge of EC-based methods only focus on pair-wise sample correlation while fully ignoring the high-order correlations of diverse views. (2) they deal directly with the co-association (CA) matrices generated from BPs, which are inevitably corrupted by noise and thus degrade the clustering performance. To address these issues, we propose a novel Double High-Order Correlation Preserved Robust Multi-View Ensemble Clustering (DC-RMEC) method, which preserves the high-order inter-view correlation and the high-order correlation of original data simultaneously. Specifically, DC-RMEC constructs a hypergraph from BPs to fuse high-level complementary information from different algorithms and incorporates multiple CA-based representations into a low-rank tensor to discover the high-order relevance underlying CA matrices, such that double high-order correlation of multi-view features could be dexterously uncovered. Moreover, a marginalized denoiser is invoked to gain robust view-specific CA matrices. Furthermore, we develop a unified framework to jointly optimize the representation tensor and the result matrix. An effective iterative optimization algorithm is designed to optimize our DC-RMEC model by resorting to the alternating direction method of multipliers. Extensive experiments on seven real-world multi-view datasets have demonstrated the superiority of DC-RMEC compared with several state-of-the-art multi-view ensemble clustering methods.

From Ensemble Clustering to Subspace Clustering: Cluster Structure Encoding.

In this study, we propose a novel algorithm to encode the cluster structure by incorporating ensemble clustering (EC) into subspace clustering (SC). First, the low-rank representation (LRR) is learned from a higher order data relationship induced by ensemble K-means coding, which exploits the cluster structure in a co-association matrix of basic partitions (i.e., clustering results). Second, to provide a fast predictive coding mechanism, an encoding function parameterized by neural networks is introduced to predict the LRR derived from partitions. These two steps are jointly proceeded to seamlessly integrate partition information and original features and thus deliver better representations than the ones obtained from each single source. Moreover, an alternating optimization framework is developed to learn the LRR, train the encoding function, and fine-tune the higher order relationship. Extensive experiments on eight benchmark datasets validate the effectiveness of the proposed algorithm on several clustering tasks compared with state-of-the-art EC and SC methods.

Consensus fuzzy clustering by sequential quadratic programming approach

Existing fuzzy clustering ensemble approaches do not consider dependability. This causes those methods to be fragile in dealing with unsuitable basic partitions. While many ensemble clustering approaches are recently introduced for improvement of the quality of the partitioning, but lack of a median partition based consensus function that considers more participate reliable clusters, remains unsolved problem. Dealing with the mentioned problem, an innovative weighting fuzzy cluster ensemble framework is proposed according to cluster dependability approximation. For combining the fuzzy clusters, a fuzzy co-association matrix is extracted in a weighted manner out of initial fuzzy clusters according to their dependabilities. The suggested objective function is a constrained nonlinear objective function and we solve it by sparse sequential quadratic programming (SSQP). Experimentations indicate our method can outperform modern clustering ensemble approaches.

A multi-view ensemble clustering approach using joint affinity matrix

In multi-view ensemble clustering, the correctly-partitioned data objects should be assigned with a higher weight, thereby helping to decrease the influence of incorrectly-partitioned data objects. Therefore, different data objects should be treated separately instead of being set the same view weight as traditional solutions. In this paper, a multi-view ensemble clustering approach is proposed using joint affinity matrix, which is generated by sample-level weight. Firstly, a new concept of core data objects is defined according to the influence index and Gaussian Mixed Model, and basic partitions and sample-level weights can be yielded for every view. Secondly, a joint affinity matrix, which maintains pairwise similarities of all data objects, is generated using the sample-level weights. Consequently, data objects can be effectively assigned to the correct partition. Thirdly, a multi-view ensemble clustering algorithm is proposed using the joint affinity matrix. In the end, experimental results on benchmark datasets validate the efficacy of the algorithm with state-of-the-art baselines.

GMHCC: high-throughput analysis of biomolecular data using graph-based multiple hierarchical consensus clustering.

Thanks to the development of high-throughput sequencing technologies, massive amounts of various biomolecular data have been accumulated to revolutionize the study of genomics and molecular biology. One of the main challenges in analyzing this biomolecular data is to cluster their subtypes into subpopulations to facilitate subsequent downstream analysis. Recently, many clustering methods have been developed to address the biomolecular data. However, the computational methods often suffer from many limitations such as high dimensionality, data heterogeneity and noise. In our study, we develop a novel Graph-based Multiple Hierarchical Consensus Clustering (GMHCC) method with an unsupervised graph-based feature ranking (FR) and a graph-based linking method to explore the multiple hierarchical information of the underlying partitions of the consensus clustering for multiple types of biomolecular data. Indeed, we first propose to use a graph-based unsupervised FR model to measure each feature by building a graph over pairwise features and then providing each feature with a rank. Subsequently, to maintain the diversity and robustness of basic partitions (BPs), we propose multiple diverse feature subsets to generate several BPs and then explore the hierarchical structures of the multiple BPs by refining the global consensus function. Finally, we develop a new graph-based linking method, which explicitly considers the relationships between clusters to generate the final partition. Experiments on multiple types of biomolecular data including 35 cancer gene expression datasets and eight single-cell RNA-seq datasets validate the effectiveness of our method over several state-of-the-art consensus clustering approaches. Furthermore, differential gene analysis, gene ontology enrichment analysis and KEGG pathway analysis are conducted, providing novel insights into cell developmental lineages and characterization mechanisms. The source code is available at GitHub: https://github.com/yifuLu/GMHCC. The software and the supporting data can be downloaded from: https://figshare.com/articles/software/GMHCC/17111291. Supplementary data are available at Bioinformatics online.

Multi-view subspace clustering via partition fusion

Multi-view clustering is an important approach for analyzing multi-view data in an unsupervised way. Among various methods, the multi-view subspace clustering approach has gained increasing attention due to its encouraging performance. Essentially, it integrates multi-view information into graphs, which are then fed into spectral clustering algorithm for final results. However, its performance may degrade due to noises existing in each individual view or inconsistencies between heterogeneous features. Orthogonal to current work, we propose to fuse multi-view information in a partition space, which enhances the robustness of Multi-view clustering. Specifically, we generate multiple partitions and integrate them to find a shared partition. The proposed model unifies graph learning, generation of basic partitions, and view weight learning. These three components co-evolve towards better quality outputs. We have conducted comprehensive experiments on benchmark datasets and our empirical results verify the effectiveness and robustness of our approach.

Dependability‐based cluster weighting in clustering ensemble

AbstractAfter observing the ensemble success in supervised learning (such as classification), it was extended into unsupervised learning. Therefore, cluster ensemble, which merges multiple basic data partitions or clusters (called as ensemble pool) into an ordinarily better clustering solution usually named as consensus partition, emerged. Any cluster ensemble method tries to optimize a particular criterion during extracting the consensus partition out of the ensemble pool. But traditional cluster ensembles consider all the pool members with the equal importance in making the consensus partition; that is to say that each basic partition or cluster participates in the cluster ensemble algorithm equivalently. Indeed, they ignore to consider any ensemble member according to its importance. But it is obvious that some clusters with more quality deserve more emphasis and some clusters with less quality deserve less emphasis during generating consensus partition. This paper proposes (a) a metric to evaluate quality of any arbitrary cluster, (b) a mechanism to project the computed quality of a cluster into a meaningful weight value, and (c) an approach to apply the weight values of the basic clusters in the cluster ensemble process. Experimental results conducted on a number of real‐world standard datasets indicate that the proposed method outperforms the state of the art methods.

Scalable spectral ensemble clustering via building representative co-association matrix

Ensemble clustering via building co-association matrix and combining multiple basic partitions from the same dataset into the consensus one has been widely used in spectral clustering and subspace clustering. However, with the ever-increasing cost of calculating the co-association matrix, the conventional ensemble clustering algorithm is no longer fit for dealing with the large-scale datasets due to its less scalability and time-consuming. In this paper, we propose a scalable spectral ensemble clustering method via building a representative co-association matrix to improve the ensemble clustering problem. Our method mainly includes constructing a sparse matrix to select the representative points and building the co-association matrix, and a robust and denoising representation for the co-association matrix can be learned through a low-rank constraint in a unified optimization framework. The experiments verify the high efficiency and scalability but less time cost of our method compared with state-of-art clustering methods in the six real-world datasets, especially in the large-scale datasets.

Clustering With Outlier Removal

Cluster analysis and outlier detection are two continuously rising topics in data mining area, which in fact connect to each other deeply. Cluster structure is vulnerable to outliers; inversely, outliers are the points belonging to none of any clusters. Unfortunately, most existing studies do not notice the coupled relationship between these two tasks and handle them separately. In this article, we consider the joint cluster analysis and outlier detection problem, and propose the Clustering with Outlier Removal (COR) algorithm. Specifically, the original space is transformed into a binary space via generating basic partitions. We employ Holoentropy to measure the compactness of each cluster without involving several outlier candidates. To provide a neat and efficient solution, an auxiliary binary matrix is introduced so that COR completely and efficiently solves the challenging problem via a unified K-means— with theoretical supports. Extensive experimental results on numerous data sets in various domains demonstrate the effectiveness and efficiency of COR significantly over state-of-the-art methods in terms of cluster validity and outlier detection. Some key factors including the basic partition number and generation strategy in COR with an application on abnormal flight trajectory detection are further analyzed for practical use.

Feature Selection with Unsupervised Consensus Guidance

Most of the unsupervised feature selection methods employ pseudo labels generated by clustering to guide the feature selection; however, noisy and irrelevant features degrade the cluster structure, which is ineffective to supervise feature selection. In light of this, we propose the Consensus Guided Unsupervised Feature Selection (CGUFS) framework, which introduces consensus clustering to generate pseudo labels for feature selection. Generally speaking, multiple diverse basic partitions are generated from the data and the consensus clustering is employed to provide the high-quality and robust partition to guide the feature selection in a one-step framework. In addition, complex constraints such as non-negative are removed due to the crisp indicators of consensus clustering. Based on the CGUFS framework, two formulations are put forward by using the utility function and co-association matrix, respectively, and we propose the (weighted) K-means-like optimization solution for efficient solutions with theoretical supports. Moreover, we extend the CGUFS framework to handle multi-view data feature selection. Extensive experiments on several single-view and multi-view data mining data sets in different domains demonstrate that our methods outperform the most recent state-of-the-art work in terms of effectiveness and efficiency. Some important impact factors and model parameters within CGUFS are thoroughly discussed for practical use.

Partition level multiview subspace clustering

Multiview clustering has gained increasing attention recently due to its ability to deal with multiple sources (views) data and explore complementary information between different views. Among various methods, multiview subspace clustering methods provide encouraging performance. They mainly integrate the multiview information in the space where the data points lie. Hence, their performance may be deteriorated because of noises existing in each individual view or inconsistent between heterogeneous features. For multiview clustering, the basic premise is that there exists a shared partition among all views. Therefore, the natural space for multiview clustering should be all partitions. Orthogonal to existing methods, we propose to fuse multiview information in partition level following two intuitive assumptions: (i) each partition is a perturbation of the consensus clustering; (ii) the partition that is close to the consensus clustering should be assigned a large weight. Finally, we propose a unified multiview subspace clustering model which incorporates the graph learning from each view, the generation of basic partitions, and the fusion of consensus partition. These three components are seamlessly integrated and can be iteratively boosted by each other towards an overall optimal solution. Experiments on four benchmark datasets demonstrate the efficacy of our approach against the state-of-the-art techniques.

Unsupervised feature selection with multi-subspace randomization and collaboration

Unsupervised feature selection has been an important technique in high-dimensional data analysis. Despite significant success, most of the existing unsupervised feature selection methods tend to estimate the underlying structure of data in the original feature space, but lack the ability to explore various subspaces in the high-dimensional space. In this paper, we argue that the use of a large number of random subspaces can significantly benefit the unsupervised feature selection accuracy. Particularly, we propose a new unsupervised feature selection approach based on multi-subspace randomization and collaboration. A balanced subspace randomization scheme is first presented to produce multiple basic feature partitions with similar-sized random subspaces. Then, multiple K-nearest neighbor graphs are constructed in these subspaces, based on which the Laplacian scores of the features in each subspace are obtained w.r.t. their locality preserving power. Thereafter, the obtained feature score vectors of different subspaces in different basic feature partitions are integrated into a full score vector of all features, which takes into account the structure information of various subspaces and can significantly enhance the performance of unsupervised feature selection. Experiments conducted on twenty high-dimensional datasets have demonstrated the high efficiency and robustness of our approach. The MATLAB source code is available at https://www.researchgate.net/publication/334520672.

Simulation of composite shells based on the generating finite elements

In the present paper, a method of generating finite elements (FE) to construct complex-shaped three-dimensional composite multi-grid finite elements (MgFE) of two types is proposed. When constructed m-grid finite element (mgFE), m nested grids are used. The fine grid is generated by the base partitioning of mgFE, taking into account its heterogeneous structure and shape. The displacement functions and stress state in the MgFE described by the equations of the three-dimensional elasticity problems (without introducing any additional simplifying hypotheses), are represented in local rectangular coordinates. When constructed MgFE (without increasing their dimension), arbitrarily small basic partitions can be used arbitrarily closely taking into account the complex inhomogeneous structure and shape of the MgFE, and arbitrarily closely describing the three-dimensional stress state in the MgFE. The 1st type MgFE region is obtained by turning a given flat generating single-grid FE (complex shape) about a given axis by a given angle, the 2nd type MgFE region is obtained by parallel moving the generating FE in a given direction by a specified distance. The 1st type MgFE is applied to calculate some composite rotational shells; the 2nd type is for composite cylindrical shells. The advantages of the proposed MgFE make it possible to account for the inhomogeneous structure of the shells, thus giving rise to small-dimensional discrete models and accurate solutions

Marginalized Multiview Ensemble Clustering.

Multiview clustering (MVC), which aims to explore the underlying cluster structure shared by multiview data, has drawn more research efforts in recent years. To exploit the complementary information among multiple views, existing methods mainly learn a common latent subspace or develop a certain loss across different views, while ignoring the higher level information such as basic partitions (BPs) generated by the single-view clustering algorithm. In light of this, we propose a novel marginalized multiview ensemble clustering (M2VEC) method in this paper. Specifically, we solve MVC in an EC way, which generates BPs for each view individually and seeks for a consensus one. By this means, we naturally leverage the complementary information of multiview data upon the same partition space. In order to boost the robustness of our approach, the marginalized denoising process is adopted to mimic the data corruptions and noises, which provides robust partition-level representations for each view by training a single-layer autoencoder. A low-rank and sparse decomposition is seamlessly incorporated into the denoising process to explicitly capture the consistency information and meanwhile compensate the distinctness between heterogeneous features. Spectral consensus graph partitioning is also involved by our model to make M2VEC as a unified optimization framework. Moreover, a multilayer M2VEC is eventually delivered in a stacked fashion to encapsulate nonlinearity into partition-level representations for handling complex data. Experimental results on eight real-world data sets show the efficacy of our approach compared with several state-of-the-art multiview and EC methods. We also showcase our method performs well with partial multiview data.

Robust Spectral Ensemble Clustering via Rank Minimization

Ensemble Clustering (EC) is an important topic for data cluster analysis. It targets to integrate multiple Basic Partitions (BPs) of a particular dataset into a consensus partition. Among previous works, one promising and effective way is to transform EC as a graph partitioning problem on the co-association matrix, which is a pair-wise similarity matrix summarized by all the BPs in essence. However, most existing EC methods directly utilize the co-association matrix, yet without considering various noises (e.g., the disagreement between different BPs and the outliers) that may exist in it. These noises can impair the cluster structure of a co-association matrix, and thus mislead the final graph partitioning process. To address this challenge, we propose a novel Robust Spectral Ensemble Clustering (RSEC) algorithm in this article. Specifically, we learn low-rank representation (LRR) for the co-association matrix to uncover its cluster structure and handle the noises, and meanwhile, we perform spectral clustering with the learned representation to seek for a consensus partition. These two steps are jointly proceeded within a unified optimization framework. In particular, during the optimizing process, we leverage consensus partition to iteratively enhance the block-diagonal structure of LRR, in order to assist the graph partitioning. To solve RSEC, we first formulate it by using nuclear norm as a convex proxy to the rank function. Then, motivated by the recent advances in non-convex rank minimization, we further develop a non-convex model for RSEC and provide it a solution by the majorization--minimization Augmented Lagrange Multiplier algorithm. Experiments on 18 real-world datasets demonstrate the effectiveness of our algorithm compared with state-of-the-art methods. Moreover, several impact factors on the clustering performance of our approach are also explored extensively.

Consensus Multiple Kernel K-Means Clustering With Late Fusion Alignment and Matrix-Induced Regularization

Multiple kernel clustering (MKC) attracts considerable attention due to its competitive performance in unsupervised learning. However, we observe that most of the existing MKC approaches do not sufficiently consider the correlation between different clustering partitions. As a result, the existing methods would cause redundant and low diversity of selected clustering partitions which deteriorate clustering performance. To address these issues, we propose an effective and efficient multiple kernel $k$ -means clustering method termed Consensus Multiple Kernel Clustering with Late Fusion Alignment and Matrix-Induced Regularization (CMKC-LFA-MR) . Specifically, the correlations between different clustering partitions are calculated as a matrix-induced regularization to encourage the diversity of clustering results. Moreover, we propose to maximally align the consensus partition with the weighted base partitions. The proposed algorithm jointly optimizes the basic clustering partitions and the optimal consensus clustering result. To solve the resultant optimization problem, a three-step alternate algorithm is proposed with both theoretically and experimentally proved convergence. As demonstrated by the experiments on six benchmark datasets, our algorithm outperforms the existing state-of-the-art multi-kernel methods in clustering performance with less time complexity, which demonstrates the effectiveness and efficiency of our proposed algorithm.

Boosting Discrimination Information Based Document Clustering Using Consensus and Classification

Adequate choice of term discrimination information measure (DIM) stipulates guaranteed document clustering. Exercise for the right choice is empirical in nature, and characteristics of data in the documents help experts to speculate a viable solution. Thus, a consistent DIM for the clustering is a mere conjecture and demands intelligent selection of the information measure. In this work, we propose an automated consensus building measure based on a text classifier. Two distinct DIMs construct basic partitions of documents and form base clusters. The consensus building measure method uses the clusters information to find concordant documents and constitute a dataset to train the text classifier. The classifier predicts labels for discordant documents from earlier clustering stage and forms new clusters. The experimentation is performed with eight standard data sets to test efficacy of the proposed technique. The improvement observed by applying the proposed consensus clustering demonstrates its superiority over individual results. Relative Risk (RR) and Measurement of Discrimination Information (MDI) are the two discrimination information measures used for obtaining the base clustering solutions in our experiments.

Identification of cancer subtypes from single-cell RNA-seq data using a consensus clustering method

BackgroundHuman cancers are complex ecosystems composed of cells with distinct molecular signatures. Such intratumoral heterogeneity poses a major challenge to cancer diagnosis and treatment. Recent advancements of single-cell techniques such as scRNA-seq have brought unprecedented insights into cellular heterogeneity. Subsequently, a challenging computational problem is to cluster high dimensional noisy datasets with substantially fewer cells than the number of genes.MethodsIn this paper, we introduced a consensus clustering framework conCluster, for cancer subtype identification from single-cell RNA-seq data. Using an ensemble strategy, conCluster fuses multiple basic partitions to consensus clusters.ResultsApplied to real cancer scRNA-seq datasets, conCluster can more accurately detect cancer subtypes than the widely used scRNA-seq clustering methods. Further, we conducted co-expression network analysis for the identified melanoma subtypes.ConclusionsOur analysis demonstrates that these subtypes exhibit distinct gene co-expression networks and significant gene sets with different functional enrichment.

Accelerating Infinite Ensemble of Clustering by Pivot Features

The infinite ensemble clustering (IEC) incorporates both ensemble clustering and representation learning by fusing infinite basic partitions and shows appealing performance in the unsupervised context. However, it needs to solve the linear equation system with the high time complexity in proportion to O(d3) where d is the concatenated dimension of many clustering results. Inspired by the cognitive characteristic of human memory that can pay attention to the pivot features in a more compressed data space, we propose an acceleration version of IEC (AIEC) by extracting the pivot features and learning the multiple mappings to reconstruct them, where the linear equation system can be solved with the time complexity O(dr2) (r ≪ d). Experimental results on the standard datasets including image and text ones show that our algorithm AIEC improves the running time of IEC greatly but achieves the comparable clustering performance.