Cluster Indicator Matrix Research Articles

Fast correntropy-based multi-view clustering with prototype graph factorization

As a consequence of the ability to incorporate information from different perspectives, multi-view clustering has gained significant attention. Nevertheless, 1) its high computational cost, particularly when processing large-scale and high-dimensional multi-view data, restricts its applications in practice; and 2) complex noise in real-world data also challenges the robustness of existing algorithms. To tackle the above challenges, we develop a fast correntropy-based multi-view clustering algorithm with prototype graph factorization (FCMCPF). FCMCPF first adopts prototype graphs to effectively mitigate the complexity associated with graph construction, thereby reducing it from a quadratic complexity to a linear one. Then, it decomposes these prototype graphs under the correntropy criterion to robustly find the cluster indicator matrix without any post-processing. To solve the non-convex and non-linear model, we devise a fast half-quadratic-based strategy to first convert it into a convex formulation and then swiftly complete the optimization via the matrix properties of orthogonality and trace. The extensive experiments conducted on noisy and real-world datasets illustrate that FCMCPF is highly efficient and robust compared to other advanced algorithms, with comparable or even superior clustering effectiveness.

Effective semi-supervised graph clustering with pairwise constraints

Semi-supervised graph clustering with constraints has received considerable attention in the last decade. They use pre-given constraints to guide the clustering process and improve the performance. Nonetheless, most of related research works still face two main issues: (1) cannot-link problem: how to ensure that instances under the cannot-link constraint are located in different clusters as much as possible. (2) The high computational cost caused by eigendecomposition. To tackle these problems, this paper proposes a novel method for directly solving the clustering indicator matrix without high complexity computational steps. Then, we propose an effective and simple algorithm guided by the pre-given constraints, which is able to simultaneously optimize the associated multiple pairs of constraints, allowing the associated constraints to achieve the related co-optimal solution. Extensive experiments are conducted to verify that our approach significantly reduces the violation rate of the prior constraints and the clustering performance of the proposed method outperforms seven other state-of-the-art semi-supervised clustering methods on 16 real benchmark datasets.

Tensor-based global block-diagonal structure radiation for incomplete multiview clustering

Incomplete multiview clustering (IMVC) has attracted extensive attention in the field of machine learning due to its excellent performance in handling incomplete multiview data. However, existing IMVC methods have certain limitions:1) the completion methods are not flexible enough for cases where view information is arbitrarily missing; 2) the existing methods cannot fully exploit the information contained in the clustering indicator matrix, which is crucial to the clustering results. To solve these problems, we propose a novel method, i.e., tensor-based global block-diagonal structure radiation for incomplete multiview clustering (TGBSR). First, we utilize the information from known samples to construct similarity graphs and integrate these matrices into a third-order tensor. Second, to explore the consensus information among the different similarity graphs, we introduce the spectral embedding technique to obtain the clustering indicator matrix, i.e., U. Third, and more importantly, to fully explore the information contained in the clustering indicator matrix, we introduce a novel matrix, i.e., the UUT, which is low-rank and naturally has an excellent block diagonal structure, and then incorporate it as a new view into the aforementioned tensor, which is constrained by the tensor nuclear norm (TNN), to mine the high-order correlations among all the different views. Finally, we combine all these steps into a unified framework and develop an effective iterative optimization procedure to solve it. The experimental results on several well-known datasets demonstrate the effectiveness of our method. The code is publicly available at https://github.com/hhipp-999/TGBSR.

Dual space-based fuzzy graphs and orthogonal basis clustering for unsupervised feature selection

Unsupervised feature selection (UFS) takes an important position because gaining the class labels is laborious or even impossible. In the domain of UFS, clustering is a major means to exploit label information. The existing methods either could not model a clear clustering structure or could not utilize the local structure of data. Consequently, a new clustering method in combination with graph learning is proposed in this work. Specifically, for clustering, orthogonal basis clustering is introduced, where orthogonal constraints are imposed on the cluster center matrix and the clustering matrix. The clustering indicator matrix is also imposed by a non-negative constraint. A clearer clustering structure and more independent clustering centers are obtained through these constraints. For local preservation, given that traditional graphs for keeping the local manifold are faced with the problem of imbalanced neighbors, the fuzzy graph is introduced to acquire a robust structure, which is applied to both data space and feature space. The topological structure in these spaces can be well maintained. For the choice of salient features, ℓ2,0-norm regularization is imposed on the projection matrix. The object function is solved alternately. Then, a feature selection algorithm is designed. Experiments are designed and performed on nine real-world data sets. The results attest to the effectiveness of the proposed algorithm compared with other relevant algorithms.

Attention non-negative spectral clustering

Deep spectral clustering is an advanced unsupervised deep learning method with widespread applications. However, when applied to large-scale datasets, it may confront problems such as an unstable training process, limited scalability, and diminished interpretability. To mitigate these limitations, a novel unsupervised deep learning algorithm named Attention Non-negative Spectral Clustering (ANSC) is proposed. Specifically, a network architecture called the Spectral Attention Network (SAN) is designed to serve as the backbone network for ANSC. This design aims to improve the stability of the training process and the model’s scalability. This attention-based network architecture includes a feature extraction network, an eigenvector mapping network, and an orthogonalization module. Furthermore, to enhance the interpretability of ANSC, the objective function incorporates a cluster indicator matrix by introducing non-negative constraints. Additionally, the ℓ2,1-norm is employed as regularization to approximate the solution of the objective function. An analysis of a lower bound for the network size of ANSC is conducted based on the VC dimension. Extensive experiments validate that ANSC outperforms state-of-the-art clustering methods in terms of clustering accuracy, normalized mutual information, and adjusted rand index.

Learning Cluster-Wise Anchors for Multi-View Clustering

Due to its effectiveness and efficiency, anchor based multi-view clustering (MVC) has recently attracted much attention. Most existing approaches try to adaptively learn anchors to construct an anchor graph for clustering. However, they generally focus on improving the diversity among anchors by using orthogonal constraint and ignore the underlying semantic relations, which may make the anchors not representative and discriminative enough. To address this problem, we propose an adaptive Cluster-wise Anchor learning based MVC method, CAMVC for short. We first make an anchor cluster assumption that supposes the prior cluster structure of target anchors by pre-defining a consensus cluster indicator matrix. Based on the prior knowledge, an explicit cluster structure of latent anchors is enforced by learning diverse cluster centroids, which can explore both inter-cluster diversity and intra-cluster consistency of anchors, and improve the subspace representation discrimination. Extensive results demonstrate the effectiveness and superiority of our proposed method compared with some state-of-the-art MVC approaches.

Spectral clustering with linear embedding: A discrete clustering method for large-scale data

In recent decades, spectral clustering has found widespread applications in various real-world scenarios, showcasing its effectiveness. Traditional spectral clustering typically follows a two-step procedure to address the optimization problem. However, this approach may result in substantial information loss and performance decline. Furthermore, the eigenvalue decomposition, a key step in spectral clustering, entails cubic computational complexity. This paper incorporates linear embedding into the objective function of spectral clustering and proposes a direct method to solve the indicator matrix. Moreover, our method achieves a linear time complexity with respect to the input data size. Our method, referred to as Spectral Clustering with Linear Embedding (SCLE), achieves a direct and efficient solution and naturally handles out-of-sample data. SCLE initiates the process with balanced and hierarchical K-means, effectively partitioning the input data into balanced clusters. After generating anchors, we compute a similarity matrix based on the distances between the input data points and the generated anchors. In contrast to the conventional two-step spectral clustering approach, we directly solve the cluster indicator matrix at a linear time complexity. Extensive experiments across multiple datasets underscore the efficiency and effectiveness of our proposed SCLE method.

Towards a unified framework for graph-based multi-view clustering

Recently, clustering data collected from various sources has become a hot topic in real-world applications. The most common methods for multi-view clustering can be divided into several categories: Spectral clustering algorithms, subspace multi-view clustering algorithms, matrix factorization approaches, and kernel methods. Despite the high performance of these methods, they directly fuse all similarity matrices of all views and separate the affinity learning process from the multiview clustering process. The performance of these algorithms can be affected by noisy affinity matrices. To overcome this drawback, this paper presents a novel method called One Step Multi-view Clustering via Consensus Graph Learning and Nonnegative Embedding (OSMGNE). Instead of directly merging the similarity matrices of different views, which may contain noise, a step of learning a consensus similarity matrix is performed. This step forces the similarity matrices of different views to be too similar, which eliminates the problem of noisy data. Moreover, the use of the nonnegative embedding matrix (soft cluster assignment matrix makes it possible to directly obtain the final clustering result without any extra step. The proposed method can solve five subtasks simultaneously. It jointly estimates the similarity matrix of all views, the similarity matrix of each view, the corresponding spectral projection matrix, the unified clustering indicator matrix, and automatically gives the weight of each view without the use of hyper-parameters. In addition, another version of our method is also studied in this paper. This method differs from the first one by using a consensus spectral projection matrix and a consensus Laplacian matrix over all views. An iterative algorithm is proposed to solve the optimization problem of these two methods. The two proposed methods are tested on several real datasets, which prove their superiority.

FINC: An Efficient and Effective Optimization Method for Normalized Cut.

The optimization methods for solving the normalized cut model usually involve three steps, i.e., problem relaxation, problem solving and post-processing. However, these methods are problematic in both performance since they do not directly solve the original problem, and efficiency since they usually depend on the time-consuming eigendecomposition and k-means (or spectral rotation) for post-processing. In this paper, we propose a fast optimization method to speedup the classical normalized cut clustering process, in which an auxiliary variable is introduced and alternatively updated along with the cluster indicator matrix. The new method is faster than the conventional three-step optimization methods since it solves the normalized cut problem in one step. Theoretical analysis reveals that the new method is able to monotonically decrease the normalized cut objective function and converge in finite iterations. Moreover, we have proposed efficient methods for adjust two regularization parameters. Extensive experimental results show the superior performance of the new method. Moreover, it is faster than the existing methods for solving the normalized cut.

Fast Multiview Anchor-Graph Clustering.

Due to its high computational complexity, graph-based methods have limited applicability in large-scale multiview clustering tasks. To address this issue, many accelerated algorithms, especially anchor graph-based methods and indicator learning-based methods, have been developed and made a great success. Nevertheless, since the restrictions of the optimization strategy, these accelerated methods still need to approximate the discrete graph-cutting problem to a continuous spectral embedding problem and utilize different discretization strategies to obtain discrete sample categories. To avoid the loss of effectiveness and efficiency caused by the approximation and discretization, we establish a discrete fast multiview anchor graph clustering (FMAGC) model that first constructs an anchor graph of each view and then generates a discrete cluster indicator matrix by solving the discrete multiview graph-cutting problem directly. Since the gradient descent-based method makes it hard to solve this discrete model, we propose a fast coordinate descent-based optimization strategy with linear complexity to solve it without approximating it as a continuous one. Extensive experiments on widely used normal and large-scale multiview datasets show that FMAGC can improve clustering effectiveness and efficiency compared to other state-of-the-art baselines.

Parameter-Free Multiview K -Means Clustering With Coordinate Descent Method.

Recently, more and more real-world datasets have been composed of heterogeneous but related features from diverse views. Multiview clustering provides a promising attempt at a solution for partitioning such data according to heterogeneous information. However, most existing methods suffer from hyper-parameter tuning trouble and high computational cost. Besides, there is still an opportunity for improvement in clustering performance. To this end, a novel multiview framework, called parameter-free multiview k -means clustering with coordinate descent method (PFMVKM), is presented to address the above problems. Specifically, PFMVKM is completely parameter-free and learns the weights via a self-weighted scheme, which can avoid the intractable process of hyper-parameters tuning. Moreover, our model is capable of directly calculating the cluster indicator matrix, with no need to learn the cluster centroid matrix and the indicator matrix simultaneously as previous multiview methods have to do. What's more, we propose an efficient optimization algorithm utilizing the idea of coordinate descent, which can not only reduce the computational complexity but also improve the clustering performance. Extensive experiments on various types of real datasets illustrate that the proposed method outperforms existing state-of-the-art competitors and conforms well with the actual situation.

Anchor-graph regularized orthogonal concept factorization for document clustering

Concept factorization (CF) has attracted widespread attention for its promising performance in document clustering. Among various CF variants, graph-regularized CF is the most impressive type, which can improve clustering effectiveness by exploring structural information. Nevertheless, their clustering efficiency is restricted by the following considerations: (1) the introduction of the full-sample graph is accompanied by an increase in computational complexity; (2) most of them require intensive multiplications in optimization, which impair the optimization efficiency. To address these issues, in this work, we propose an anchor-graph regularized orthogonal concept factorization (AROCF) method to enhance the clustering efficiency and effectiveness in document clustering tasks. Firstly, AROCF approximates the full-sample graph with a small-scale anchor graph to reduce the complexity of graph construction from quadratic to linear. Then, one of the factor matrices is constrained as the cluster indicator matrix in our method, which can avoid extra efficiency loss in K-means after optimization. Finally, an orthogonal constraint is employed to restrict the freedom of factorization to increase the clustering effectiveness. To optimize the AROCF model, we develop a fast optimization strategy by combining the trace and orthogonality of matrices. Extensive experiments on various document datasets demonstrate the effectiveness and efficiency of AROCF.

Discrete and Balanced Spectral Clustering with Scalability.

Spectral Clustering (SC) has been the main subject of intensive research due to its remarkable clustering performance. Despite its successes, most existing SC methods suffer from several critical issues. Firstly, they typically involve two independent stages, i.e., learning the continuous relaxation matrix followed by the discretization of the cluster indicator matrix. This two-stage approach can result in suboptimal solutions that negatively impact the clustering performance. Secondly, these methods are hard to maintain the balance property of clusters inherent in many real-world data, which restricts their practical applicability. Finally, these methods are computationally expensive and hence unable to handle large-scale datasets. In light of these limitations, we present a novel Discrete and Balanced Spectral Clustering with Scalability (DBSC) model that integrates the learning the continuous relaxation matrix and the discrete cluster indicator matrix into a single step. Moreover, the proposed model also maintains the size of each cluster approximately equal, thereby achieving soft-balanced clustering. What's more, the DBSC model incorporates an anchor-based strategy to improve its scalability to large-scale datasets. The experimental results demonstrate that our proposed model outperforms existing methods in terms of both clustering performance and balance performance. Specifically, the clustering accuracy of DBSC on CMUPIE data achieved a 17.93% improvement compared with that of the SOTA methods (LABIN, EBSC, etc.).

Multi-View Discrete Clustering: A Concise Model.

In most existing graph-based multi-view clustering methods, the eigen-decomposition of the graph Laplacian matrix followed by a post-processing step is a standard configuration to obtain the target discrete cluster indicator matrix. However, we can naturally realize that the results obtained by the two-stage process will deviate from that obtained by directly solving the primal clustering problem. In addition, it is essential to properly integrate the information from different views for the enhancement of the performance of multi-view clustering. To this end, we propose a concise model referred to as Multi-view Discrete Clustering (MDC), aiming at directly solving the primal problem of multi-view graph clustering. We automatically weigh the view-specific similarity matrix, and the discrete indicator matrix is directly obtained by performing clustering on the aggregated similarity matrix without any post-processing to best serve graph clustering. More importantly, our model does not introduce an additive, nor does it has any hyper-parameters to be tuned. An efficient optimization algorithm is designed to solve the resultant objective problem. Extensive experimental results on both synthetic and real benchmark datasets verify the superiority of the proposed model.

JGSED: An End-to-End Spectral Clustering Model for Joint Graph Construction, Spectral Embedding and Discretization

Most of the existing graph-based clustering models performed clustering by adopting a two-stage strategy which first completes the spectral embedding from a given fixed graph and then resorts to other clustering methods such as <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$k$</tex-math></inline-formula> means to achieve discrete cluster results. On one hand, such a discretization operation easily causes that the obtained results deviate far from the true solution. On the other hand, clustering performance heavily relies on the quality of graph; therefore, the fixed graph is usually not optimal enough. In addition, clustering by separated steps inevitably breaks the underlying connections among the graph construction, spectral embedding and discretization. To address these drawbacks, in this paper, we propose a new spectral clustering model termed JGSED which integrates the graph construction, spectral embedding and spectral rotation together into a unified objective. JGSED is an end-to-end framework to directly take data as input and output the final binary cluster indicator matrix. An efficient algorithm is proposed to optimize the model variables in JGSED, which can be co-evolved towards the optimum. Extensive experiments are conducted on both synthetic and real data sets and the results demonstrate that JGSED outperforms the other state-of-the-art spectral clustering models, indicating the effectiveness of joint optimization.

Dissimilarity-based indicator graph learning for clustering

Although learning a similarity graph with a 0–1 value is regarded as an important issue for clustering tasks, this topic has been scarcely reported in the literature since its NP-hard. Along these lines, this issue is addressed in the work by introducing a novel concept. Concretely, the indicator graph clustering (IGC) method was proposed. Based on the dissimilarity and coordinate descent method, IGC is able to learn an indicator graph with a 0–1 value, which is just the similarity graph with a 0–1 value. Meanwhile, the clustering result can be directly obtained by the indicator graph. From the experimental results on several datasets, it was verified that IGC performs faster than k-means, especially on high-dimensional datasets. Furthermore, IGC was extended to a multi-view version, named indicator graph clustering for multi-view (IGCMV). Our model fused the clustering indicator matrices from different views, which can be obtained by IGC, into the final clustering indicator matrix by a projection matrix. To the best of our knowledge, this work is the first attempt to learn a consensus indicator graph from indicator graphs of all views. In addition different from most of the existing multi-view clustering algorithms, the proposed IGCMV enables the production of the final clustering result, as well as each view’s clustering result simultaneously. This is helpful for measuring the quality of each view. IGCMV can be efficiently optimized by an alternative iteration strategy. Extensive experiments were also carried out on several benchmark multi-view datasets demonstrating the effectiveness of the developed algorithm to the state-of-the-art multi-view clustering algorithms.

Multiview Clustering via Hypergraph Induced Semi-Supervised Symmetric Nonnegative Matrix Factorization

Nonnegative matrix factorization (NMF) based multiview technique has been commonly used in multiview data clustering tasks. However, previous NMF based multiview clustering approaches fail to take advantage of a small amount of supervisory information to effectively improve the clustering performance, and are easily affected by the additional post-processing method in clustering tasks. To cope with these issues, a novel framework named multiview clustering via hypergraph induced semi-supervised symmetric NMF (MVCHSS) is proposed in this paper for multiview data clustering applications. Specifically, the proposed method has the following features: 1) a new multiview based hypergraph pairwise constraints propagation (MHPCP) algorithm is developed in MVCHSS to construct a set of informative similarity matrices, revealing the high-order relationships effectively and fully utilizing the limited pairwise constraint supervisory information among samples of each view data; 2) the obtained similarity matrices with much supervisory information are not only enforced into the symmetric NMF (SNMF) model, but also incorporated into the graph regularization for each view data; 3) the optimization problem of MVCHSS is formulated for multiview data clustering tasks to acquire a more discriminative clustering indicator matrix (or called consensus assignment matrix) without additional post-processing method. Moreover, the proof of convergence and the computational complexity for MVCHSS are presented. Extensive experiments on five multiview datasets demonstrate that the proposed MVCHSS framework outperforms several state-of-the-art multiview clustering methods.

C2IMUFS: Complementary and Consensus Learning-Based Incomplete Multi-View Unsupervised Feature Selection

Multi-view unsupervised feature selection (MUFS) has been demonstrated as an effective technique to reduce the dimensionality of multi-view unlabeled data. The existing methods assume that all of views are complete. However, multi-view data are usually incomplete, i.e., a part of instances are presented on some views but not all views. Besides, learning the complete similarity graph, as an important promising technology in existing MUFS methods, cannot achieve due to the missing views. In this paper, we propose a complementary and consensus learning-based incomplete multi-view unsupervised feature selection method (C <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> IMUFS) to address the aforementioned issues. Concretely, C <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> IMUFS integrates feature selection into an extended weighted non-negative matrix factorization model equipped with adaptive learning of view-weights and a sparse <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\ell _{2,p}$</tex-math></inline-formula> -norm, which can offer better adaptability and flexibility. By the sparse linear combinations of multiple similarity matrices derived from different views, a complementary learning-guided similarity matrix reconstruction model is presented to obtain the complete similarity graph in each view. Furthermore, C <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> IMUFS learns a consensus clustering indicator matrix across different views and embeds it into a spectral graph term to preserve the local geometric structure. Comprehensive experimental results on real-world datasets demonstrate the effectiveness of C <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> IMUFS compared with state-of-the-art methods.

Balance guided incomplete multi-view spectral clustering

There is a large volume of incomplete multi-view data in the real-world. How to partition these incomplete multi-view data is an urgent realistic problem since almost all of the conventional multi-view clustering methods are inapplicable to cases with missing views. In this paper, a novel graph learning-based incomplete multi-view clustering (IMVC) method is proposed to address this issue. Different from existing works, our method aims at learning a common consensus graph from all incomplete views and obtaining a clustering indicator matrix in a unified framework. To achieve a stable clustering result, a relaxed spectral clustering model is introduced to obtain a probability consensus representation with all positive elements that reflect the data clustering result. Considering the different contributions of views to the clustering task, a weighted multi-view learning mechanism is introduced to automatically balance the effects of different views in model optimization. In this way, the intrinsic information of the incomplete multi-view data can be fully exploited. The experiments on several incomplete multi-view datasets show that our method outperforms the compared state-of-the-art clustering methods, which demonstrates the effectiveness of our method for IMVC.

Graph-Based Soft-Balanced Fuzzy Clustering

Spectral clustering have attracted more and more attention due to their well-defined mathematical frameworks and superior performance. However, there still exist two limitations to be solved: 1) most spectral clustering methods consist of two independent stages, which may cause unpredictable deviation of obtained clustering results from the genuine ones and lead to severe information loss and performance degradation; 2) spectral clustering methods employ the hard clustering mode, which lacks interpretability for data points in the boundary area belonging to multiple clusters. To simultaneously address these challenging issues for spectral clustering, we propose a Graph based soft-Balanced Fuzzy Clustering (GBFC) model. Specifically, we explicitly preserve the nonnegative property of the clustering indicator matrix to enhance the interpretability of clustering results. Moreover, row normalization is imposed on the cluster indicator matrix to show the membership of each data point to different clusters. Additionally, a novel balanced constraint is designed to regularize the clustering results and constrain the size of clusters. We can directly obtain the clustering assignments without any post-processing, and the limitations of the previous two-stage clustering framework can be effectively addressed. Extensive experiments performed on both synthetic datasets and real world datasets demonstrate the superiority and effectiveness of the proposed algorithm compared with several state-of-the-art methods.