Multi-view Subspace Clustering Research Articles

Consistency-Induced Multiview Subspace Clustering.

Multiview clustering has received great attention and numerous subspace clustering algorithms for multiview data have been presented. However, most of these algorithms do not effectively handle high-dimensional data and fail to exploit consistency for the number of the connected components in similarity matrices for different views. In this article, we propose a novel consistency-induced multiview subspace clustering (CiMSC) to tackle these issues, which is mainly composed of structural consistency (SC) and sample assignment consistency (SAC). To be specific, SC aims to learn a similarity matrix for each single view wherein the number of connected components equals to the cluster number of the dataset. SAC aims to minimize the discrepancy for the number of connected components in similarity matrices from different views based on the SAC assumption, that is, different views should produce the same number of connected components in similarity matrices. CiMSC also formulates cluster indicator matrices for different views, and shared similarity matrices simultaneously in an optimization framework. Since each column of similarity matrix can be used as a new representation of the data point, CiMSC can learn an effective subspace representation for the high-dimensional data, which is encoded into the latent representation by reconstruction in a nonlinear manner. We employ an alternating optimization scheme to solve the optimization problem. Experiments validate the advantage of CiMSC over 12 state-of-the-art multiview clustering approaches, for example, the accuracy of CiMSC is 98.06% on the BBCSport dataset.

Mixed structure low-rank representation for multi-view subspace clustering

Mixed structure low-rank representation for multi-view subspace clustering

Enhanced tensor low-rank representation learning for multi-view clustering

Multi-view subspace clustering (MSC), assuming the multi-view data are generated from a latent subspace, has attracted considerable attention in multi-view clustering. To recover the underlying subspace structure, a successful approach adopted recently is subspace clustering based on tensor nuclear norm (TNN). But there are some limitations to this approach that the existing TNN-based methods usually fail to exploit the intrinsic cluster structure and high-order correlations well, which leads to limited clustering performance. To address this problem, the main purpose of this paper is to propose a novel tensor low-rank representation (TLRR) learning method to perform multi-view clustering. First, we construct a 3rd-order tensor by organizing the features from all views, and then use the t-product in the tensor space to obtain the self-representation tensor of the tensorial data. Second, we use the ℓ1,2 norm to constrain the self-representation tensor to make it capture the class-specificity distribution, that is important for depicting the intrinsic cluster structure. And simultaneously, we rotate the self-representation tensor, and use the tensor singular value decomposition-based weighted TNN as a tighter tensor rank approximation to constrain the rotated tensor. For the challenged mathematical optimization problem, we present an effective optimization algorithm with a theoretical convergence guarantee and relatively low computation complexity. The constructed convergent sequence to the Karush–Kuhn–Tucker (KKT) critical point solution is mathematically validated in detail. We perform extensive experiments on four datasets and demonstrate that TLRR outperforms state-of-the-art multi-view subspace clustering methods.

Simultaneous Laplacian embedding and subspace clustering for incomplete multi-view data

In many real multi-view data, some views often lose some information, resulting in incomplete views. It is a challenging task to extract and fuse valuable information from the multi-view data with incomplete views for improving clustering performance. Incomplete multi-view clustering (IMC) is designed to fully mine patterns in data to reduce the negative impact of missing views. Many previous IMC methods are committed to discovers a consensus representation shared by all views. Nevertheless, this representation could severely deviate from the inherent representation of original data due to the loss of information caused by incomplete views. Besides, most existing spectral clustering-based multi-view subspace clustering independently performs similarity graph learning, Laplacian embedding, and discrete indicator matrix learning. This multi-step strategy might result in sharp performance degradation. In this paper, we propose a novel IMC method, referred to as Simultaneous Laplacian Embedding and Subspace Clustering (SLESC), to address the above issues. Specifically, in the paradigm for data self-representation, the proposed SLESC method learns a similarity graph for each view instead of learning a consensus graph for invoking traditional spectral clustering. Similarity graph learning, Laplacian embedding learning, weighting for each view, and discrete indicator matrix learning are seamlessly incorporated into the unified framework. The joint optimal clustering outcomes are therefore possible. Experimental results on real-world datasets in the IMC task demonstrate the effectiveness of the proposed method compared with state-of-the-art baselines.

Multi-view subspace clustering network with block diagonal and diverse representation

Learning multi-view data, and especially multi-view data clustering, is a popular area in computer vision and pattern recognition. The multi-view subspace clustering has achieved a better clustering quality than the single-view subspace clustering, mainly because of the complementarity of multi-view information. First, for not directly pursuing a block diagonal representation matrix of previous ℓ1 or ℓ2 regularizers in a deep subspace clustering network, a k-block diagonal regularizer is proposed to replace traditional regularizers. This block diagonal representation module is integrated into this multi-view subspace clustering network, and it can improve the clustering quality. Secondly, there exists some redundancy among the representation matrices, and a diverse representation module can be introduced into this network. This can boost the diversity of representation matrices, and make learned representation matrices more discriminative and help improve the clustering performance. In this paper, based on the deep subspace clustering network, we integrate the block diagonal and diverse representation into the network, and a multi-view subspace clustering network with the block diagonal and the diverse representation is proposed. The experimental results on the UCI Digit, Caltech101-20, COIL100 and Caltech101-7 datasets have demonstrated the superior performance of the proposed algorithm over other popular multi-view algorithms.

Self-Supervised Information Bottleneck for Deep Multi-View Subspace Clustering.

In this paper, we explore the problem of deep multi-view subspace clustering framework from an information-theoretic point of view. We extend the traditional information bottleneck principle to learn common information among different views in a self-supervised manner, and accordingly establish a new framework called Self-supervised Information Bottleneck based Multi-view Subspace Clustering (SIB-MSC). Inheriting the advantages from information bottleneck, SIB-MSC can learn a latent space for each view to capture common information among the latent representations of different views by removing superfluous information from the view itself while retaining sufficient information for the latent representations of other views. Actually, the latent representation of each view provides a kind of self-supervised signal for training the latent representations of other views. Moreover, SIB-MSC attempts to disengage the other latent space for each view to capture the view-specific information by introducing mutual information based regularization terms, so as to further improve the performance of multi-view subspace clustering. Extensive experiments on real-world multi-view data demonstrate that our method achieves superior performance over the related state-of-the-art methods.

Fast Self-Guided Multi-View Subspace Clustering.

Multi-view subspace clustering is an important topic in cluster analysis. Its aim is to utilize the complementary information conveyed by multiple views of objects to be clustered. Recently, view-shared anchor learning based multi-view clustering methods have been developed to speed up the learning of common data representation. Although widely applied to large-scale scenarios, most of the existing approaches are still faced with two limitations. First, they do not pay sufficient consideration on the negative impact caused by certain noisy views with unclear clustering structures. Second, many of them only focus on the multi-view consistency, yet are incapable of capturing the cross-view diversity. As a result, the learned complementary features may be inaccurate and adversely affect clustering performance. To solve these two challenging issues, we propose a Fast Self-guided Multi-view Subspace Clustering (FSMSC) algorithm which skillfully integrates the view-shared anchor learning and global-guided-local self-guidance learning into a unified model. Such an integration is inspired by the observation that the view with clean clustering structures will play a more crucial role in grouping the clusters when the features of all views are concatenated. Specifically, we first learn a locally-consistent data representation shared by all views in the local learning module, then we learn a globally-discriminative data representation from multi-view concatenated features in the global learning module. Afterwards, a feature selection matrix constrained by the l2,1 -norm is designed to construct a guidance from global learning to local learning. In this way, the multi-view consistent and diverse information can be simultaneously utilized and the negative impact caused by noisy views can be overcame to some extent. Extensive experiments on different datasets demonstrate the effectiveness of our proposed fast self-guided learning model, and its promising performance compared to both, the state-of-the-art non-deep and deep multi-view clustering algorithms. The code of this paper is available at https://github.com/chenzhe207/FSMSC.

Bi-Nuclear Tensor Schatten-p Norm Minimization for Multi-View Subspace Clustering.

Multi-view subspace clustering aims to integrate the complementary information contained in different views to facilitate data representation. Currently, low-rank representation (LRR) serves as a benchmark method. However, we observe that these LRR-based methods would suffer from two issues: limited clustering performance and high computational cost since (1) they usually adopt the nuclear norm with biased estimation to explore the low-rank structures; (2) the singular value decomposition of large-scale matrices is inevitably involved. Moreover, LRR may not achieve low-rank properties in both intra-views and inter-views simultaneously. To address the above issues, this paper proposes the Bi-nuclear tensor Schatten- p norm minimization for multi-view subspace clustering (BTMSC). Specifically, BTMSC constructs a third-order tensor from the view dimension to explore the high-order correlation and the subspace structures of multi-view features. The Bi-Nuclear Quasi-Norm (BiN) factorization form of the Schatten- p norm is utilized to factorize the third-order tensor as the product of two small-scale third-order tensors, which not only captures the low-rank property of the third-order tensor but also improves the computational efficiency. Finally, an efficient alternating optimization algorithm is designed to solve the BTMSC model. Extensive experiments with ten datasets of texts and images illustrate the performance superiority of the proposed BTMSC method over state-of-the-art methods.

Elastic Multi-view Subspace Clustering with Pairwise and High-order Correlations

Elastic Multi-view Subspace Clustering with Pairwise and High-order Correlations

Consensus Low-Rank Multi-View Subspace Clustering With Cross-View Diversity Preserving

Consensus Low-Rank Multi-View Subspace Clustering With Cross-View Diversity Preserving

Enhanced Multi-View Subspace Clustering via Twist Tensor Nuclear Norm and Constraint Propagation

Enhanced Multi-View Subspace Clustering via Twist Tensor Nuclear Norm and Constraint Propagation

Low-rank tensor multi-view subspace clustering via cooperative regularization

Low-rank tensor multi-view subspace clustering via cooperative regularization

Multi-view Subspace Clustering Based on Unified Measure Standard

Multi-view Subspace Clustering Based on Unified Measure Standard

Self-Guided Deep Multiview Subspace Clustering via Consensus Affinity Regularization

Multiview subspace clustering (MVSC) leverages the complementary information among different views of multiview data and seeks a consensus subspace clustering result better than that using any individual view. Though proved effective in some cases, existing MVSC methods often obtain unsatisfactory results since they perform subspace analysis with raw features that are often of high dimensions and contain noises. To remedy this, we propose a self-guided deep multiview subspace clustering (SDMSC) model that performs joint deep feature embedding and subspace analysis. SDMSC comprehensively explores multiview data and strives to obtain a consensus data affinity relationship agreed by features from not only all views but also all intermediate embedding spaces. With more constraints being cast, the desirable data affinity relationship is supposed to be more reliably recovered. Besides, to secure effective deep feature embedding without label supervision, we propose to use the data affinity relationship obtained with raw features as the supervision signals to self-guide the embedding process. With this strategy, the risk that our deep clustering model being trapped in bad local minima is reduced, bringing us satisfactory clustering results in a higher possibility. The experiments on seven widely used datasets show the proposed method significantly outperforms the state-of-the-art clustering methods. Our code is available at https://github.com/kailigo/dmvsc.git.

Hypergraph regularized low-rank tensor multi-view subspace clustering via L1 norm constraint

Hypergraph regularized low-rank tensor multi-view subspace clustering via L1 norm constraint

Multiview Subspace Dual Clustering.

A single clustering refers to the partitioning of data such that the similar data are assigned into the same group, whereas the dissimilar data are separated into different groups. Recently, multiview clustering has received significant attention in recent years. However, most existing works tackle the single-clustering scenario, which only use single clustering to partition the data. In practice, nevertheless, the real-world data are complex and can be clustered in multiple ways depending on different interpretations of the data. Unlike these methods, in this article, we apply dual clustering to multiview subspace clustering. We propose a multiview dual-clustering method to simultaneously explore consensus representation and dual-clustering structure in a unified framework. First, multiview features are integrated into a latent embedding representation through a multiview learning process. Second, the dual-clustering segmentation is incorporated into the subspace clustering framework. Finally, the learned dual representations are assigned to the corresponding clusterings. The proposed approach is efficiently solved using an alternating optimization scheme. Extensive experiments demonstrate the superiority of our method on real-world multiview dual- and single-clustering datasets.

Facilitated low-rank multi-view subspace clustering

Low-rank multi-view subspace clustering has recently attracted increasing attention in the multi-view learning research. Despite significant progress, most existing approaches still suffer from two issues. First, they mostly focus on exploiting the low-rank consistency across multiple views, but often ignore the low-rank structure within each view. Second, they often encounter the expensive time overhead, typically owing to the costs of matrix inversion and singular value decomposition (SVD) at each iteration. In light of this, we propose an efficient and effective approach termed Facilitated Low-rank Multi-view Subspace Clustering (FLMSC) in this paper. In terms of efficiency, our approach factorizes the view-specific representation matrix into two small factor matrices, i.e., an orthogonal dictionary and a latent representation, which mitigates the computation cost of solving SVD problems. In terms of effectiveness, our approach preserves the latent low-rank structure within each view, while at the same time encourages the structural consistency across different views by imposing an agreement term. Based on this, our approach can fully explore the underlying subspace structure of multiple views, so as to better serve for the following spectral clustering. Besides, a facilitated iterative algorithm is developed for the resultant optimization problem, upon which the matrix inversion operation is substantially accelerated during iterations. Experimental results on a variety of multi-view data sets demonstrate the effectiveness and efficiency of our approach. The source code and data sets are available at: https://www.researchgate.net/publication/365349930_FLMSC_v1.

Pure graph-guided multi-view subspace clustering

Multi-view subspace clustering approaches have shown outstanding performance in revealing similarity relationships and complex structures hidden in data. Despite the progress, previous multi-view clustering methods still face two challenges: (1) it is difficult to simultaneously achieve sparsity and connectivity of the affinity graph; (2) existing methods usually separate the graph learning step from the clustering process, which leads to unsatisfactory clustering performance as the final results critically rely on the learned graph. In this paper, we propose to achieve a structured consensus graph for multi-view subspace clustering by leveraging the sparsity and connectivity of each affinity graph. In the proposed method, the pure graph for each view is searched by finding the good neighbors. The multiple pure graphs are further fused into a consensus graph with a block-diagonal structure. That is, the consensus graph is enforced to contain exactly c connected components where c is the number of the clusters. Hence the label to each sample can be directly assigned since each connected component precisely corresponds to an individual cluster. As a result, the proposed model seamlessly accomplishes the subtasks including graph construction, pure graph learning (i.e., good neighbors searching), and cluster label allocation in a mutual reinforcement manner. Extensive experimental results demonstrate the superiority and reliability of our proposed method.

Robust multi-view subspace enhanced representation based on collaborative constraints and HSIC induction

The design of effective multi-view subspace clustering (MSC) algorithms has recently garnered significant research attention. Herein, to effectively improve the recognition performance and anti-noise interference ability of an MSC model, we propose a novel MSC algorithm, termed as robust multi-view subspace enhancement representation, based on collaborative constraints and a Hilbert–Schmidt independence criterion (HSIC) induction method. To mine the complementary information between different views, we apply the HSIC as a diversity regularization term. Specifically, to enhance the diagonal block structure of a subspace representation, a new sparse constraint is introduced on the product of itself and the transpose of the subspace representation matrix in a multi-view subspace learning model. Furthermore, hypergraph regularization and a low-rank idea are considered to capture the local geometric structure and clean data. In addition, to optimize our model, we adopt an augmented Lagrangian multiplier method and discuss the convergence of the model. Extensive experiments on six challenging datasets reveal that the proposed method achieves a highly competent objective performance with and without noisy views, as compared with several state-of-the-art multi-view clustering methods.

Scalable one-stage multi-view subspace clustering with dictionary learning

The clustering of large numbers of heterogeneous features is a hot topic in multi-view communities. Most existing multi-view clustering (MvC) methods employ matrix factorization or anchor strategies to handle large-scale datasets. The former operates on the original data and is, therefore, sensitive to noise and feature redundancy, which is reflected in the final clustering performance. The latter requires post-processing steps to generate the clustering results, which may be suboptimal owing to the isolation steps. To address the above problems, we propose one-stage multi-view subspace clustering with dictionary learning (OSMvSC). Specifically, we integrate dictionary learning, representation coefficient matrix learning, and matrix factorization as a unified learning framework, which directly learns the dictionary and representation coefficient matrix to encode the original multi-view data, and obtains the clustering results with linear time complexity without any postprocessing step. By manipulating the class centroid with the nuclear norm, a more compact and discriminative class centroid representation can be obtained to further improve clustering performance. An effective optimization algorithm with guaranteed convergence is designed to solve the proposed method. Substantial experiments on various real-world multi-view datasets demonstrate the effectiveness and superiority of the proposed method. The source code is available at https://github.com/justcallmewilliam/OSMvSC.