Multi-view Subspace Clustering Research Articles

Incomplete multi-view subspace clustering based on missing-sample recovering and structural information learning

The incomplete multi-view clustering (IMVC) focuses on exploring hidden information from missing samples and complementary information from multiple views to enhance clustering performance. Most of existing methods for IMVC only attempt to obtain a shared consistent representation or graph from the non-missing samples based on matrix factorization. However, these methods rely too much on view alignment information, neglecting the specific-view structural information of different views and the hidden information of missing samples. To address the above problems, a novel method, called incomplete multi-view subspace clustering based on missing-sample recovering and structural information learning (MSR_SIL), is proposed in this paper. Overall, MSR_SIL incorporates the missing-sample recovering in feature space and self-expressiveness coefficient learning in representation space into a unified framework. Specifically, a reconstruction term is introduced to recover missing samples with the help of non-missing ones so that incomplete multi-view is completed. Next, the self-expressiveness coefficient of the recovered complete multi-view is learned. Then, each self-expressiveness coefficient is decomposed into a consistent part to capture the similarity information of view-paired and a specific part to hold the unique information of each view. Especially, to retain the global manifold structure of the multi-view data, Schatten-p norm is applied. Finally, the clustering results can be obtained on the basis of the consistent and specific representation. Experimental results on eleven well-known benchmark multi-view datasets demonstrate that the proposed MSR_SIL yields significant improvements on clustering performance.

Frobenius norm-regularized robust graph learning for multi-view subspace clustering

Frobenius norm-regularized robust graph learning for multi-view subspace clustering

Efficient One-Pass Multi-View Subspace Clustering with Consensus Anchors

Multi-view subspace clustering (MVSC) optimally integrates multiple graph structure information to improve clustering performance. Recently, many anchor-based variants are proposed to reduce the computational complexity of MVSC. Though achieving considerable acceleration, we observe that most of them adopt fixed anchor points separating from the subsequential anchor graph construction, which may adversely affect the clustering performance. In addition, post-processing is required to generate discrete clustering labels with additional time consumption. To address these issues, we propose a scalable and parameter-free MVSC method to directly output the clustering labels with optimal anchor graph, termed as Efficient One-pass Multi-view Subspace Clustering with Consensus Anchors (EOMSC-CA). Specially, we combine anchor learning and graph construction into a uniform framework to boost clustering performance. Meanwhile, by imposing a graph connectivity constraint, our algorithm directly outputs the clustering labels without any post-processing procedures as previous methods do. Our proposed EOMSC-CA is proven to be linear complexity respecting to the data size. The superiority of our EOMSC-CA over the effectiveness and efficiency is demonstrated by extensive experiments. Our code is publicly available at https://github.com/Tracesource/EOMSC-CA.

Low-rank tensor approximation with local structure for multi-view intrinsic subspace clustering

Among various multi-view clustering approaches, tensor-based multi-view subspace clustering methods aim to explore the high-order correlations across varying views and have achieved encouraging effects. Nevertheless, there are still some demerits in them: (1) View-specific information hinders the mining of global consensus. (2) The local structure inside individual view lacks consideration. (3) Clustering results are not utilized to reversely guide the low-rank tensor optimization. In order to tackle these drawbacks, we propose a unified model termed as Low-rank Tensor Approximation with Local Structure for Multi-view Intrinsic Subspace Clustering. Specifically, the proposed model learns multiple intrinsic subspace representations via the rank preserving decomposition, which is to mitigate the impact of view-specific information on enhancing the global consistency. Then, these intrinsic subspace representations are assembled into a 3-order target tensor with tensor nuclear norm constraint. To preserve the consistent locality, we adopt the manifold regularization to constrain each view when mapping into the intrinsic subspace. Furthermore, since the learned label indicator matrix implicitly characterizes the cluster structure, which is used to guide the optimization of the low-rank tensor representation. Finally, abundant experiments on six real-word datasets demonstrate that the proposed method is superior over other state-of-the-art clustering methods.

Fusing Local and Global Information for One-Step Multi-View Subspace Clustering

Multi-view subspace clustering has drawn significant attention in the pattern recognition and machine learning research community. However, most of the existing multi-view subspace clustering methods are still limited in two aspects. (1) The subspace representation yielded by the self-expression reconstruction model ignores the local structure information of the data. (2) The construction of subspace representation and clustering are used as two individual procedures, which ignores their interactions. To address these problems, we propose a novel multi-view subspace clustering method fusing local and global information for one-step multi-view clustering. Our contribution lies in three aspects. First, we merge the graph learning into the self-expression model to explore the local structure information for constructing the specific subspace representations of different views. Second, we consider the multi-view information fusion by integrating these specific subspace representations into one common subspace representation. Third, we combine the subspace representation learning, multi-view information fusion, and clustering into a joint optimization model to realize the one-step clustering. We also develop an effective optimization algorithm to solve the proposed method. Comprehensive experimental results on nine popular multi-view data sets confirm the effectiveness and superiority of the proposed method by comparing it with many state-of-the-art multi-view clustering methods.

Hyper-Laplacian regularized multi-view subspace clustering with jointing representation learning and weighted tensor nuclear norm constraint

In recent years, tensor-Singular Value Decomposition (t-SVD) based tensor nuclear norm has achieved remarkable progress in multi-view subspace clustering. However, most existing clustering methods still have the following shortcomings: (a) It has no meaning in practical applications for singular values to be treated equally. (b) They often ignore that data samples in the real world usually exist in multiple nonlinear subspaces. In order to solve the above shortcomings, we propose a hyper-Laplacian regularized multi-view subspace clustering model that joints representation learning and weighted tensor nuclear norm constraint, namely JWHMSC. Specifically, in the JWHMSC model, firstly, in order to capture the global structure between different views, the subspace representation matrices of all views are stacked into a low-rank constrained tensor. Secondly, hyper-Laplace graph regularization is adopted to preserve the local geometric structure embedded in the high-dimensional ambient space. Thirdly, considering the prior information of singular values, the weighted tensor nuclear norm (WTNN) based on t-SVD is introduced to treat singular values differently, which makes the JWHMSC more accurately obtain the sample distribution of classification information. Finally, representation learning, WTNN constraint and hyper-Laplacian graph regularization constraint are integrated into a framework to obtain the overall optimal solution of the algorithm. Compared with the state-of-the-art method, the experimental results on eight benchmark datasets show the good performance of the proposed method JWHMSC in multi-view clustering.

Nonconvex low-rank and sparse tensor representation for multi-view subspace clustering

Nonconvex low-rank and sparse tensor representation for multi-view subspace clustering

Multi-view Subspace Clustering with View Correlations via low-rank tensor learning

With the purpose of boosting clustering performance, the manner of excavating underlying view correlations is an important issue of multi-view subspace clustering (MSC). Nevertheless, regarding most MSC approaches are centered on the view correlations in multiple subspace representations and neglect the view correlations in multiple feature representations. To address this limitation, this paper introduces a method, dubbed Multi-view Subspace Clustering with View Correlations (MSCVC), which excavates underlying view correlations in both multiple feature representations and multiple subspace representations simultaneously. Specifically, the Multi-view Principal Component Analysis (MPCA) is designed to capture the view correlations contained in multiple feature representations by using the orthogonal mapping and low-rank tensor constraint. As a consequence, view correlations in multiple feature representations can be effectively investigated and simultaneously embedded in the learned new feature representations. For view correlations in multiple subspace representations, the new feature representations learned in MPCA are employed and the low-rank tensor constraint is used as well. Experiments are conducted on nine benchmark datasets to demonstrate the progressiveness of our MSCVC compared to several state-of-the-art algorithms.

Double structure scaled simplex representation for multi-view subspace clustering

In the era of big data, there are an increasing number of multisource information data, and multi-view clustering (MVC) algorithms have developed rapidly. However, the affinity matrix learned by most MVC methods is not clean and precise enough and cannot describe the latent structure of multi-view data accurately, which results in poor clustering performance. In this paper, we propose a novel Double Structure Scaled Simplex Representation (DSSSR) method for MVC. Initially, we concatenate the multi-view data into a joint representation. Then, we use the scaled simplex representation (SSR) method on the concatenated data to obtain the affinity matrix. However, the affinity matrix is not clean and precise. Therefore, we use the SSR method again on the obtained affinity matrix to obtain a more accurate and clean affinity matrix. Furthermore, the two-step SSR is integrated into a unified optimization framework, a clean and accurate affinity matrix can be obtained, and the sum of each column vector of the affinity matrix is constrained to be nonnegative and equal to s (0<s<1), which can be adjusted to obtain the best clustering performance. Finally, an efficient optimization algorithm based on the augmented Lagrangian method (ALM) for solving the objective function is also designed. The experimental results on some datasets show that this algorithm has better clustering performance than some state-of-the-art algorithms.

Robust deep multi-view subspace clustering networks with a correntropy-induced metric

Robust deep multi-view subspace clustering networks with a correntropy-induced metric

Auto-Weighted Graph Regularization and Residual Compensation for Multi-view Subspace Clustering

Auto-Weighted Graph Regularization and Residual Compensation for Multi-view Subspace Clustering

Multi-view Subspace Clustering via Two Dimensional Structures Preservation on Heterogeneous Features

Multi-view Subspace Clustering via Two Dimensional Structures Preservation on Heterogeneous Features

Robust multi-view subspace clustering based on consensus representation and orthogonal diversity

The main purpose of multi-view subspace clustering is to reveal the intrinsic low-dimensional architecture of data points according to their multi-view characteristics. Exploring the potential relationship from views is one of the most essential research focuses of the multi-view task. To better utilize the complementary and consistency information from distinct views, we propose a novel robust subspace clustering approach based on consensus representation and orthogonal diversity (RMSCCO). A novel defined orthogonality term is adopted to improve the diversity and decrease the redundance of learning subspace representation. The consensus representation and subspace learning are integrated into one unified framework to characterize the consistency from views. The grouping-enhanced representation is utilized to maintain the local geometric architecture in the original data space. The ℓ2,1-norm regularizer constraint to the noise is applied to improve the robustness. Finally, an optimization algorithm is exploited to solve RMSCCO with the Alternating Direction Method of Multipliers (ADMM). Extensive experimental results on six challenging datasets demonstrate that our approach has accomplished highly qualified performance.

Multi-view subspace clustering with inter-cluster consistency and intra-cluster diversity among views

Multi-view subspace clustering with inter-cluster consistency and intra-cluster diversity among views

Constrained Tensor Representation Learning for Multi-View Semi-Supervised Subspace Clustering

Multi-view subspace clustering is an effective method to partition data into their corresponding categories. Nevertheless, existing multi-view subspace clustering approaches generally operate in a purely unsupervised manner, while ignoring the valuable weakly supervised information that can be readily obtained in many practical applications. In this paper, we consider the weakly supervised form of sample pair constraints, and devote to promoting the performance of multi-view subspace clustering with the aid of such prior knowledge. To achieve this goal, inspired by the intrinsic block diagonal structure of ideal low-rank representation (LRR), we propose a novel regularization to integrate must-link, cannot-link and normalization constraints into a unified formulation. The proposed regularization can be regarded as a general description for sample pairwise constraints, and thus provides a flexible framework for multi-view semi-supervised subspace clustering task. Furthermore, we devise a contrained tensor representation learning (CTRL) model that takes advantage of our proposed regularization to facilitate the learning of the desired representation tensor. An efficient optimization algorithm based on alternating direction minimization strategy is carefully designed to solve the proposed CTRL model. Extensive experiments on eight challenging real-world datasets are conducted, and the results validate the effectiveness of our designed pairwise constraints regularization, as well as the superiority of the proposed CTRL model.

Self-Paced Enhanced Low-Rank Tensor Kernelized Multi-View Subspace Clustering

This paper addresses the multi-view subspace clustering problem and proposes the self-paced enhanced low-rank tensor kernelized multi-view subspace clustering (SETKMC) method, which is based on two motivations: (1) singular values of the representations and multiple instances should be treated differently. The reasons are that larger singular values of the representations usually quantify the major information and should be less penalized; samples with different degrees of noise may have various reliability for clustering. (2) many existing methods may cause the degraded performance when multi-view features reside in different nonlinear subspaces. This is because they usually assumed that multiple features lie within the union of several linear subspaces. SETKMC integrates the nonconvex tensor norm, self-paced learning, and kernel trick into a unified model for multi-view subspace clustering. The nonconvex tensor norm imposes different weights on different singular values. The self-paced learning gradually involves instances from more reliable to less reliable ones while the kernel trick aims to handle the multi-view data in nonlinear subspaces. One iterative algorithm is proposed based on the alternating direction method of multipliers. Extensive results on seven real-world datasets show the effectiveness of the proposed SETKMC compared to fifteen state-of-the-art multi-view clustering methods.

Hyper-Laplacian Regularized Nonconvex Low-Rank Representation for Multi-View Subspace Clustering

Multi-view subspace clustering methods used consensus and supplementary principles to learn the shared self-representation matrix or tensor have been applied to multiple fields. The existing advanced multi-view subspace clustering methods are mainly based on the extension of low-rank representation from matrix to tensor. However, the tensor optimization methods have two limitations: they cannot retain the local geometric structure of data features residing in multiple nonlinear subspaces; they represent the low-rank structure based on the tensor nuclear norm, which will cause undesirable low-rank approximation. To solve these problems, we propose a hyper-Laplacian regularized Nonconvex Low-rank Representation (HNLR) method for multi-view subspace clustering. HNLR uses hyper-Laplacian regularizer to capture the high-order local geometry structure of each view. In addition, by introducing a nonconvex Laplace function to replace the tensor nuclear norm, HNLR can greatly improve the approximate performance of the global low-rank structure. Based on the alternating direction method of multiplier, we design an effective alternate iteration strategy to optimize HNLR model. Experimental results on eight real datasets have proved the superiority of our proposed method.

Fast Parameter-Free Multi-View Subspace Clustering With Consensus Anchor Guidance.

Multi-view subspace clustering has attracted intensive attention to effectively fuse multi-view information by exploring appropriate graph structures. Although existing works have made impressive progress in clustering performance, most of them suffer from the cubic time complexity which could prevent them from being efficiently applied into large-scale applications. To improve the efficiency, anchor sampling mechanism has been proposed to select vital landmarks to represent the whole data. However, existing anchor selecting usually follows the heuristic sampling strategy, e.g. k -means or uniform sampling. As a result, the procedures of anchor selecting and subsequent subspace graph construction are separated from each other which may adversely affect clustering performance. Moreover, the involved hyper-parameters further limit the application of traditional algorithms. To address these issues, we propose a novel subspace clustering method termed Fast Parameter-free Multi-view Subspace Clustering with Consensus Anchor Guidance (FPMVS-CAG). Firstly, we jointly conduct anchor selection and subspace graph construction into a unified optimization formulation. By this way, the two processes can be negotiated with each other to promote clustering quality. Moreover, our proposed FPMVS-CAG is proved to have linear time complexity with respect to the sample number. In addition, FPMVS-CAG can automatically learn an optimal anchor subspace graph without any extra hyper-parameters. Extensive experimental results on various benchmark datasets demonstrate the effectiveness and efficiency of the proposed method against the existing state-of-the-art multi-view subspace clustering competitors. These merits make FPMVS-CAG more suitable for large-scale subspace clustering. The code of FPMVS-CAG is publicly available at https://github.com/wangsiwei2010/FPMVS-CAG.

Logarithmic Schatten- p Norm Minimization for Tensorial Multi-View Subspace Clustering.

The low-rank tensor could characterize inner structure and explore high-order correlation among multi-view representations, which has been widely used in multi-view clustering. Existing approaches adopt the tensor nuclear norm (TNN) as a convex approximation of non-convex tensor rank function. However, TNN treats the different singular values equally and over-penalizes the main rank components, leading to sub-optimal tensor representation. In this paper, we devise a better surrogate of tensor rank, namely the tensor logarithmic Schatten- p norm ([Formula: see text]N), which fully considers the physical difference between singular values by the non-convex and non-linear penalty function. Further, a tensor logarithmic Schatten- p norm minimization ([Formula: see text]NM)-based multi-view subspace clustering ([Formula: see text]NM-MSC) model is proposed. Specially, the proposed [Formula: see text]NM can not only protect the larger singular values encoded with useful structural information, but also remove the smaller ones encoded with redundant information. Thus, the learned tensor representation with compact low-rank structure will well explore the complementary information and accurately characterize the high-order correlation among multi-views. The alternating direction method of multipliers (ADMM) is used to solve the non-convex multi-block [Formula: see text]NM-MSC model where the challenging [Formula: see text]NM problem is carefully handled. Importantly, the algorithm convergence analysis is mathematically established by showing that the sequence generated by the algorithm is of Cauchy and converges to a Karush-Kuhn-Tucker (KKT) point. Experimental results on nine benchmark databases reveal the superiority of the [Formula: see text]NM-MSC model.

Large-Scale Multi-View Clustering via Fast Essential Subspace Representation Learning

Large-scale Multi-View Clustering (LMVC) is a hot research problem in the fields of signal processing and machine learning, and many anchor-based multi-view subspace clustering algorithms are proposed in recent years. However, most existing methods usually concentrate on the issue of reducing the time cost and ignore the exploration of the complementary information during the clustering process. To this end, we propose a Fast Essential Subspace Representation Learning (FESRL) method for large-scale multi-view subspace clustering. Specifically, FESRL introduces the orthogonal transformation to investigate both the complementary and consensus information across multiple views. The essential subspace representation can be learned in a linear time cost. Experiments conducted on several benchmark datasets illustrate the competitiveness of the proposed method.