Multi-view Clustering Methods Research Articles

CSMVC: A Multiview Method for Multivariate Time-Series Clustering.

Multivariate time-series (MTS) clustering is a fundamental technique in data mining with a wide range of real-world applications. To date, though some approaches have been developed, they suffer from various drawbacks, such as high computational cost or loss of information. Most existing approaches are single-view methods without considering the benefits of mutual-support multiple views. Moreover, due to its data structure, MTS data cannot be handled well by most multiview clustering methods. Toward this end, we propose a consistent and specific non-negative matrix factorization-based multiview clustering (CSMVC) method for MTS clustering. The proposed method constructs a multilayer graph to represent the original MTS data and generates multiple views with a subspace technique. The obtained multiview data are processed through a novel non-negative matrix factorization (NMF) method, which can explore the view-consistent and view-specific information simultaneously. Furthermore, an alternating optimization scheme is proposed to solve the corresponding optimization problem. We conduct extensive experiments on 13 benchmark datasets and the results demonstrate the superiority of our proposed method against other state-of-the-art algorithms under a wide range of evaluation metrics.

Tensor Completion-Based Incomplete Multiview Clustering.

Incomplete multiview clustering is a challenging problem in the domain of unsupervised learning. However, the existing incomplete multiview clustering methods only consider the similarity structure of intraview while neglecting the similarity structure of interview. Thus, they cannot take advantage of both the complementary information and spatial structure embedded in similarity matrices of different views. To this end, we complete the incomplete graph with missing data referring to tensor complete and present a novel and effective model to handel the incomplete multiview clustering task. To be specific, we consider the similarity of the interview graphs via the tensor Schatten p -norm-based completion technique to make use of both the complementary information and spatial structure. Meanwhile, we employ the connectivity constraint for similarity matrices of different views such that the connected components approximately represent clusters. Thus, the learned entire graph not only has the low-rank structure but also well characterizes the relationship between unmissing data. Extensive experiments show the promising performance of the proposed method comparing with several incomplete multiview approaches in the clustering tasks.

Semi-Supervised Clustering via Cannot Link Relationship for Multiview Data

Due to the diversity of data modalities, the research interest of multi-view clustering is gradually increasing, in the field of large-data analytics, particularly in clustering. However, the greater part of current multi-view clustering methods is mainly in view of unsupervised learning, which leads to unpredictable results and algorithmic instability. Besides, they ignore the diversity of graphs, which is not desirable in practical applications, because the characteristic properties of each view are different. To solve these problems, inspired by the outstanding performance of semi-supervised learning in machine learning, we propose a valid semi-supervised multi-view spectral clustering algorithm. We use the pre-set labels as prior knowledge to obtain the overall distribution of the remaining unlabeled data. Tensor minimization Schatten <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$p$ </tex-math></inline-formula> -norm is utilized to mine the mutual information hidden in multiple views. Meanwhile, we also use the cannot-link as another semi-supervised constraint to update the graph. Our proposed algorithm is generally 5%-10% better than the comparison algorithms in view of the experimental results on five datasets, and our algorithm is relatively fast with the computational complexity of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathcal {O}({T({n^{2}}\log (n) + {n^{2}} + {u^{2}}l + ulc + uc\log (c))})$ </tex-math></inline-formula> , where <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$T$ </tex-math></inline-formula> denotes the number of iterations and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$n$ </tex-math></inline-formula> , <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$l$ </tex-math></inline-formula> , <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$u$ </tex-math></inline-formula> represent the number of samples, the number of labeled and unlabeled samples, respectively, which shows that our proposed method has broad application prospects.

Graph-filtering and high-order bipartite graph based multiview graph clustering

Multiview clustering, which partitions data into different groups, has attracted wide attention. With increasing data, bipartite graph-based multiview clustering has become an important topic since it can achieve efficient clustering by establishing relationship between data points and anchor points instead of all samples. Current most methods learn graph structure using one-order bipartite graph from original multi-view feature. However, original data inevitably contain noise feature and its structure is complex. To address this issue, a novel graph filtering and high-order bipartite graph-based multiview graph clustering method is presented, which consider the influence of noise feature and complex graph structure relationship. Concretely, we first employ graph filtering to original data feature space, then adopt a two-step random walk approach to establish the bipartite graph structural relationships of each view. At last, based on constructed high-order bipartite graph, a self-weight bipartite graph-based multiview graph fusion framework is proposed, which reduces annoying weight parameter selection and obtains a joint bipartite graph. Experimental results on several benchmark datasets demonstrate that this method achieves better clustering performance than state-of-the-art multiview clustering methods.

Pure graph-guided multi-view subspace clustering

Multi-view subspace clustering approaches have shown outstanding performance in revealing similarity relationships and complex structures hidden in data. Despite the progress, previous multi-view clustering methods still face two challenges: (1) it is difficult to simultaneously achieve sparsity and connectivity of the affinity graph; (2) existing methods usually separate the graph learning step from the clustering process, which leads to unsatisfactory clustering performance as the final results critically rely on the learned graph. In this paper, we propose to achieve a structured consensus graph for multi-view subspace clustering by leveraging the sparsity and connectivity of each affinity graph. In the proposed method, the pure graph for each view is searched by finding the good neighbors. The multiple pure graphs are further fused into a consensus graph with a block-diagonal structure. That is, the consensus graph is enforced to contain exactly c connected components where c is the number of the clusters. Hence the label to each sample can be directly assigned since each connected component precisely corresponds to an individual cluster. As a result, the proposed model seamlessly accomplishes the subtasks including graph construction, pure graph learning (i.e., good neighbors searching), and cluster label allocation in a mutual reinforcement manner. Extensive experimental results demonstrate the superiority and reliability of our proposed method.

Robust dimensionality reduction method based on relaxed energy and structure preserving embedding for multiview clustering

Existing multiview clustering methods often learn inferior quality affinity graphs from original data, which suffer from excessive dimensionality and contain noise and outliers. Therefore, dimensionality reduction must be performed on the original data, which leads to the following problems: 1) the learned projection matrix does not preserve the main discriminative information of the original data; 2) the local and global structures are not preserved simultaneously; and 3) the negative effects of noise, outliers, and corruption cannot be overcome. To alleviate the aforementioned problems, a robust dimensionality reduction method based on a relaxed energy-preserving and structure-preserving embedding for multiview clustering is proposed in this study. In particular, the proposed architecture is a double-relaxed energy-preserving embedding model, which uses two projection matrices to extract the main discriminant features from the original data using two energy-preserving embedding routes, yielding clean data and a representation matrix in low-dimensional space. Subsequently, an affinity matrix that captures both local and global data structures is learned by performing locality-preserving projection and global low-rank-preserving embedding on the clean data and the representation matrix, respectively. Experimental results on benchmark datasets validate the effectiveness of the proposed algorithm.

Robust multi-view subspace enhanced representation based on collaborative constraints and HSIC induction

The design of effective multi-view subspace clustering (MSC) algorithms has recently garnered significant research attention. Herein, to effectively improve the recognition performance and anti-noise interference ability of an MSC model, we propose a novel MSC algorithm, termed as robust multi-view subspace enhancement representation, based on collaborative constraints and a Hilbert–Schmidt independence criterion (HSIC) induction method. To mine the complementary information between different views, we apply the HSIC as a diversity regularization term. Specifically, to enhance the diagonal block structure of a subspace representation, a new sparse constraint is introduced on the product of itself and the transpose of the subspace representation matrix in a multi-view subspace learning model. Furthermore, hypergraph regularization and a low-rank idea are considered to capture the local geometric structure and clean data. In addition, to optimize our model, we adopt an augmented Lagrangian multiplier method and discuss the convergence of the model. Extensive experiments on six challenging datasets reveal that the proposed method achieves a highly competent objective performance with and without noisy views, as compared with several state-of-the-art multi-view clustering methods.

Scalable one-stage multi-view subspace clustering with dictionary learning

The clustering of large numbers of heterogeneous features is a hot topic in multi-view communities. Most existing multi-view clustering (MvC) methods employ matrix factorization or anchor strategies to handle large-scale datasets. The former operates on the original data and is, therefore, sensitive to noise and feature redundancy, which is reflected in the final clustering performance. The latter requires post-processing steps to generate the clustering results, which may be suboptimal owing to the isolation steps. To address the above problems, we propose one-stage multi-view subspace clustering with dictionary learning (OSMvSC). Specifically, we integrate dictionary learning, representation coefficient matrix learning, and matrix factorization as a unified learning framework, which directly learns the dictionary and representation coefficient matrix to encode the original multi-view data, and obtains the clustering results with linear time complexity without any postprocessing step. By manipulating the class centroid with the nuclear norm, a more compact and discriminative class centroid representation can be obtained to further improve clustering performance. An effective optimization algorithm with guaranteed convergence is designed to solve the proposed method. Substantial experiments on various real-world multi-view datasets demonstrate the effectiveness and superiority of the proposed method. The source code is available at https://github.com/justcallmewilliam/OSMvSC.

An improved multi-view spectral clustering based on tissue-like P systems

Multi-view spectral clustering is one of the multi-view clustering methods widely studied by numerous scholars. The first step of multi-view spectral clustering is to construct the similarity matrix of each view. Consequently, the clustering performance will be greatly affected by the quality of the similarity matrix of each view. To solve this problem well, an improved multi-view spectral clustering based on tissue-like P systems is proposed in this paper. The optimal per-view similarity matrix is generated in an iterative manner. In addition, spectral clustering is combined with the symmetric nonnegative matrix factorization method to directly output the clustering results to avoid the secondary operation, such as k-means or spectral rotation. Furthermore, improved multi-view spectral clustering is integrated with the tissue-like P system to enhance the computational efficiency of the multi-view clustering algorithm. Extensive experiments verify the effectiveness of this algorithm over other state-of-the-art algorithms.

Anchor-based incomplete multi-view spectral clustering

In the past decade, multi-view clustering has become a research hot spot of machine learning. In traditional multi-view clustering methods, all views of the data points are assumed to be complete. However, in the practical applications, some views of data points may be missing and incomplete multi-view clustering methods are developed to handle these incomplete multi-view data. The existing incomplete multi-view clustering methods still have some defects such as insufficient use of missing information or neglecting the underlying relations among different views. To address these limitations, we propose an Anchor-based Incomplete Multi-view Spectral Clustering (AIMSC) approach. Specifically, AIMSC utilizes anchor points to connect all instances of each view and recover the missing information. Then, the similarities between all data points are derived from the similarities between data points and anchor points. Finally, anchor-based spectral clustering is executed to generate the clustering results. Experimental results on multiple benchmark datasets demonstrate the superiority of AIMSC.

Low-rank constraint bipartite graph learning

Bipartite Graph-based multi-view clustering has become an active topic recently due to its efficiency in tackling large scale multi-view data. However, most existing bipartite graph-based multi-view clustering methods have the following disadvantages: 1) the clustering performance heavily depends on the predefined bipartite graph; 2) they fail to explore the complementary information embedded in multiple bipartite graphs and spatial low-rank structure hidden in each bipartite graph. To address these issues, we propose an efficient multi-view clustering method, Learning Constraint Bipartite Graphs for Multi-view Clustering (LCBG). LCBG adaptively learns each graph S(v)∈RN×M such that it well characterizes the relationship between M (M≪N) anchors and N samples in the v-th view, and takes both the intra-view and inter-view spatial low-rank structures of the learned bipartite graphs into account by minimizing tensor Schatten p-norm and nuclear norm, respectively. Finally, an efficient algorithm, which scales linearly with the data size, is proposed to solve LCBG. Extensive experimental results on five benchmark datasets indicate our proposed LCBG is superior to the state-of-the-art methods.

Pseudo-Label Guided Collective Matrix Factorization for Multiview Clustering.

Multiview clustering has aroused increasing attention in recent years since real-world data are always comprised of multiple features or views. Despite the existing clustering methods having achieved promising performance, there still remain some challenges to be solved: 1) most existing methods are unscalable to large-scale datasets due to the high computational burden of eigendecomposition or graph construction and 2) most methods learn latent representations and cluster structures separately. Such a two-step learning scheme neglects the correlation between the two learning stages and may obtain a suboptimal clustering result. To address these challenges, a pseudo-label guided collective matrix factorization (PLCMF) method that jointly learns latent representations and cluster structures is proposed in this article. The proposed PLCMF first performs clustering on each view separately to obtain pseudo-labels that reflect the intraview similarities of each view. Then, it adds a pseudo-label constraint on collective matrix factorization to learn unified latent representations, which preserve the intraview and interview similarities simultaneously. Finally, it intuitively incorporates latent representation learning and cluster structure learning into a joint framework to directly obtain clustering results. Besides, the weight of each view is learned adaptively according to data distribution in the joint framework. In particular, the joint learning problem can be solved with an efficient iterative updating method with linear complexity. Extensive experiments on six benchmark datasets indicate the superiority of the proposed method over state-of-the-art multiview clustering methods in both clustering accuracy and computational efficiency.

Adaptive Transition Probability Matrix Learning for Multiview Spectral Clustering.

Multiview clustering as an important unsupervised method has been gathering a great deal of attention. However, most multiview clustering methods exploit the self-representation property to capture the relationship among data, resulting in high computation cost in calculating the self-representation coefficients. In addition, they usually employ different regularizers to learn the representation tensor or matrix from which a transition probability matrix is constructed in a separate step, such as the one proposed by Wu et al.. Thus, an optimal transition probability matrix cannot be guaranteed. To solve these issues, we propose a unified model for multiview spectral clustering by directly learning an adaptive transition probability matrix (MCA2M), rather than an individual representation matrix of each view. Different from the one proposed by Wu et al., MCA2M utilizes the one-step strategy to directly learn the transition probability matrix under the robust principal component analysis framework. Unlike existing methods using the absolute symmetrization operation to guarantee the nonnegativity and symmetry of the affinity matrix, the transition probability matrix learned from MCA2M is nonnegative and symmetric without any postprocessing. An alternating optimization algorithm is designed based on the efficient alternating direction method of multipliers. Extensive experiments on several real-world databases demonstrate that the proposed method outperforms the state-of-the-art methods.

Incomplete multi-view clustering via virtual-label guided matrix factorization

Multi-view clustering methods have achieved appealing performance by exploring the data from multiple sources. It often assumes that the multiple-view data are complete. However, such rigorous assumption does not be satisfied in many real-world applications, in which only some views of data are available. The existence of the incomplete-view data would disable the current multi-view clustering algorithms. To address this challenge, in this article, we present a novel virtual-label guided matrix factorization (VLMF) to facilitate the incomplete multi-view clustering. Specifically, in VLMF, we first employ the graph regularization of each view to mine the geometrical structure of data. Then, we utilize the virtual label to guide matrix factorization. Furthermore, the clustering process and the consensus latent representation learning of data are integrated into a joint framework. Experimental results on several incomplete multi-view datasets illustrate the validity of the proposed method.

Diversity embedding deep matrix factorization for multi-view clustering

Multi-view clustering has attracted increasing attention by reason of its ability to leverage the complementarity of multi-view data. Existing multi-view clustering methods have explored nonnegative matrix factorization to decompose a matrix into multiple matrices for feature representations from multi-view data, which are not discriminative enough to deal with the natural data containing complex information. Moreover, most of multi-view clustering methods prioritize the consensus information among multi-view data, leaving a large amount of information redundant and the clustering performance deteriorated. To address these issues, this paper proposes a multi-view clustering framework that adopts a diversity loss for deep matrix factorization and reduces feature redundancy while obtaining more discriminative features. We then bridge the relation between deep auto-encoder and deep matrix factorization to optimize the objective function. This method avoids the challenges in the optimization process. Extensive experiments demonstrate that the proposed method is superior to state-of-the-art methods.

Representation Learning in Multi-view Clustering: A Literature Review

Multi-view clustering (MVC) has attracted more and more attention in the recent few years by making full use of complementary and consensus information between multiple views to cluster objects into different partitions. Although there have been two existing works for MVC survey, neither of them jointly takes the recent popular deep learning-based methods into consideration. Therefore, in this paper, we conduct a comprehensive survey of MVC from the perspective of representation learning. It covers a quantity of multi-view clustering methods including the deep learning-based models, providing a novel taxonomy of the MVC algorithms. Furthermore, the representation learning-based MVC methods can be mainly divided into two categories, i.e., shallow representation learning-based MVC and deep representation learning-based MVC, where the deep learning-based models are capable of handling more complex data structure as well as showing better expression. In the shallow category, according to the means of representation learning, we further split it into two groups, i.e., multi-view graph clustering and multi-view subspace clustering. To be more comprehensive, basic research materials of MVC are provided for readers, containing introductions of the commonly used multi-view datasets with the download link and the open source code library. In the end, some open problems are pointed out for further investigation and development.

Multi-view clustering with adaptive procrustes on Grassmann manifold

Multi-view clustering plays an important role in a wide spectrum of applications. In this article, we propose a multi-view clustering approach with adaptive Procrustes on Grassmann manifold (MC-APGM) to overcome the three demerits in existing graph-based multi-view clustering methods, namely, insufficient mining of subspace information of views, a requirement for post-processing, and high computational complexity. Specifically, in the proposed model, the indicator matrix is directly learned from multiple orthogonal spectral embeddings, avoiding the random clustering results caused by post-processing; The orthogonal form of the indicator matrix approximates multiple orthogonal spectral embeddings on the Grassmann manifold, fully uncovering subspace information of views and thus improving clustering performance; Both implicitly and explicitly weighted learning mechanisms are established to capture inconsistencies among different views. Moreover, an efficient algorithm with rigorous convergence guarantee is derived to optimize the proposed model. Finally, experimental results on both toy and real-world datasets demonstrate the effectiveness and efficiency of this proposed method.

Centroids-guided deep multi-view K-means clustering

With the progress of deep learning used in unsupervised learning, deep approach based multi-view clustering methods have been increasingly proposed in recent years. However, in most of these methods, deep representation learning is not organically integrated into the multi-view clustering process. They either conduct deep representation learning and clustering in a separate manner, or use the pseudo cluster labels to supervise deep representation learning. In this paper, we propose a centroids-guided deep multi-view k-means clustering method, which organically incorporates deep representation learning into the multi-view k-means objective by using the cluster centroids in multi-view k-means to guide the deep learning of each view. In turn, more k-means-friendly representations are produced to further optimize the multi-view k-means objective. The cluster centroids of each view obtained under a common clustering partition not only represent the semantic information of the clusters but also imply consistency among different views. By reducing the loss between each representation and its assigned cluster centroid with respect to the network parameters of each view, the representations of different views will be more k-means-friendly toward a common partition. Experiments on several datasets demonstrate the effectiveness of our method.

Incomplete multi-view clustering via independent self-representation learning

Incomplete multi-view clustering is a learning paradigm that combines heterogeneous and incomplete characteristics of multi-view data to obtain data structure and improve clustering performance. In practical application, each view will also be affected by missing values and outliers, in addition to missing some complete samples, which makes most traditional incomplete multi-view clustering methods ineffective. An incomplete multi-view clustering method based on independent self-presentation learning is proposed to solve the above issue. The proposed method fills in the missing features and learns the unique self-presentation matrix of each view using self-representation reconstruction. Then, to better mine the structure, low-rank constraints are added to the self-presentation matrix, and the diversity among different views is measured by introducing the Hilbert-Schmidt independence criterion. Experimental results on multiple data sets show that the proposed method can achieve better clustering than other advanced methods in most cases.

Agglomerative Neural Networks for Multiview Clustering.

Conventional multiview clustering methods seek a view consensus through minimizing the pairwise discrepancy between the consensus and subviews. However, pairwise comparison cannot portray the interview relationship precisely if some of the subviews can be further agglomerated. To address the above challenge, we propose the agglomerative analysis to approximate the optimal consensus view, thereby describing the subview relationship within a view structure. We present an agglomerative neural network (ANN) based on constrained Laplacian rank to cluster multiview data directly without a dedicated postprocessing step (e.g., using K -means). We further extend ANN with a learnable data space to handle data of complex scenarios. Our evaluations against several state-of-the-art multiview clustering approaches on four popular data sets show the promising view-consensus analysis ability of ANN. We further demonstrate ANN's capability in analyzing complex view structures, extensibility through our case study and robustness and effectiveness of data-driven modifications.