Multi-view Clustering Framework Research Articles

Multi-view clustering via pseudo-label guide learning and latent graph structure recovery

Multi-view clustering (MvC) accomplishes sample classification tasks by exploring information from different views. Currently, researchers have paid greater attention to graph-based MvC methods. However, most existing methods only consider the original graph structure and pay relatively little attention to the graph structure in the latent space. In addition, most methods need to pay more attention to the consistency of information on different labels. Otherwise, this may lead to the loss of label information. This paper presents a new multi-view clustering framework to address the above issues. The proposed method considers both the information in the latent space and the original data space, which firstly obtains the pseudo-label by latent representation learning and then lets the pseudo-label guide the learning of the complementary information between the raw data views. To ensure the integrity of the data structure, a latent graph structure recovery strategy is designed in the latent space. Finally, an enhanced label fusion strategy is designed to fusion the different types of labels, yielding an information-rich label matrix for clustering. Experimental results demonstrate the proposed method’s effectiveness compared to other advanced approaches.

A multiview clustering framework for detecting deceptive reviews

Online reviews, which play a key role in the ecosystem of nowadays business, have been the primary source of consumer opinions. Due to their importance, professional review writing services are employed for paid reviews and even being exploited to conduct opinion spam. Posting deceptive reviews could mislead customers, yield significant benefits or losses to service vendors, and erode confidence in the entire online purchasing ecosystem. In this paper, we ferret out deceptive reviews originated from professional review writing services. We do so even when reviewers leverage a number of pseudonymous identities to avoid the detection. To unveil the pseudonymous identities associated with deceptive reviewers, we leverage the multiview clustering method. This enables us to characterize the writing style of reviewers (deceptive vs normal) and cluster the reviewers based on their writing style. Furthermore, we explore different neural network models to model the writing style of deceptive reviews. We select the best performing neural network to generate the representation of reviews. We validate the effectiveness of the multiview clustering framework using real-world Amazon review data under different experimental scenarios. Our results show that our approach outperforms previous research. We further demonstrate its superiority through a large-scale case study based on publicly available Amazon datasets.

Clustering Enhanced Multiplex Graph Contrastive Representation Learning.

Multiplex graph representation learning has attracted considerable attention due to its powerful capacity to depict multiple relation types between nodes. Previous methods generally learn representations of each relation-based subgraph and then aggregate them into final representations. Despite the enormous success, they commonly encounter two challenges: 1) the latent community structure is overlooked and 2) consistent and complementary information across relation types remains largely unexplored. To address these issues, we propose a clustering-enhanced multiplex graph contrastive representation learning model (CEMR). In CEMR, by formulating each relation type as a view, we propose a multiview graph clustering framework to discover the potential community structure, which promotes representations to incorporate global semantic correlations. Moreover, under the proposed multiview clustering framework, we develop cross-view contrastive learning and cross-view cosupervision modules to explore consistent and complementary information in different views, respectively. Specifically, the cross-view contrastive learning module equipped with a novel negative pairs selecting mechanism enables the view-specific representations to extract common knowledge across views. The cross-view cosupervision module exploits the high-confidence complementary information in one view to guide low-confidence clustering in other views by contrastive learning. Comprehensive experiments on four datasets confirm the superiority of our CEMR when compared to the state-of-the-art rivals.

Parameter-Free and Scalable Incomplete Multiview Clustering With Prototype Graph.

Multiview clustering (MVC) seamlessly combines homogeneous information and allocates data samples into different communities, which has shown significant effectiveness for unsupervised tasks in recent years. However, some views of samples may be incomplete due to unfinished data collection or storage failure in reality, which refers to the so-called incomplete multiview clustering (IMVC). Despite many IMVC pioneer frameworks have been introduced, the majority of their approaches are limited by the cubic time complexity and quadratic space complexity which heavily prevent them from being employed in large-scale IMVC tasks. Moreover, the massively introduced hyper-parameters in existing methods are not practical in real applications. Inspired by recent unsupervised multiview prototype progress, we propose a novel parameter-free and scalable incomplete multiview clustering framework with the prototype graph termed PSIMVC-PG to solve the aforementioned issues. Different from existing full pair-wise graph studying, we construct an incomplete prototype graph to flexibly capture the relations between existing instances and discriminate prototypes. Moreover, PSIMVC-PG can directly obtain the prototype graph without pre-process of searching hyper-parameters. We conduct massive experiments on various incomplete multiview tasks, and the performances show clear advantages over existing methods. The code of PSIMVC-PG can be publicly downloaded at https://github.com/wangsiwei2010/PSIMVC-PG.

Deep Multiview Clustering by Pseudo-Label Guided Contrastive Learning and Dual Correlation Learning.

Deep multiview clustering (MVC) is to learn and utilize the rich relations across different views to enhance the clustering performance under a human-designed deep network. However, most existing deep MVCs meet two challenges. First, most current deep contrastive MVCs usually select the same instance across views as positive pairs and the remaining instances as negative pairs, which always leads to inaccurate contrastive learning (CL). Second, most deep MVCs only consider learning feature or cluster correlations across views, failing to explore the dual correlations. To tackle the above challenges, in this article, we propose a novel deep MVC framework by pseudo-label guided CL and dual correlation learning. Specifically, a novel pseudo-label guided CL mechanism is designed by using the pseudo-labels in each iteration to help removing false negative sample pairs, so that the CL for the feature distribution alignment can be more accurate, thus benefiting the discriminative feature learning. Different from most deep MVCs learning only one kind of correlation, we investigate both the feature and cluster correlations among views to discover the rich and comprehensive relations. Experiments on various datasets demonstrate the superiority of our method over many state-of-the-art compared deep MVCs. The source implementation code will be provided at https://github.com/ShizheHu/Deep-MVC-PGCL-DCL.

NetMUG: a novel network-guided multi-view clustering workflow for dissecting genetic and facial heterogeneity.

Introduction: Multi-view data offer advantages over single-view data for characterizing individuals, which is crucial in precision medicine toward personalized prevention, diagnosis, or treatment follow-up. Methods: Here, we develop a network-guided multi-view clustering framework named netMUG to identify actionable subgroups of individuals. This pipeline first adopts sparse multiple canonical correlation analysis to select multi-view features possibly informed by extraneous data, which are then used to construct individual-specific networks (ISNs). Finally, the individual subtypes are automatically derived by hierarchical clustering on these network representations. Results: We applied netMUG to a dataset containing genomic data and facial images to obtain BMI-informed multi-view strata and showed how it could be used for a refined obesity characterization. Benchmark analysis of netMUG on synthetic data with known strata of individuals indicated its superior performance compared with both baseline and benchmark methods for multi-view clustering. The clustering derived from netMUG achieved an adjusted Rand index of 1 with respect to the synthesized true labels. In addition, the real-data analysis revealed subgroups strongly linked to BMI and genetic and facial determinants of these subgroups. Discussion: netMUG provides a powerful strategy, exploiting individual-specific networks to identify meaningful and actionable strata. Moreover, the implementation is easy to generalize to accommodate heterogeneous data sources or highlight data structures.

Contrastive Multi-View Kernel Learning

Kernel method is a proven technique in multi-view learning. It implicitly defines a Hilbert space where samples can be linearly separated. Most kernel-based multi-view learning algorithms compute a kernel function aggregating and compressing the views into a single kernel. However, existing approaches compute the kernels independently for each view. This ignores complementary information across views and thus may result in a bad kernel choice. In contrast, we propose the Contrastive Multi-view Kernel - a novel kernel function based on the emerging contrastive learning framework. The Contrastive Multi-view Kernel implicitly embeds the views into a joint semantic space where all of them resemble each other while promoting to learn diverse views. We validate the method's effectiveness in a large empirical study. It is worth noting that the proposed kernel functions share the types and parameters with traditional ones, making them fully compatible with existing kernel theory and application. On this basis, we also propose a contrastive multi-view clustering framework and instantiate it with multiple kernel k-means, achieving a promising performance. To the best of our knowledge, this is the first attempt to explore kernel generation in multi-view setting and the first approach to use contrastive learning for a multi-view kernel learning.

Biotypes of major depressive disorder identified by a multiview clustering framework

BackgroundThe advances in resting-state functional magnetic resonance imaging techniques motivate parsing heterogeneity in major depressive disorder (MDD) through neurophysiological subtypes (i.e., biotypes). Based on graph theories, researchers have observed the functional organization of the human brain as a complex system with modular structures and have found wide-spread but variable MDD-related abnormality regarding the modules. The evidence implies the possibility of identifying biotypes using high-dimensional functional connectivity (FC) data in ways that suit the potentially multifaceted biotypes taxonomy. MethodsWe proposed a multiview biotype discovery framework that involves theory-driven feature subspace partition (i.e., “view”) and independent subspace clustering. Six views were defined using intra- and intermodule FC regarding three MDD focal modules (i.e., the sensory-motor system, default mode network, and subcortical network). For robust biotypes, the framework was applied to a large multisite sample (805 MDD participants and 738 healthy controls). ResultsTwo biotypes were stably obtained in each view, respectively characterized by significantly increased and decreased FC compared to healthy controls. These view-specific biotypes promoted the diagnosis of MDD and showed different symptom profiles. By integrating the view-specific biotypes into biotype profiles, a broad spectrum in the neural heterogeneity of MDD and its separation from symptom-based subtypes was further revealed. LimitationsThe power of clinical effects is limited and the cross-sectional nature cannot predict the treatment effects of the biotypes. ConclusionsOur findings not only contribute to the understanding of heterogeneity in MDD, but also provide a novel subtyping framework that could transcend current diagnostic boundaries and data modality.

Neighbor-aware deep multi-view clustering via graph convolutional network

Multi-view clustering (MVC) enhances the clustering performance of data by combining correlation information from different views. However, most existing MVC approaches process each sample independently and ignore the correlation amongst samples, resulting in reduced clustering performance. Although graph convolution network (GCN) can naturally capture correlation amongst samples by integrating the neighbors and structural information into representation learning, it is used in the semi-supervised learning scenario. In this paper, we propose a neighbor-aware deep MVC framework based on GCN (NMvC-GCN) for clustering multi-view samples and training GCN in a fully unsupervised manner. In addition, we design a consensus regularization to learn the common representations and introduce a clustering embedding layer to jointly optimize the clustering task and representation learning, so that the correlation amongst samples and that between the clustering task and representation learning can be fully explored. Extensive experiments on 10 datasets illustrate that NMvC-GCN significantly outperforms the state-of-the-art MVC methods. Our code will be released at https://github.com/dugzzuli/NMvC-GCN.

Incomplete Multiview Clustering via Cross-View Relation Transfer

In this paper, we consider the problem of multi-view clustering on incomplete views. Compared with complete multi-view clustering, the view-missing problem increases the difficulty of learning common representations from different views. To address the challenge, we propose a novel incomplete multi-view clustering framework, which incorporates cross-view relation transfer and multi-view fusion learning. Specifically, based on the consistency existing in multi-view data, we devise a cross-view relation transfer-based completion module, which transfers known similar inter-instance relationships to the missing view and recovers the missing data via graph networks based on the transferred relationship graph. Then the view-specific encoders are designed to extract the recovered multi-view data, and an attention-based fusion layer is introduced to obtain the common representation. Moreover, to reduce the impact of the error caused by the inconsistency between views and obtain a better clustering structure, a joint clustering layer is introduced to optimize recovery and clustering simultaneously. Extensive experiments conducted on several real datasets demonstrate the effectiveness of the proposed method.

Auto-weighted collective matrix factorization with graph dual regularization for multi-view clustering

Multi-view clustering (MVC) is an attractive clustering paradigm that can incorporate comprehensive information from multiple views. Among the MVC schemes, collective matrix factorization (CMF) has shown its great power in extracting shared information of multi-view data. Based on CMF, we propose a novel unified MVC framework, named Auto-weighted Collective Matrix Factorization with Graph Dual Regularization (ACMF-GDR). Specifically, we assign adaptive weights for each view and incorporate the smoothing cluster structure learning term to construct a unified auto-weighted CMF for MVC. Our ACMF-GDR model can obtain the cluster labels and common representations of the samples in a one-step manner. Furthermore, to make the common representations discriminative, graph dual regularization terms with orthogonality constraints are adopted on multiple views to preserve the geometrical structure of the decomposed factors simultaneously. Experimental results show the superior clustering performance of the proposed method.

Multi-view clustering with orthogonal mapping and binary graph

The multi-view clustering problem has attracted considerable attention over recent years for the remarkable clustering performance due to exploiting complementary information from multiple views. Most existing related research work processes data in the decimal real value space that is not the most compatible space for computers. Binary code learning, also known as hashing technology, is well-known for fast Hamming distance computation, less storage requirement and accurate calculation results. The Hamming space is most enjoyed by computers because of binary/hash codes. Several studies combine multi-view clustering with binary code learning for improving clustering performance. However, there is much redundant information contained in the learned binary codes, which negatively affects the clustering performance, but these studies ignore eliminating redundant information for learning compact codes. In addition, they don’t give a unified (one-step) clustering framework with binary graph structure, which doesn’t lead to the optimal clustering result due to the information loss during the two-step process. In this paper, to cope with the two issues, we propose an orthogonal mapping binary graph method (OMBG) for the multi-view clustering problem, which makes the mapping matrix of every view orthogonalize for eliminating redundant information and embeds a binary graph structure into the unified binary multi-view clustering framework for extracting local geometric structure information of binary codes and achieving the optimal clustering result. Furthermore, we design an effective optimization algorithm based on alternating direction minimization to solve the model of OMBG. Extensive experiments performed on four frequently-used benchmark multi-view datasets illustrate the superiority of OMBG which is compared with some state-of-the-art clustering baselines.

Diversity embedding deep matrix factorization for multi-view clustering

Multi-view clustering has attracted increasing attention by reason of its ability to leverage the complementarity of multi-view data. Existing multi-view clustering methods have explored nonnegative matrix factorization to decompose a matrix into multiple matrices for feature representations from multi-view data, which are not discriminative enough to deal with the natural data containing complex information. Moreover, most of multi-view clustering methods prioritize the consensus information among multi-view data, leaving a large amount of information redundant and the clustering performance deteriorated. To address these issues, this paper proposes a multi-view clustering framework that adopts a diversity loss for deep matrix factorization and reduces feature redundancy while obtaining more discriminative features. We then bridge the relation between deep auto-encoder and deep matrix factorization to optimize the objective function. This method avoids the challenges in the optimization process. Extensive experiments demonstrate that the proposed method is superior to state-of-the-art methods.

Multi-Mode Tensor Space Clustering Based on Low-Tensor-Rank Representation

Traditional subspace clustering aims to cluster data lying in a union of linear subspaces. The vectorization of high-dimensional data to 1-D vectors to perform clustering ignores much of the structure intrinsic to such data. To preserve said structure, in this work we exploit clustering in a high-order tensor space rather than a vector space. We develop a novel low-tensor-rank representation (LTRR) for unfolded matrices of tensor data lying in a low-rank tensor space. The representation coefficient matrix of an unfolding matrix is tensorized to a 3-order tensor, and the low-tensor-rank constraint is imposed on the transformed coefficient tensor to exploit the self-expressiveness property. Then, inspired by the multi-view clustering framework, we develop a multi-mode tensor space clustering algorithm (MMTSC) that can deal with tensor space clustering with or without missing entries. The tensor is unfolded along each mode, and the coefficient matrices are obtained for each unfolded matrix. The low tensor rank constraint is imposed on a tensor combined from transformed coefficient tensors of each mode, such that the proposed method can simultaneously capture the low rank property for the data within each tensor space and maintain cluster consistency across different modes. Experimental results demonstrate that the proposed MMTSC algorithm can outperform existing clustering algorithms in many cases.

A multi-view clustering framework via integrating K-means and graph-cut

Graph-cut and K-means are two classical clustering methods, which are used by most of existing clustering methods. First, a multi-view clustering framework via K-means and graph is proposed and a efficient clustering method is developed based on this framework. Our method calculates the discrete cluster assignment matrix directly and includes the auto-weighted strategy to consider the different contributions of views. Additionally, by generating the distance matrices for views with the corresponding graphs, our method fits both Gaussian and non-Gaussian distributed datasets. Moreover, the proposed algorithm is able to obtain robust results because it does not need initializing and computing cluster centroids. Extensive experiments on a synthetic dataset and four real-world datasets indicate the effectiveness of our method.

Multi-view clustering by joint spectral embedding and spectral rotation

Discrete Spectral clustering is an effective tool for directly getting discrete labels. However, existing spectral clustering carry out spectral embedding and spectral rotation separately, which may limit their clustering performance. Seamlessly connecting above two processes is challenging. In this paper, we propose a new multi-view clustering framework, namely Multi-view Clustering by Joint Spectral Embedding and Spectral Rotation. In the framework, the differences of Laplacian matrices from different views are learned adaptively. Moreover, the real-valued cluster indicator matrix is approximated by continuous orthogonalization of the discrete clustering index matrix. By doing so, our method has better convergence, which is also strictly mathematically proven. Extensive experiments indicate that our method is superior to several state-of-the-art methods.

Knowledge Graph Embedding Based on Multi-View Clustering Framework

Knowledge representation is one of the critical problems in knowledge engineering and artificial intelligence, while knowledge embedding as a knowledge representation methodology indicates entities and relations in knowledge graph as low-dimensional, continuous vectors. In this way, knowledge graph is compatible with numerical machine learning models. Major knowledge embedding methods employ geometric translation to design score function, which is weak-semantic for natural language processing. To overcome this disadvantage, in this paper, we propose our model based on multi-view clustering framework, which could generate semantic representations of knowledge elements (i.e., entities/relations ). With our semantic model, we also present an empowered solution to entity retrieval with entity description. Extensive experiments show that our model achieves substantial improvements against baselines on the task of knowledge graph completion, triple classification, entity classification, and entity retrieval.

Incomplete multi-view gene clustering with data regeneration using Shape Boltzmann Machine

Deciphering patterns in the structural and functional anatomy of genes can prove to be very helpful in understanding genetic biology and genomics. Also, the availability of the multiple omics data, along with the advent of machine learning techniques, aids medical professionals in gaining insights about various biological regulations. Gene clustering is one of the many such computation techniques that can help in understanding gene behavior. However, more comprehensive and reliable insights can be gained if different modalities/views of biomedical data are considered. However, in most multi-view cases, each view contains some missing data, leading to incomplete multi-view clustering. In this study, we have presented a deep Boltzmann machine-based incomplete multi-view clustering framework for gene clustering. Here, we seek to regenerate the data of the three NCBI datasets in the incomplete modalities using Shape Boltzmann Machines. The overall performance of the proposed multi-view clustering technique has been evaluated using the Silhouette index and Davies–Bouldin index, and the comparative analysis shows an improvement over state-of-the-art methods. Finally, to prove that the improvement attained by the proposed incomplete multi-view clustering is statistically significant, we perform Welch's t-test. Availability of data and materialshttps://github.com/piyushmishra12/IMC.

A Unified Multi-view Clustering Algorithm Using Multi-objective Optimization Coupled with Generative Model

There is a large body of works on multi-view clustering that exploit multiple representations (or views) of the same input data for better convergence. These multiple views can come from multiple modalities (image, audio, text) or different feature subsets. Obtaining one consensus partitioning after considering different views is usually a non-trivial task. Recently, multi-objective based multi-view clustering methods have suppressed the performance of single objective based multi-view clustering techniques. One key problem is that it is difficult to select a single solution from a set of alternative partitionings generated by multi-objective techniques on the final Pareto optimal front. In this article, we propose a novel multi-objective based multi-view clustering framework that overcomes the problem of selecting a single solution in multi-objective based techniques. In particular, our proposed framework has three major components as follows: (i) multi-view based multi-objective algorithm, Multiview-AMOSA, for initial clustering of data points; (ii) a generative model for generating a combined solution having probabilistic labels; and (iii) K -means algorithm for obtaining the final labels. As the first component, we have adopted a recently developed multi-view based multi-objective clustering algorithm to generate different possible consensus partitionings of a given dataset taking into account different views. A generative model is coupled with the first component to generate a single consensus partitioning after considering multiple solutions. It exploits the latent subsets of the non-dominated solutions obtained from the multi-objective clustering algorithm and combines them to produce a single probabilistic labeled solution. Finally, a simple clustering algorithm, namely K -means, is applied on the generated probabilistic labels to obtain the final cluster labels. Experimental validation of our proposed framework is carried out over several benchmark datasets belonging to three different domains; UCI datasets, multi-view datasets, search result clustering datasets, and patient stratification datasets. Experimental results show that our proposed framework achieves an improvement of around 2%--4% over different evaluation metrics in all the four domains in comparison to state-of-the art methods.

A novel semi-supervised multi-view clustering framework for screening Parkinson's disease.

In recent years, there are many research cases for the diagnosis of Parkinson's disease (PD) with the brain magnetic resonance imaging (MRI) by utilizing the traditional unsupervised machine learning methods and the supervised deep learning models. However, unsupervised learning methods are not good at extracting accurate features among MRIs and it is difficult to collect enough data in the field of PD to satisfy the need of training deep learning models. Moreover, most of the existing studies are based on single-view MRI data, of which data characteristics are not sufficient enough. In this paper, therefore, in order to tackle the drawbacks mentioned above, we propose a novel semi-supervised learning framework called Semi-supervised Multi-view learning Clustering architecture technology (SMC). The model firstly introduces the sliding window method to grasp different features, and then uses the dimensionality reduction algorithms of Linear Discriminant Analysis (LDA) to process the data with different features. Finally, the traditional single-view clustering and multi-view clustering methods are employed on multiple feature views to obtain the results. Experiments show that our proposed method is superior to the state-of-art unsupervised learning models on the clustering effect. As a result, it may be noted that, our work could contribute to improving the effectiveness of identifying PD by previous labeled and subsequent unlabeled medical MRI data in the realistic medical environment.