Multi-view Clustering Algorithms Research Articles

COPS: A novel platform for multi-omic disease subtype discovery via robust multi-objective evaluation of clustering algorithms.

Recent research on multi-view clustering algorithms for complex disease subtyping often overlooks aspects like clustering stability and critical assessment of prognostic relevance. Furthermore, current frameworks do not allow for a comparison between data-driven and pathway-driven clustering, highlighting a significant gap in the methodology. We present the COPS R-package, tailored for robust evaluation of single and multi-omics clustering results. COPS features advanced methods, including similarity networks, kernel-based approaches, dimensionality reduction, and pathway knowledge integration. Some of these methods are not accessible through R, and some correspond to new approaches proposed with COPS. Our framework was rigorously applied to multi-omics data across seven cancer types, including breast, prostate, and lung, utilizing mRNA, CNV, miRNA, and DNA methylation data. Unlike previous studies, our approach contrasts data- and knowledge-driven multi-view clustering methods and incorporates cross-fold validation for robustness. Clustering outcomes were assessed using the ARI score, survival analysis via Cox regression models including relevant covariates, and the stability of the results. While survival analysis and gold-standard agreement are standard metrics, they vary considerably across methods and datasets. Therefore, it is essential to assess multi-view clustering methods using multiple criteria, from cluster stability to prognostic relevance, and to provide ways of comparing these metrics simultaneously to select the optimal approach for disease subtype discovery in novel datasets. Emphasizing multi-objective evaluation, we applied the Pareto efficiency concept to gauge the equilibrium of evaluation metrics in each cancer case-study. Affinity Network Fusion, Integrative Non-negative Matrix Factorization, and Multiple Kernel K-Means with linear or Pathway Induced Kernels were the most stable and effective in discerning groups with significantly different survival outcomes in several case studies.

Efficient Multi-View Clustering via Unified and Discrete Bipartite Graph Learning.

Although previous graph-based multi-view clustering (MVC) algorithms have gained significant progress, most of them are still faced with three limitations. First, they often suffer from high computational complexity, which restricts their applications in large-scale scenarios. Second, they usually perform graph learning either at the single-view level or at the view-consensus level, but often neglect the possibility of the joint learning of single-view and consensus graphs. Third, many of them rely on the k -means for discretization of the spectral embeddings, which lack the ability to directly learn the graph with discrete cluster structure. In light of this, this article presents an efficient MVC approach via u nified and d iscrete b ipartite g raph l earning (UDBGL). Specifically, the anchor-based subspace learning is incorporated to learn the view-specific bipartite graphs from multiple views, upon which the bipartite graph fusion is leveraged to learn a view-consensus bipartite graph with adaptive weight learning. Furthermore, the Laplacian rank constraint is imposed to ensure that the fused bipartite graph has discrete cluster structures (with a specific number of connected components). By simultaneously formulating the view-specific bipartite graph learning, the view-consensus bipartite graph learning, and the discrete cluster structure learning into a unified objective function, an efficient minimization algorithm is then designed to tackle this optimization problem and directly achieve a discrete clustering solution without requiring additional partitioning, which notably has linear time complexity in data size. Experiments on a variety of multi-view datasets demonstrate the robustness and efficiency of our UDBGL approach. The code is available at https://github.com/huangdonghere/UDBGL.

Asymmetric double-winged multi-view clustering network for exploring diverse and consistent information

In unsupervised scenarios, deep contrastive multi-view clustering (DCMVC) is becoming a hot research spot, which aims to mine the potential relationships between different views. Most existing DCMVC algorithms focus on exploring the consistency information for the deep semantic features, while ignoring the diverse information on shallow features. To fill this gap, we propose a novel multi-view clustering network termed CodingNet to explore the diverse and consistent information simultaneously in this paper. Specifically, instead of utilizing the conventional auto-encoder, we design an asymmetric structure network to extract shallow and deep features separately. Then, by approximating the similarity matrix on the shallow feature to the zero matrix, we ensure the diversity for the shallow features, thus offering a better description of multi-view data. Moreover, we propose a dual contrastive mechanism that maintains consistency for deep features at both view-feature and pseudo-label levels. Our framework’s efficacy is validated through extensive experiments on six widely used benchmark datasets, outperforming most state-of-the-art multi-view clustering algorithms.

Multi-view clustering via dynamic unified bipartite graph learning

Multi-view clustering algorithms based on graph learning have the ability to extract the potential association between data samples, which has been a concern of many researchers in recent years. However, existing algorithms have two limitations: (1) they directly learn from the raw graph, which includes noise and outliers, and they construct the graph filter statically, biasing the clustering results; (2) during graph construction, they mainly use the information of a single structure and fail to fully extract the multi-granular structural information among the data. To address these issues, this paper proposes a novel multi-view clustering method via dynamic unified bipartite graph learning. Specifically, a learnable graph filter is first refined to dynamically filter the original data feature space, gradually filtering out the undesirable high-frequency noise and achieving a clustering-friendly smooth representation. Second, a unified bipartite graph is constructed by combining the multi-granular structural information of different views to better explore the distinct and common information of each view. In one framework, the dynamic filter and multi-granular structure information are combined to iteratively learn the unified bipartite graph. An efficient iterative algorithm is designed to decompose the objective function into small-scale subproblems for solving. Extensive experiments on benchmark datasets show the superiority of the proposed algorithm over several existing state-of-the-art multi-view clustering algorithms.

Multi-view clustering via latent consistency multi-graph fusion

Multi-view clustering (MVC) algorithms are prevalent in machine learning tasks due to their superiority in mining complementary information from multi-view data. Despite their widespread use, existing MVC approaches ignore the presence of inconsistencies in the multi-view features during multi-view data fusion, which leads to suboptimal clustering performance. To address this challenge, the paper presents a novel approach called multi-view clustering via latent consistency multi-graph fusion. Specifically, a novel latent representation learning model is devised to learn a unified representation from multi-view features, simultaneously capturing the high-order relationships within the multi-view data. Additionally, to alleviate the adverse effects caused by the inconsistency multi-view feature, a consistent multi-graph fusion module is proposed to provide consistent constraints for latent representations. To be specific, the graphs that reflect the manifold structure for each view are fused to guide a consistent latent representation learning. After that, an alternating optimization algorithm is designed to update variables iteratively and alternatively, among which the fusion weights are assigned adaptively. Finally, comprehensive experiments conducted on seven widely recognized benchmark multi-view datasets affirm the efficacy of the proposed method, showcasing its superiority over state-of-the-art approaches.

Multi-view reduced dimensionality K-means clustering with [formula omitted]-norm and Schatten [formula omitted]-norm

Recently, multi-view high dimensional data obtained from diverse domains or various feature extractors has drawn great attention due to its reflection of different properties or distributions. In this paper, we propose a novel unsupervised multi-view clustering method, which is called Multi-View Reduced Dimensionality K-means clustering (MRDKM) and integrates the dimension reduction mechanism, σ-norm, Schatten p-norm, and multi-view K-means clustering. Moreover, an unsupervised optimization scheme was proposed to solve the minimization problem with good convergence properties. Comprehensive evaluations of five benchmark datasets and comparisons with several multi-view clustering algorithms demonstrate the superiority of the proposed work.

Large-scale multi-view clustering via matrix factorization of consensus graph

Recently, anchors-based multi-view clustering methods have been widely concerned for they can not only significantly reduce the time complexity but also have good interpretability. However, the time consumption of optimization and spectral embedding with Singular Value Decomposition (SVD) is expensive for large-scale multi-view clustering due to the large-scale consensus graph. This paper proposes to factorize the consensus graph to directly obtain the low-dimension consensus embedding matrix by optimizing the objective function. Specifically, the consensus graph is factorized into a transition matrix and a low-dimension embedding matrix. Among them, the transition matrix is used to prevent the clustering time consumption from increasing significantly with the number of anchors, and the low-dimension embedding matrix is used to mine the low-dimension consensus information of each view. The proposed method demonstrates its superiority by outperforming eight multi-view clustering algorithms on nine datasets, as evidenced by the clustering results.

Multi-view and Multi-order Graph Clustering via Constrained [formula omitted]-norm

The graph-based multi-view clustering algorithms achieve decent clustering performance by consensus graph learning of the first-order graphs from different views. However, the first-order graphs are often sparse, lacking essential must-link information, which leads to suboptimal consensus graph. While high-order graphs can address this issue, a two-step strategy involving the selection of a fixed number of high-order graphs followed by their fusion may result in information loss or redundancy, restricting the exploration of high-order information. To address these challenges, we propose Multi-view and Multi-order Graph Clustering via Constrained l1,2-norm (MoMvGC), which mitigates the impact of graph sparsity on multi-view clustering. By innovatively designing constrained l1,2-norm, the model ingeniously integrates the selection of multi-order graphs and corresponding weight learning into a unified framework. Furthermore, MoMvGC not only enable sparse selection of multi-order graphs but also simultaneous selection of views. Afterwards, we design an efficient alternative optimization algorithm to solve the optimization problems in MoMvGC. The proposed model achieves state-of-the-art clustering performance on nine real-world datasets, with particularly notable improvements observed on the MSRC dataset, where the clustering accuracy is increased by 5.24% compared to the best baseline. Comprehensive experiments demonstrate the effectiveness and superiority of our model. The code is available at https://github.com/haonanxin/MoMvGC_code.

Deep embedding based tensor incomplete multi-view clustering

The majority of multi-view data are extracted from real life, which often lose information in some views. To solve this problem, existing incomplete multi-view clustering algorithms explore the valuable information from incomplete data while populating the missing information. Nevertheless, they suffer from the following two limitations: 1) The non-linear relationships of high-dimensional available data are not considered. 2) Although some methods utilize information from different views to fill in missing information, they still cannot precisely explore the missing information of each view. To this end, this article proposes a novel one-step incomplete multi-view framework, referred to as deep embedding based tensor incomplete multi-view clustering (DETIMC). Concretely, in this framework, the high-dimensional available data are projected into the low-dimensional embedding space by deep non-negative matrix factorization (NMF), which can obtain a clean space while capturing the complex non-linear relationship of available data. Moreover, a novel double-enhanced missing-view inferring strategy is developed, in which the weighted higher-order information of different views and the clustering structure information are simultaneously exploited. Comprehensive experiments on several benchmark datasets exhibit the superiority of DETIMC over conventional and state-of-the-art algorithms.

Dual auto-weighted multi-view clustering via autoencoder-like nonnegative matrix factorization

Multi-view clustering (MVC) can exploit the complementary information among multi-view data to achieve the satisfactory performance, thus having extensive potentials for practical applications. Although Nonnegative Matrix Factorization (NMF) has emerged as an effective technique for MVC, the existing NMF-based methods still have two main limitations: 1) They solely focus on the reconstruction of original data, which can be regarded as the decoder of an autoencoder, while neglecting the low-dimensional representation learning. 2) They lack the ability to effectively capture both linear and nonlinear structures of data. To solve these problems, in this paper, we propose a Dual Auto-weighted multi-view clustering model based on Autoencoder-like NMF (DA2NMF), which enables a comprehensive exploration of both linear and nonlinear structures. Specifically, we establish an autoencoder-like NMF model that learns linear low-dimensional representations by integrating data reconstruction and representation learning within a unified framework. Moreover, the adaptive graph learning is introduced to explore the nonlinear structures in data. We further design a dual auto-weighted strategy to adaptively compute weights for different views and low-dimensional representations, thereby obtaining an enhanced consistent graph. An effective algorithm based on Multiplicative Update Rule (MUR) is developed to solve the DA2NMF with the theoretical convergence guarantee. Experimental results show that the proposed DA2NMF can effectively improve the clustering performance compared with the state-of-the-art MVC algorithms.

Multi-OMICS approaches in cancer biology: New era in cancer therapy

Innovative multi-omics frameworks integrate diverse datasets from the same patients to enhance our understanding of the molecular and clinical aspects of cancers. Advanced omics and multi-view clustering algorithms present unprecedented opportunities for classifying cancers into subtypes, refining survival predictions and treatment outcomes, and unravelling key pathophysiological processes across various molecular layers. However, with the increasing availability of cost-effective high-throughput technologies (HTT) that generate vast amounts of data, analyzing single layers often falls short of establishing causal relations. Integrating multi-omics data spanning genomes, epigenomes, transcriptomes, proteomes, metabolomes, and microbiomes offers unique prospects to comprehend the underlying biology of complex diseases like cancer. This discussion explores algorithmic frameworks designed to uncover cancer subtypes, disease mechanisms, and methods for identifying pivotal genomic alterations. It also underscores the significance of multi-omics in tumor classifications, diagnostics, and prognostications. Despite its unparalleled advantages, the integration of multi-omics data has been slow to find its way into everyday clinics. A major hurdle is the uneven maturity of different omics approaches and the widening gap between the generation of large datasets and the capacity to process this data. Initiatives promoting the standardization of sample processing and analytical pipelines, as well as multidisciplinary training for experts in data analysis and interpretation, are crucial for translating theoretical findings into practical applications.

Anchor-based scalable multi-view subspace clustering

Multi-view clustering is a major topic in pattern recognition and machine learning. Common multi-view clustering algorithms construct similarity graphs from original samples and use them to perform spectral clustering. The time complexity of the singular value decomposition process in graph construction and spectral clustering is high, leading to high computational and memory costs. In addition, subsequent K-means clustering is sensitive to the initial points, yielding unstable clustering results. To address these issues, this study proposes a novel approach to reduce time overhead and memory space from two perspectives. First, a new anchor selection method is proposed to reduce the dimension of the original data by lowering the cost. Second, the self-representation matrix of multi-views is fused into a consistent graph matrix using the post-fusion technique, and the fused graph is directly processed in postprocessing. Furthermore, the proposed method directly obtains clustering results based on the connectivity of the fusion graph, eliminating the need for K-means postprocessing, which avoids the issue of unstable clustering results. Experimental results on artificial and real multi-view datasets indicate that the proposed algorithm is superior to existing algorithms.

Towards a unified framework for graph-based multi-view clustering

Recently, clustering data collected from various sources has become a hot topic in real-world applications. The most common methods for multi-view clustering can be divided into several categories: Spectral clustering algorithms, subspace multi-view clustering algorithms, matrix factorization approaches, and kernel methods. Despite the high performance of these methods, they directly fuse all similarity matrices of all views and separate the affinity learning process from the multiview clustering process. The performance of these algorithms can be affected by noisy affinity matrices. To overcome this drawback, this paper presents a novel method called One Step Multi-view Clustering via Consensus Graph Learning and Nonnegative Embedding (OSMGNE). Instead of directly merging the similarity matrices of different views, which may contain noise, a step of learning a consensus similarity matrix is performed. This step forces the similarity matrices of different views to be too similar, which eliminates the problem of noisy data. Moreover, the use of the nonnegative embedding matrix (soft cluster assignment matrix makes it possible to directly obtain the final clustering result without any extra step. The proposed method can solve five subtasks simultaneously. It jointly estimates the similarity matrix of all views, the similarity matrix of each view, the corresponding spectral projection matrix, the unified clustering indicator matrix, and automatically gives the weight of each view without the use of hyper-parameters. In addition, another version of our method is also studied in this paper. This method differs from the first one by using a consensus spectral projection matrix and a consensus Laplacian matrix over all views. An iterative algorithm is proposed to solve the optimization problem of these two methods. The two proposed methods are tested on several real datasets, which prove their superiority.

A Similarity Matrix Low-Rank Approximation and Inconsistency Separation Fusion Approach for Multiview Clustering

In recent years, multi-view clustering algorithms have achieved promising performance by exploiting the com-plementarity and consistency of different views. However, many multi-view spectral clustering methods only focus on the con-sistent information of views, and the time cost of feature decomposition is expensive. Moreover, these methods also require post-processing (e.g., k-means) to obtain the final clustering results. To overcome these limitations simultaneously, we propose a novel multi-view clustering algorithm. Firstly, the method removes the inconsistent information of the views through cross-view measurement to maintain consistent information. These inconsistencies may be caused by noise, corruptions, or view-specific properties and will affect the quality of the similarity matrix. Then, we learn a consensus embedding matrix with non-negative constraints by performing a low-rank decomposition of the consistency information. In this way, we can replace the eigendecomposition of the n × n Laplacian matrix in spectral clustering with the singular value decomposition of a n × c low-rank matrix to reduce the computational burden, where c ≪ n. Furthermore, due to the non-negative constraint, we can directly obtain the clustering results. Also, to consider the diversity of views, adaptive weighting is applied to different view data. Compared to state-of-the-art multi-view clustering methods on five benchmark multi-view datasets, we demonstrate the superiority and effectiveness of our approach. We release the source code at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/hulu88/FAMvC</uri> .

Multi-View Comprehensive Graph Clustering

Multi-view clustering algorithms have been successfully used in different consumer electronic products, such as common digital cameras and unmanned vehicles. Currently, existing multi-view graph clustering (MVGC) methods learn the similarity of directly connected samples for clustering. However, these MVGC methods can not fully consider the indirect relation among samples and high-order relation across multi-view data. In this paper, a new multi-view comprehensive graph clustering (MCGC) method is devised, which can fully learn the similarity based on (1) first-order proximity (FOP) (i.e. the direct relation of pairwise samples); (2) second-order proximity (SOP) (i.e. the indirect relation of pairwise samples); and (3) third-order proximity (TOP) (i.e. the three-order relation of multiple views). Since the operations of these three components are iteratively carried out, the interaction between similarity learning can be encouraged and the comprehensive graph can be generated effectively for clustering. In-depth experiments on six commonly benchmark datasets show the superiority of the MCGC method.

Anchor-based multi-view subspace clustering with hierarchical feature descent

Multi-view clustering has attracted growing attention owing to its capabilities of aggregating information from various sources and its promising horizons in public affairs. Up till now, many advanced approaches have been proposed in recent literature. However, there are several ongoing difficulties to be tackled. One common dilemma occurs while attempting to align the features of different views. Moreover, due to the fact that many existing multi-view clustering algorithms stem from spectral clustering, this results to cubic time complexity w.r.t. the number of dataset. However, we propose Anchor-based Multi-view Subspace Clustering with Hierarchical Feature Descent(MVSC-HFD) to tackle the discrepancy among views through hierarchical feature descent and project to a common subspace( STAGE 1), which reveals dependency of different views. We further reduce the computational complexity to linear time cost through a unified sampling strategy in the common subspace( STAGE 2), followed by anchor-based subspace clustering to learn the bipartite graph collectively( STAGE 3). Extensive experimental results on public benchmark datasets demonstrate that our proposed model consistently outperforms the state-of-the-art techniques. Our code is publicly available at https://github.com/QiyuanOu/MVSC-HFD/tree/main.

Multi-view fuzzy C-means clustering with kernel metric and local information for color image segmentation

PurposeMulti-view fuzzy clustering algorithms are not widely used in image segmentation, and many of these algorithms are lacking in robustness. The purpose of this paper is to investigate a new algorithm that can segment the image better and retain as much detailed information about the image as possible when segmenting noisy images.Design/methodology/approachThe authors present a novel multi-view fuzzy c-means (FCM) clustering algorithm that includes an automatic view-weight learning mechanism. Firstly, this algorithm introduces a view-weight factor that can automatically adjust the weight of different views, thereby allowing each view to obtain the best possible weight. Secondly, the algorithm incorporates a weighted fuzzy factor, which serves to obtain local spatial information and local grayscale information to preserve image details as much as possible. Finally, in order to weaken the effects of noise and outliers in image segmentation, this algorithm employs the kernel distance measure instead of the Euclidean distance.FindingsThe authors added different kinds of noise to images and conducted a large number of experimental tests. The results show that the proposed algorithm performs better and is more accurate than previous multi-view fuzzy clustering algorithms in solving the problem of noisy image segmentation.Originality/valueMost of the existing multi-view clustering algorithms are for multi-view datasets, and the multi-view fuzzy clustering algorithms are unable to eliminate noise points and outliers when dealing with noisy images. The algorithm proposed in this paper has stronger noise immunity and can better preserve the details of the original image.

Efficient and Effective One-Step Multiview Clustering.

Multiview clustering algorithms have attracted intensive attention and achieved superior performance in various fields recently. Despite the great success of multiview clustering methods in realistic applications, we observe that most of them are difficult to apply to large-scale datasets due to their cubic complexity. Moreover, they usually use a two-stage scheme to obtain the discrete clustering labels, which inevitably causes a suboptimal solution. In light of this, an efficient and effective one-step multiview clustering (E 2 OMVC) method is proposed to directly obtain clustering indicators with a small-time burden. Specifically, according to the anchor graphs, the smaller similarity graph of each view is constructed, from which the low-dimensional latent features are generated to form the latent partition representation. By introducing a label discretization mechanism, the binary indicator matrix can be directly obtained from the unified partition representation which is formed by fusing all latent partition representations from different views. In addition, by coupling the fusion of all latent information and the clustering task into a joint framework, the two processes can help each other and obtain a better clustering result. Extensive experimental results demonstrate that the proposed method can achieve comparable or better performance than the state-of-the-art methods. The demo code of this work is publicly available at https://github.com/WangJun2023/EEOMVC.

Multi-view [formula omitted]-means clustering algorithm based on redundant and sparse feature learning

In recent years, multi-view clustering has been widely concerned and studied. Being its data generally has high dimension and noise, how to effectively deal with redundant features in data from various views and improve the clustering effect is an important issue in multi-view clustering. In this paper, a robust multi-view clustering algorithm Multi-view K-means clustering algorithm based on redundant and sparse feature learning (RSFMVKM) considering redundancy and sparse feature learning is proposed, which can effectively reduce the dimensions of data from different views. Firstly, the K-means algorithm is extended to multi-view clustering, and the importance of each views is represented by learning the weight of each view. Secondly, the sparse feature representation is obtained by applying the norm constraint to the projection matrix in a single view, and the cosine similarity is used to depict the redundancy matrix, which can remove the redundant information and improve the clustering effect. Moreover, the proposed algorithm can effectively avoid the generation of empty classes, and the clustering index matrix is discrete, so the clustering results can be obtained directly. Finally, the proposed algorithm is compared with six state-of-the-art multi-view clustering algorithms on six public datasets, and the experimental results show that the algorithm is effective.

Dissimilarity-based indicator graph learning for clustering

Although learning a similarity graph with a 0–1 value is regarded as an important issue for clustering tasks, this topic has been scarcely reported in the literature since its NP-hard. Along these lines, this issue is addressed in the work by introducing a novel concept. Concretely, the indicator graph clustering (IGC) method was proposed. Based on the dissimilarity and coordinate descent method, IGC is able to learn an indicator graph with a 0–1 value, which is just the similarity graph with a 0–1 value. Meanwhile, the clustering result can be directly obtained by the indicator graph. From the experimental results on several datasets, it was verified that IGC performs faster than k-means, especially on high-dimensional datasets. Furthermore, IGC was extended to a multi-view version, named indicator graph clustering for multi-view (IGCMV). Our model fused the clustering indicator matrices from different views, which can be obtained by IGC, into the final clustering indicator matrix by a projection matrix. To the best of our knowledge, this work is the first attempt to learn a consensus indicator graph from indicator graphs of all views. In addition different from most of the existing multi-view clustering algorithms, the proposed IGCMV enables the production of the final clustering result, as well as each view’s clustering result simultaneously. This is helpful for measuring the quality of each view. IGCMV can be efficiently optimized by an alternative iteration strategy. Extensive experiments were also carried out on several benchmark multi-view datasets demonstrating the effectiveness of the developed algorithm to the state-of-the-art multi-view clustering algorithms.