Multi-view Clustering Problem Research Articles

Layer-wise normalized deep incomplete multiview nonnegative matrix factorization

Deep (semi-)nonnegative matrix factorization based models have demonstrated promising performance in the multiview clustering tasks. However, they are all developed upon the assumption that the multiview datasets are complete in all views, and need to be pre-trained layer-by-layer to get reasonable results. Addressing these issues, an algorithm named layer-wise normalized deep incomplete multiview nonnegative matrix factorization (LWNdimNMF) is presented in this paper to tackle the multiview clustering problem with arbitrary missing views. Specifically, a novel layer-wise normalization strategy is developed to constrain the value distribution of the factor matrix in each layer to ensure the numerical stability of the model and make it robust to the initialization. Another benefit of this strategy is that it can effectively depress the objective function value and make the model fit data well. An instance selection alignment is adopted to fuse the incomplete multiple views by aligning the paired instances. To discover the high-order geometrical information in each incomplete view, a hypergraph regularization is incorporated into the proposed model. An effective iterative optimization algorithm based on the multiplicative updating rules is designed to solve the minimizing problem. Extensive experiments are conducted to evaluate the effectiveness of the proposed model by comparing with several state-of-the-art (SOTA) incomplete multiview clustering (IMC) methods on ten real-world multiview datasets. The source code is available at: https://github.com/GuoshengCui/LWNdimNMF.

Data Completion-Guided Unified Graph Learning for Incomplete Multi-View Clustering

Due to its heterogeneous property, multi-view data has been widely concerned over single-view data for performance improvement. Unfortunately, some instances may be with partially available information because of some uncontrollable factors, for which the incomplete multi-view clustering (IMVC) problem is raised. IMVC aims to partition unlabeled incomplete multi-view data into their clusters by exploiting the heterogeneity of multi-view data and overcoming the difficulty of data loss. However, most existing IMVC methods like BSV, MIC, OMVC, and IVC tend to conduct basic completion processing on the input data, without taking advantage of the correlation between samples and information redundancy. To overcome the above issue, we propose one novel IMVC method named data completion-guided unified graph learning (DCUGL), which could complete the data of missing views and fuse multiple learned view-specific similarity matrices into one unified graph. Specifically, we first reduce the dimension of the input data to learn multiple view-specific similarity matrices. By stacking all view-specific similarity matrices, DCUGL constructs a third-order tensor with the low-rank constraint, such that sample correlation within and between views can be well explored. Finally, by dividing the original data into observed data and unobserved data, DCUGL can infer and complete the missing data according to the view-specific similarity matrices, and obtain a unified graph, which can be directly used for clustering. To solve the proposed model, we design an iterative algorithm, which is based on the alternating direction method of multipliers framework. The proposed model proves to be superior by benchmarking on six challenging datasets compared with state-of-the-art IMVC methods.

DVSAI: Diverse View-Shared Anchors Based Incomplete Multi-View Clustering

In numerous real-world applications, it is quite common that sample information is partially available for some views due to machine breakdown or sensor failure, causing the problem of incomplete multi-view clustering (IMVC). While several IMVC approaches using view-shared anchors have successfully achieved pleasing performance improvement, (1) they generally construct anchors with only one dimension, which could deteriorate the multi-view diversity, bringing about serious information loss; (2) the constructed anchors are typically with a single size, which could not sufficiently characterize the distribution of the whole samples, leading to limited clustering performance. For generating view-shared anchors with multi-dimension and multi-size for IMVC, we design a novel framework called Diverse View-Shared Anchors based Incomplete multi-view clustering (DVSAI). Concretely, we associate each partial view with several potential spaces. In each space, we enable anchors to communicate among views and generate the view-shared anchors with space-specific dimension and size. Consequently, spaces with various scales make the generated view-shared anchors enjoy diverse dimensions and sizes. Subsequently, we devise an integration scheme with linear computational and memory expenditures to integrate the outputted multi-scale unified anchor graphs such that running spectral algorithm generates the spectral embedding. Afterwards, we theoretically demonstrate that DVSAI owns linear time and space costs, thus well-suited for tackling large-size datasets. Finally, comprehensive experiments confirm the effectiveness and advantages of DVSAI.

Adaptive Feature Imputation with Latent Graph for Deep Incomplete Multi-View Clustering

In recent years, incomplete multi-view clustering (IMVC), which studies the challenging multi-view clustering problem on missing views, has received growing research interests. Previous IMVC methods suffer from the following issues: (1) the inaccurate imputation for missing data, which leads to suboptimal clustering performance, and (2) most existing IMVC models merely consider the explicit presence of graph structure in data, ignoring the fact that latent graphs of different views also provide valuable information for the clustering task. To overcome such challenges, we present a novel method, termed Adaptive feature imputation with latent graph for incomplete multi-view clustering (AGDIMC). Specifically, it captures the embbedded features of each view by incorporating the view-specific deep encoders. Then, we construct partial latent graphs on complete data, which can consolidate the intrinsic relationships within each view while preserving the topological information. With the aim of estimating the missing sample based on the available information, we utilize an adaptive imputation layer to impute the embedded feature of missing data by using cross-view soft cluster assignments and global cluster centroids. As the imputation progresses, the portion of complete data increases, contributing to enhancing the discriminative information contained in global pseudo-labels. Meanwhile, to alleviate the negative impact caused by inferior impute samples and the discrepancy of cluster structures, we further design an adaptive imputation strategy based on the global pseudo-label and the local cluster assignment. Experimental results on multiple real-world datasets demonstrate the effectiveness of our method over existing approaches.

Predicting Heart Disease Using Collaborative Clustering and Ensemble Learning Techniques

Different data types are frequently included in clinical data. Applying machine learning algorithms to mixed data can be difficult and impact the output accuracy and quality. This paper proposes a hybrid model of unsupervised and supervised learning techniques, which can be used in modelling and processing mixed data with an application in heart disease diagnosis. The model consists of two main components: collaborative clustering and combining decisions (the ensemble approach). The mixed data clustering problem is considered as a multi-view clustering problem; each view is processed using specialised clustering algorithms. Since each algorithm operates on a different space of the data set’s features, a novel collaborative framework was proposed that promotes the clustering process through information exchange between the different clustering algorithms, thereby producing expert models that model other spaces of the data set’s features. The expectation maximisation algorithm forms the foundation for this optimisation process, enhancing the collaborative term representing entropy; excellent convergence characteristics are therefore ensured. An ensemble approach similar to the stacking approach was used. The logistic regression model was utilised as a meta-classifier, training the expert model prediction results, and was subsequently used to predict the final output. The results prove the efficacy of this collaborative approach in optimising different clustering algorithms and meta-classifier outcomes.

Geometric-inspired graph-based Incomplete Multi-view Clustering

Multi-view clustering methods group data into different clusters by discovering the consensus in heterogeneous sources, which however becomes difficult when partial views of real-world data are missing. Consequently, reducing the impact of missing views and leveraging available views are the key concerns for the Incomplete Multi-view Clustering (IMvC) problem. In this research, we take an innovative, geometry-based perspective to investigate the IMvC problem under a commonly-used weight aggregation framework. We conduct a geometric analysis to understand how missing views shift the aggregation solution from the one achieved with full views, subsequently impacting the clustering result. Drawing from our analysis, we introduce a weight reallocation approach that minimizes the shift and approximates the full-view solution by reallocating the factual weight of each available view. Furthermore, we address the IMvC problem by using our reallocation method on a graph aggregation algorithm to obtain reliable clusters. Our extensive experiments demonstrate that our proposed approach outperforms previous IMvC methods, reporting superior results on four datasets with three metrics. Especially, on the Caltech101-7 dataset with 40 percent missing data, our method achieves an accuracy of 0.686, which significantly outperforms the results of other comparison methods that are no larger than 0.662. Further, our method can be used as a flexible plugin to improve other weight aggregation algorithms. The source code of this work is publicly available at https://github.com/bjlfzs/Geometric-Inspired-Graph-based-Incomplete-Multi-view-Clustering.

Multiview Deep Online Clustering: An Application to Online Research Topic Modeling and Recommendations

In today&#x2019;s scenario, a large number of scientific articles on various domains are being published everyday, resulting in a rapid change in the trends of research topics. Retrieving the trending topics, evaluating the trends and extracting the scope of topics could be beneficial to young researchers, which can be recommended for future scope. Publication of articles is a continuous process, and so is the evolution of topics as well as the scope. The dynamic behavior of topics can be handled by continuously updating the partitioning of incoming articles arriving in a streaming manner. This article proposes an online clustering approach utilizing the autoencoder, which is capable of handling the online stream of articles. The training is performed by considering the multiple views of articles and posing the topic modeling (TM) problem as a multiview (MV) clustering problem. In addition, we employ an evolutionary-based approach to the latent representation of data to automatically determine the number of clusters. To learn the nonlinear mapping appropriately and to generate clusters effectively, the model simultaneously optimizes the reconstruction loss and clustering loss in an MV framework. The developed method has experimented on <i>ArXiv</i> dataset to determine the trending topics, scope, and recommendation of topics. The superiority of the proposed method in generating clusters as well as in determining scope is shown by comparing the results with many existing and baseline methods.

Multi-View Adjacency-Constrained Hierarchical Clustering

This paper explores the problem of multi-view clustering, which aims to promote clustering performance with multi-view data. The majority of existing methods have problems with parameter adjustment and high computational complexity. Moreover, in the past, there have been few works based on hierarchical clustering to learn the granular information of multiple views. To overcome these limitations, we propose a simple but efficient framework: Multi-view adjacency-Constrained Hierarchical Clustering (MCHC). Specifically, MCHC mainly consists of three parts: including the Fusion Distance matrices with Extreme Weights (FDEW); adjacency-Constrained Nearest Neighbor Clustering (CNNC); and the internal evaluation Index based on Rawls' Max-Min criterion (MMI). FDEW aims to learn a fusion distance matrix set, which not only uses complementary information among multiple views, but exploits the information from each single view. CNNC is utilized to generate multiple partitions based on FDEW, and MMI is designed for choosing the best one from the multiple partitions. In addition, we propose a parameter-free version of MCHC (MCHC-PF). Without any parameter selection, MCHC-PF can give partitions at different granularity levels with a low time complexity. Comprehensive experiments tested on eight real-world datasets validate the superiority of the proposed methods compared with the 13 current state-of-the-art methods.

Multiview Clustering with Self Representation and Structural Constraints

Multi-view clustering, which divides objects into multiple clusters with high intra-cluster and low inter- cluster similarity for all perspectives, is of enormous significance for uncovering the mechanisms of systems. Multi-view data effectively model and characterise the underlying complex systems. Because they only consider the shared characteristics of objects or their correlation, current algorithms are criticised for their subpar performance because they ignore the heterogeneity and structural constraints of different views. A brand-new network-based method called Multi-view Clustering with Self-representation and Structural Constraints (MCSSC), which combines matrix factorization with low-rank representation of various perspectives, is presented to address these issues. In particular, a network is built for each view to reduce heterogeneity from multi-view data, converting the multi-view clustering problem into the multi-layer networks clustering problem. The MCSSC factorises network-related matrices by projecting them into a shared space and simultaneously trains an affinity graph for objects in multiple views with self-representation in order to extract the shared properties of multiple views. The structural constraint is applied to the affinity graph, where the clusters are identified, to aid with clustering. Numerous tests show that MCSSC performs noticeably better than the state-of-the-art in terms of accuracy, indicating the superiority of the suggested method

Robust and optimal neighborhood graph learning for multi-view clustering

In recent years, researchers have proposed many graph-based multi-view clustering (GMC) algorithms to solve the multi-view clustering (MVC) problem. However, there are still some limitations in the existing GMC algorithm. In these algorithms, a graph is usually constructed to represent the relationship between the samples in a view; however, it cannot represent the relationship very well since it is often preconstructed. In addition, these algorithms ignore the robustness problem of each graph and high-level information between different graphs. Then, in the paper, we propose a novel MVC method, i.e., robust and optimal neighborhood graph learning for MVC (RONGL/MVC). Specifically, we first build an initial graph for each view. However, these initial graphs cannot represent the relationship between the samples in each view well, so we look for the optimal graph with k connected components in the neighborhood of each initial graph, where k is the number of clusters. Then, to improve the robustness of RONGL/MVC, we reconstruct the optimal graph with the self-representation matrix. Furthermore, we stack all the self-representation matrices into a tensor and impose the tensor low-rank constraint, which can maximize consistent features and explore the high-order relationship between optimal graphs. In addition, we provide an optimization strategy by utilizing the Augmented Lagrange Multiplier (ALM) method. Experimental results on several datasets indicate that RONGL/MVC outperforms state-of-the-art methods.

Effective Incomplete Multi-View Clustering via Low-Rank Graph Tensor Completion

In the past decade, multi-view clustering has received a lot of attention due to the popularity of multi-view data. However, not all samples can be observed from every view due to some unavoidable factors, resulting in the incomplete multi-view clustering (IMC) problem. Up until now, most efforts for the IMC problem have been made on the learning of consensus representations or graphs, while many missing views are ignored, making it impossible to capture the information hidden in the missing view. To overcome this drawback, we first analyzed the low-rank relationship existing inside each graph and among all graphs, and then propose a novel method for the IMC problem via low-rank graph tensor completion. Specifically, we first stack all similarity graphs into a third-order graph tensor and then exploit the low-rank relationship from each mode using the matrix nuclear norm. In this way, the connection hidden between the missing and available instances can be recovered. The consensus representation can be learned from all completed graphs via multi-view spectral clustering. To obtain the optimal multi-view clustering result, incomplete graph recovery and consensus representation learning are integrated into a joint framework for optimization. Extensive experimental results on several incomplete multi-view datasets demonstrate that the proposed method can obtain a better clustering performance in comparison with state-of-the-art incomplete multi-view clustering methods.

Incomplete Multiview Clustering via Cross-View Relation Transfer

In this paper, we consider the problem of multi-view clustering on incomplete views. Compared with complete multi-view clustering, the view-missing problem increases the difficulty of learning common representations from different views. To address the challenge, we propose a novel incomplete multi-view clustering framework, which incorporates cross-view relation transfer and multi-view fusion learning. Specifically, based on the consistency existing in multi-view data, we devise a cross-view relation transfer-based completion module, which transfers known similar inter-instance relationships to the missing view and recovers the missing data via graph networks based on the transferred relationship graph. Then the view-specific encoders are designed to extract the recovered multi-view data, and an attention-based fusion layer is introduced to obtain the common representation. Moreover, to reduce the impact of the error caused by the inconsistency between views and obtain a better clustering structure, a joint clustering layer is introduced to optimize recovery and clustering simultaneously. Extensive experiments conducted on several real datasets demonstrate the effectiveness of the proposed method.

Graph Contrastive Partial Multi-View Clustering

With the diversity of information acquisition, data is stored and transmitted in an increasing number of modalities. Nevertheless, it is not unusual for parts of the data to be lost in some views due to unavoidable acquisition, transmission or storage errors. In this paper, we propose an augmentation-free graph contrastive learning framework to solve the problem of partial multi-view clustering. Notably, we suppose that the representations of similar samples (i.e., belonging to the same cluster) should be similar. This is distinct from the general unsupervised contrastive learning that assumes an image and its augmentations share a similar representation. Specifically, relation graphs are constructed using the nearest neighbors to identify existing similar samples, then the constructed inter-instance relation graphs are transferred to the missing views to build graphs on the corresponding missing data. Subsequently, two main components, within-view graph contrastive learning and cross-view graph consistency learning, are devised to maximize the mutual information of different views within a cluster. The proposed approach elevates instance-level contrastive learning and missing data inference to the cluster-level, effectively mitigating the impact of individual missing data on clustering. Experiments on several challenging datasets demonstrate the superiority of our proposed methods.

Multi-View Clustering via Triplex Information Maximization.

In this paper, we address the problem of multi-view clustering (MVC), integrating the close relationships among views to learn a consistent clustering result, via triplex information maximization (TIM). TIM works by proposing three essential principles, each of which is realized by a formulation of maximization of mutual information. (1) Principle 1: Contained. The first and foremost thing for MVC is to fully employ the self-contained information in each view. (2) Principle 2: Complementary. The feature-level complementary information across pairwise views should be first quantified and then integrated for improving clustering. (3) Principle 3: Compatible. The rich cluster-level shared compatible information among individual clustering of each view is significant for ensuring a better final consistent result. Following these principles, TIM can enjoy the best of view-specific, cross-view feature-level, and cross-view cluster-level information within/among views. For principle 2, we design an automatic view correlation learning (AVCL) mechanism to quantify how much complementary information across views by learning the cross-view weights between pairwise views automatically, instead of view-specific weights as most existing MVCs do. Specifically, we propose two different strategies for AVCL, i.e., feature-based and cluster-based strategy, for effective cross-view weight learning, thus leading to two versions of our method, TIM-F and TIM-C, respectively. We further present a two-stage method for optimization of the proposed methods, followed by the theoretical convergence and complexity analysis. Extensive experimental results suggest the effectiveness and superiority of our methods over many state-of-the-art methods.

Cross-view graph matching for incomplete multi-view clustering

Multi-view clustering (MVC) focuses on adaptively partitioning data from diverse sources into the respective groups and has been widely studied under the assumption of complete data. However, real-world applications often encounter a more realistic incomplete multi-view clustering (IMVC) problem, where data samples are missing in certain views. There are two challenges in IMVC: 1) how to reduce the impact of the missing instances; 2) how to effectively extract the consistent information to cluster the multi-view data. To address the challenges, we propose an adaptive graph learning framework for IMVC, which optimizes the missing information to fit the intrinsic structure of each view and clusters the multi-view data by cross-view graph matching. The proposed method mainly consists of three steps. Firstly, owing to the outstanding performance of the intrinsic structure of data, we adapt it to complete the missing data of each view. Secondly, the connection graph of each view from a projection space is adaptively constructed wherein the data points are connected if and only if they belong to the same cluster. Thirdly, we further introduce a cross-view graph matching strategy to appropriately utilize complementary multi-views information and preserve view-specific semantic information. We develop an iterative algorithm for solving the proposed model. Numerical experiments on several standard datasets demonstrate the effectiveness of the proposed method.

Adversarial Incomplete Multiview Subspace Clustering Networks.

Multiview clustering aims to leverage information from multiple views to improve the clustering performance. Most previous works assumed that each view has complete data. However, in real-world datasets, it is often the case that a view may contain some missing data, resulting in the problem of incomplete multiview clustering (IMC). Previous approaches to this problem have at least one of the following drawbacks: 1) employing shallow models, which cannot well handle the dependence and discrepancy among different views; 2) ignoring the hidden information of the missing data; and 3) being dedicated to the two-view case. To eliminate all these drawbacks, in this work, we present the adversarial IMC (AIMC) framework. In particular, AIMC seeks the common latent representation of multiview data for reconstructing raw data and inferring missing data. The elementwise reconstruction and the generative adversarial network are integrated to evaluate the reconstruction. They aim to capture the overall structure and get a deeper semantic understanding, respectively. Moreover, the clustering loss is designed to obtain a better clustering structure. We explore two variants of AIMC, namely: 1) autoencoder-based AIMC (AAIMC) and 2) generalized AIMC (GAIMC), with different strategies to obtain the multiview common representation. Experiments conducted on six real-world datasets show that AAIMC and GAIMC perform well and outperform the baseline methods.

Multi-view clustering with orthogonal mapping and binary graph

The multi-view clustering problem has attracted considerable attention over recent years for the remarkable clustering performance due to exploiting complementary information from multiple views. Most existing related research work processes data in the decimal real value space that is not the most compatible space for computers. Binary code learning, also known as hashing technology, is well-known for fast Hamming distance computation, less storage requirement and accurate calculation results. The Hamming space is most enjoyed by computers because of binary/hash codes. Several studies combine multi-view clustering with binary code learning for improving clustering performance. However, there is much redundant information contained in the learned binary codes, which negatively affects the clustering performance, but these studies ignore eliminating redundant information for learning compact codes. In addition, they don’t give a unified (one-step) clustering framework with binary graph structure, which doesn’t lead to the optimal clustering result due to the information loss during the two-step process. In this paper, to cope with the two issues, we propose an orthogonal mapping binary graph method (OMBG) for the multi-view clustering problem, which makes the mapping matrix of every view orthogonalize for eliminating redundant information and embeds a binary graph structure into the unified binary multi-view clustering framework for extracting local geometric structure information of binary codes and achieving the optimal clustering result. Furthermore, we design an effective optimization algorithm based on alternating direction minimization to solve the model of OMBG. Extensive experiments performed on four frequently-used benchmark multi-view datasets illustrate the superiority of OMBG which is compared with some state-of-the-art clustering baselines.

Unbalanced Incomplete Multi-View Clustering Via the Scheme of View Evolution: Weak Views are Meat; Strong Views Do Eat

Incomplete multi-view clustering is an important technique to deal with real-world incomplete multi-view data. Previous methods assume that all views have the same incompleteness, i.e., balanced incompleteness. However, different views often have distinct incompleteness, i.e., unbalanced incompleteness, which results in <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">strong views</i> (low-incompleteness views) and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">weak views</i> (high-incompleteness views). The unbalanced incompleteness prevents us from directly using previous methods. In this paper, inspired by the effective biological evolution theory, we design the novel scheme of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">view evolution</i> to cluster strong and weak views. Moreover, we propose an Unbalanced Incomplete Multi-view Clustering method (UIMC), which is the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">first</i> effective method based on view evolution for unbalanced incomplete multi-view clustering. Compared with previous methods, UIMC has two unique advantages: 1) it proposes weighted multi-view subspace clustering to integrate unbalanced incomplete views, which effectively solves the unbalanced incomplete multi-view clustering problem; 2) it designs the low-rank representation to recover the data, which diminishes the impact of the incompleteness and noises. Extensive experimental results demonstrate that UIMC improves the clustering performance by up to 40% on three evaluation metrics over other state-of-the-art methods. We provide codes for all of our experiments in <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/ZeusDavide/TETCI_UIMC</uri> .

An attention-based framework for multi-view clustering on Grassmann manifold

The key problem of multi-view clustering is to handle the inconsistency among multiple views. This article proposes an attention-based framework for multi-view clustering on Grassmann manifold (AMCGM). To be specific, the proposed AMCGM framework aims to learn a representative element on Grassmann manifold with the following four highlights: 1) AMCGM framework performs an attention-based weighted-learning scheme to capture the difference of views; 2) The clustering results can be directly generated by the structured graph learned via AMCGM, avoiding the randomness caused by traditional label-generation procedures, such as K-means clustering; 3) AMCGM has high extensibility since it can generate many multi-view clustering models on Grassmann manifold; 4) On Grassmann manifold, the relationship between the projection metric (PM)-based multi-view clustering model and squared projection metric (SPM)-based model is studied. Based on AMCGM framework, we propose some generated models and provide some useful conclusions. Moreover, to solve the optimization problems involved in the proposed AMCGM framework and generated models, we propose an efficiently iterative algorithm and provide rigorous convergence analysis. Extensive experimental results demonstrate the superb performance of our framework.

Unified Tensor Framework for Incomplete Multi-view Clustering and Missing-view Inferring

In this paper, we propose a novel method, referred to as incomplete multi-view tensor spectral clustering with missing-view inferring (IMVTSC-MVI) to address the challenging multi-view clustering problem with missing views. Different from the existing methods which commonly focus on exploring the certain information of the available views while ignoring both of the hidden information of the missing views and the intra-view information of data, IMVTSC-MVI seeks to recover the missing views and explore the full information of such recovered views and available views for data clustering. In particular, IMVTSC-MVI incorporates the feature space based missing-view inferring and manifold space based similarity graph learning into a unified framework. In such a way, IMVTSC-MVI allows these two learning tasks to facilitate each other and can well explore the hidden information of the missing views. Moreover, IMVTSC-MVI introduces the low-rank tensor constraint to capture the high-order correlations of multiple views. Experimental results on several datasets demonstrate the effectiveness of IMVTSC-MVI for incomplete multi-view clustering.