Multi-view Data Research Articles

Tensor-Based Incomplete Multi-View Clustering With Low-Rank Data Reconstruction and Consistency Guidance

We propose a new approach, called Tensor-based Incomplete Multi-view Clustering with Low-rank data Reconstruction and Consistency guidance (TIMC-RC), to perform clustering on multi-view data with missing views. Existing methods usually leverage original incomplete data to explore the partial correlations among multiple views, and do not make sufficient use of both consistent and complementary information across views. To explore the full information of missing and available views, TIMC-RC introduces low-rank data reconstruction and consistency view establishment. Specifically, 1) it adopts a low-rank constraint to reconstruct data representations so as to reduce the negative effect of missing data and obtain more reasonable data representations. 2) It builds a new consistency view by self-representation matrices and therefore explores the consistent correlation of different views. 3) It formalizes view-specific self-representation matrices and the consistent matrix as a tensor and utilizes the tensor singular value decomposition-based nuclear norm to enhance the consistency and complementarity of multi-view representations. Experiments conducted on eight benchmarks verify the effectiveness and advancement of the proposed TIMC-RC.

A multi-view assisted registration network for MRI registration pre- and post-therapy.

Image registration of magnetic resonance imaging (MRI) pre- and post-therapy is an important part of evaluating the effect of therapy in tumor patients. The accuracy of evaluation results heavily relies on the alignment of the MRI image after registration. Although recent advancements have been made in medical image registration, applying these methods to MRI registration pre- and post-therapy remains challenging. Existing methods typically utilize single-view data for registration. However, when applied to MRI data where some slices are clear while others are blurred, these methods can be misled by erroneous spatial information in the blurred regions, leading to poor registration outcomes. To mitigate the interference caused by erroneous spatial information in single-view data, this paper proposes a multi-stream fusion-assisted registration network that incorporates different-view MRIs of the same patient at the same site. Additionally, a cross-attention guided fusion module is designed within the network to effectively utilize accurate spatial information from multi-view data. The proposed approach was evaluated on clinical data, and the experimental results demonstrated that incorporating multiple view data as auxiliary information significantly enhances the accuracy of MRI image registration before and after radiotherapy.

Joint Shared-and-Specific Information for Deep Multi-View Clustering

Multi-view data describes an image sample with different modalities of features, thus provides a more comprehensive description of data. Its three basic characteristics, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e</i> ., consensus, complementary and redundancy, determine its performances in computer vision tasks. In this paper, we effectively exploit the above three characteristics to propose a deep learning scheme with joint shared-and-specific information (JSSI) for multi-view clustering. Aiming at facilitating the consensus, JSSI extracts shared information of multi-view data via an adversarial similarity constraint, which is realized by classification and discrimination interactions. Aiming at reducing the redundancy, JSSI separate out view-specific features and prevent them from interfering with the shared features via a difference constraint. Aiming at ensuring the complementary, JSSI aligns the shared features and then concatenates them with the specific features. We examine the effectiveness of JSSI with multi-view clustering on real-world datasets, such as faces and indoor scenes. Extensive experiments and comparisons show that JSSI outperforms other state-of-the-art methods in most of these datasets.

Multiview Subspace Clustering With Multilevel Representations and Adversarial Regularization.

Multiview subspace clustering has turned into a promising technique due to its encouraging ability to discover the underlying subspace structure. In recent studies, a lot of subspace clustering methods have been developed to strengthen the clustering performance of multiview data, but these methods rarely consider simultaneously the nonlinear structure and multilevel representation (MLR) information in multiview data as well as the data distribution of latent representation. To address these problems, we develop a new Multiview Subspace Clustering with MLRs and Adversarial Regularization (MvSC-MRAR), where multiple deep auto-encoders are utilized to model nonlinear structure information of multiview data, multiple self-expressive layers are introduced into each deep auto-encoder to extract multilevel latent representations of each view data, and diversity regularizations are designed to preserve complementary information contained in different layers and different views. Furthermore, a universal discriminator based on adversarial training is developed to enforce the output of each encoder to obey a given prior distribution, so that the affinity matrix for spectral clustering (SPC) is more realistic. Comprehensive empirical evaluation with nine real-world multiview datasets indicates that our proposed MvSC-MRAR achieves significant improvements than several state-of-the-art methods in terms of clustering accuracy (ACC) and normalized mutual information (NMI).

A Comprehensive Survey on Multi-View Clustering

The development of information gathering and extraction technology has led to the popularity of multi-view data, which enables samples to be seen from numerous perspectives. Multi-view clustering (MVC), which groups data samples by leveraging complementary and consensual information from several views, is gaining popularity. Despite the rapid evolution of MVC approaches, there has yet to be a study that provides a full MVC roadmap for both stimulating technical improvements and orienting research newbies to MVC. In this article, we review recent MVC techniques with the purpose of exhibiting the concepts of popular methodologies and their advancements. This survey not only serves as a unique MVC comprehensive knowledge for researchers but also has the potential to spark new ideas in MVC research. We summarise a large variety of current MVC approaches based on two technical mechanisms: heuristic-based multi-view clustering (HMVC) and neural network-based multi-view clustering (NNMVC). We end with four technological approaches within the category of HMVC: nonnegative matrix factorisation, graph learning, latent representation learning, and tensor learning. Deep representation learning and deep graph learning are two technical methods that we demonstrate in NNMVC. We also show 15 publicly available multi-view datasets and examine how representative MVC approaches perform on them. In addition, this study identifies the potential research directions that may require further investigation in order to enhance the further development of MVC.

Omics‐Imaging Data Integration via Mediation Analysis with High‐Dimensional Exposures and Mediators

AbstractBackgroundMultimodal technologies have transformed AD research in recent years, by collecting different types of data from the same group of subjects and enabling the investigation of complex interrelated mechanisms underlying AD development.MethodIn this article, a mediation analysis approach with high‐dimensional exposures and high‐dimensional mediators is proposed to integrate data collected from multiple platforms. The proposed method combines principal component analysis with penalized least squares estimation for a set of linear structural equation models. The former reduces the dimensionality and produces uncorrelated linear combinations of the exposure variables, whereas the latter achieves simultaneous path selection and effect estimation while allowing the mediators to be correlated.ResultApplying the method to the AD data, it identifies numerous interesting protein peptides, brain regions, and protein‐structure‐memory paths, which are in accordance with and also supplement existing findings of AD research.ConclusionThe proposed approach is designed to integrate multiview data via a mediation framework. Under biological mechanistic assumptions, it articulates the underlying mechanisms and interrelations between data modalities and further infers disease pathology.

Transferable Takagi-Sugeno-Kang Fuzzy Classifier With Multi-Views for EEG-Based Driving Fatigue Recognition in Intelligent Transportation

The safety monitoring system of intelligent transportation provides driving fatigue warning and risk control. Electroencephalogram (EEG) signals can directly reflect the neuronal activity of the brain. The detection and early warning of driving fatigue using EEG signals has important practical significance. However, because of the non-stationarity and timeliness of EEG signals, the single feature detection method is significantly impacted by data distribution differences. In this paper, in the framework of multi-input multi-output (MIMO) Takagi-Sugeno-Kang (TSK) fuzzy system, transferable TSK fuzzy classifier with multi-views (T-TSK-MV) is developed for EEG-based driving fatigue recognition in intelligent transportation. First, in view-specific consequent parameter learning, the view-specific consequent regularizer is designed based on technologies of ridge regression, maximum mean discrepancy (MMD), and manifold regularization, which becomes the bridge to transfer the discriminative information from the related domain to the target domain. In addition, the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\ell_{2,1} $</tex-math> </inline-formula> -norm sparse constraint on consequent parameters is used to simplify fuzzy rules. Then multi-view learning is integrated into the consequent parameter learning, in which T-TSK-MV explores the view-shared consequent regularizer and adaptively assigns weights to each view. The <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\ell_{2,1} $</tex-math> </inline-formula> -norm sparse constraint on view-shared consequent regularizer can effectively exploit the local structure of multi-view data. Finally, the fuzzy classifier is constructed on view-specific regularizers and view weights. The experiment on real-word datasets shows that the proposed fuzzy classifier can significantly improve the driving fatigue recognition performance.

Multi-view Deep Gaussian Processes for Supervised Learning.

Multi-view learning is a widely studied topic in machine learning, which considers learning with multiple views of samples to improve the prediction performance. Even though some approaches have sprung up recently, it is still challenging to jointly explore information contained in different views. Multi-view deep Gaussian processes have shown strong advantages in unsupervised representation learning. However, they are limited when dealing with labeled multi-view data for supervised learning, and ignore the application potential of uncertainty estimation. In this paper, we propose a supervised multi-view deep Gaussian process model (named SupMvDGP), which uses the label of the views to further improve the performance, and takes the quantitative uncertainty estimation as a supplement to assist humans to make better use of prediction. According to the diversity of views, the SupMvDGP can establish asymmetric depth structure to better model different views, so as to make full use of the property of each view. We provide a variational inference method to effectively solve the complex model. Finally, we conduct comprehensive comparative experiments on multiple real world datasets to evaluate the performance of SupMvDGP. The experimental results show that the SupMvDGP achieves the state-of-the-art results in multiple tasks, which verifies the effectiveness and superiority of the proposed approach. Meanwhile, we provide a case study to show that the SupMvDGP has the ability to provide uncertainty estimation than alternative deep models, which can alert people to better treat the prediction results in high-risk applications.

Multiview Data Clustering with Similarity Graph Learning Guided Unsupervised Feature Selection.

In multiview data clustering, consistent or complementary information in the multiview data can achieve better clustering results. However, the high dimensions, lack of labeling, and redundancy of multiview data certainly affect the clustering effect, posing a challenge to multiview clustering. A clustering algorithm based on multiview feature selection clustering (MFSC), which combines similarity graph learning and unsupervised feature selection, is designed in this study. During the MFSC implementation, local manifold regularization is integrated into similarity graph learning, with the clustering label of similarity graph learning as the standard for unsupervised feature selection. MFSC can retain the characteristics of the clustering label on the premise of maintaining the manifold structure of multiview data. The algorithm is systematically evaluated using benchmark multiview and simulated data. The clustering experiment results prove that the MFSC algorithm is more effective than the traditional algorithm.

Joint learning of data recovering and graph contrastive denoising for incomplete multi-view clustering

Incomplete multi-view clustering is pivotal in machine learning because complex systems are inherently difficult to be fully observed and therefore pose a great challenge to revealing the mechanisms and structure of underlying systems. Current methods are criticized for disregarding features with missing views or inadequately exploring the local structure within incomplete views. To solve these limitations, we propose a novel Unified Graph Contrastive Learning Framework for incomplete multi-view clustering (called UGCF), which jointly learns data restoration, graph contrastive denoising, and clustering. Specifically, UGCF first restores the missing values by exploiting the conserved relations in each view, and the local structure within data is preserved. Second, UGCF removes the heterogeneity of multi-view data by learning an affinity structure of objects for each view and constructs a unified graph for multi-view data by manipulating the topological structure of affinity graphs. To enhance the quality of the features of vertices, graph contrastive learning is executed on the unified graphs by selecting positive and negative samples, which considerably improves the discriminative of features, thereby eliminating noise in data. Finally, UGCF integrates data restoration, graph contrastive denoising, and clustering into an overall objective. Experiments demonstrate the superiority of UGCF over state-of-the-art baselines in incomplete multi-view clustering with various metrics.

Robust knowledge-guided biclustering for multi-omics data.

Biclustering is a useful method for simultaneously grouping samples and features and has been applied across various biomedical data types. However, most existing biclustering methods lack the ability to integratively analyze multi-modal data such as multi-omics data such as genome, transcriptome and epigenome. Moreover, the potential of leveraging biological knowledge represented by graphs, which has been demonstrated to be beneficial in various statistical tasks such as variable selection and prediction, remains largely untapped in the context of biclustering. To address both, we propose a novel Bayesian biclustering method called Bayesian graph-guided biclustering (BGB). Specifically, we introduce a new hierarchical sparsity-inducing prior to effectively incorporate biological graph information and establish a unified framework to model multi-view data. We develop an efficient Markov chain Monte Carlo algorithm to conduct posterior sampling and inference. Extensive simulations and real data analysis show that BGB outperforms other popular biclustering methods. Notably, BGB is robust in terms of utilizing biological knowledge and has the capability to reveal biologically meaningful information from heterogeneous multi-modal data.

Magnetic tile fault detection of high‐voltage electric machines: A multiview feature selection method via consistent soft label

AbstractImage‐based scheme has attracted wide attention in the fault detection of high‐voltage permanent magnet motors, but it often suffers from the shooting conditions. Multiview feature selection allows multisource information to be fused, which can improve the accuracy and robustness of image detection. Therefore, we propose multiview unsupervised consistency via soft‐label feature selection (MUCSFS). This method constructs consistent pseudo labels through soft labels from clustering affinity of each view sample and builds the model by integrating selection constraints into the graph model. This model filters the fault data set to obtain the feature subset, which is used for clustering. We verify the method's effectiveness by simulating multiview data. The fault clustering experiment on the high‐voltage motors' magnetic tile fault data set confirm that our method can effectively cluster the fault categories.

Two-step multi-view data classification based on dynamic Graph-ELM

This paper focuses on the classification problem of multi-view data, aiming to improve the classification accuracy of current algorithms on multi-view data. Previous multi-view classification algorithms are usually based on exploiting the complementarity of different views and fusing features from different views. A representative category is the graph-based method, which builds a graph matrix for each view, and then fuses the graph matrices of different views to obtain a unified graph. These methods have the following problems: firstly, the graph matrix is simply based on sample similarity usually; secondly, the learned graph matrix does not change dynamically; thirdly, the weight of the graph representation matrix for a single view cannot be learned in the unified graph matrix. Therefore, this paper designs a Two-step classification algorithm based on Dynamic Graph-ELM, called TSDGELM. In the TSDGELM, the dynamic Graph-ELM is used to obtain the graph representation matrix of each view to save the local neighbor information of the data, and then a joint graph learning algorithm is designed based on the GBS (Graph-Based System) mechanism to fuse the graph matrix of the single-view, and finally the united graph is input into the classifier. To evaluate the effectiveness of the proposed method in this work, we conduct a series of experiments on eight datasets, and the results demonstrate the superiority of the proposed method.

Semi-supervised multi-view concept decomposition

Concept Factorization (CF), as a novel paradigm of representation learning, has demonstrated superior performance in multi-view clustering tasks. It overcomes limitations such as the non-negativity constraint imposed by traditional matrix factorization methods and leverages kernel methods to learn latent representations that capture the underlying structure of the data, thereby improving data representation. However, existing multi-view concept factorization methods fail to consider the limited labeled information inherent in real-world multi-view data. This often leads to significant performance loss. To overcome these limitations, we propose a novel semi-supervised multi-view concept factorization model, named SMVCF. In the SMVCF model, we first extend the conventional single-view CF to a multi-view version, enabling more effective exploration of complementary information across multiple views. We then integrate multi-view CF, label propagation, and manifold learning into a unified framework to leverage and incorporate valuable information present in the data. Additionally, an adaptive weight vector is introduced to balance the importance of different views in the clustering process. We further develop targeted optimization methods specifically tailored for the SMVCF model. Finally, extensive experiments on four diverse datasets with varying label ratios demonstrated the effectiveness and superiority of the proposed SMVCF in multi-view clustering tasks.

DMVSVDD: Multi-View Data Novelty Detection with Deep Autoencoding Support Vector Data Description

Novelty detection is usually defined as the identification of new or abnormal objects (outliers) from the normal ones (inliers), which has wide potential applications in the real world. Recently, an effective algorithm called Deep Support Vector Data Description (Deep SVDD) has been proposed for novelty detection, which jointly trains a deep neural network while optimizing a data-enclosing hypersphere in output space. However, some constraints such as hypersphere collapse, limit the adaptability of the model and may affect the model performance. Moreover, most of the existing studies concerning Deep SVDD focus on the novelty detection for single-view data, which may fail to provide an accurate and reliable decision because the single-view data sometimes cannot fully reflect the actual condition of the problem. In this study, we developed an end-to-end deep learning method of novelty detection for multi-view data, i.e., the Deep Multi-View SVDD (DMVSVDD). To fully preserve the correlative and complementary information of multi-view data, we jointly trained multiple deep autoencoding neural networks for multiple views while adaptively optimizing the data-enclosing hypersphere of each view in latent space. A global objective function was proposed, which takes both of the sample reconstruction error minimization and the hypersphere volume minimization into consideration simultaneously to prevent hypersphere collapse in the model. In the global objective function, the hypersphere centers and view weights of different views were designed to adaptively select the better representative features after each epoch during the training process by embedding multi-view target samples into multiple data-enclosing hyperspheres with minimum volumes. The experimental results on the MNIST-USPS and NUS-WIDE-OBJECT datasets reveal that our proposed method learns the target class effectively and is superior to some state-of-the-art methods.

Multi-view clustering using a flexible and optimal multi-graph fusion method

Recently, many multi-view clustering (MVC) methods based on graphs have been proposed to address prevalent multi-view data. For these methods, the multi-graph fusion step, aim of which is to obtain a consensus graph is vital for obtaining good clustering performance. However, these methods still have two problems. First, the column sum of the consensus graph is restricted to one, which is not flexible enough for practical applications. Affected by the sum-to-one constraint, the similarity between some noisy points may be large, which may cause these abnormal points to be grouped into a separate cluster. Second, the cluster structure of the consensus graph is often not considered. To address these problems, in this paper, we propose a novel multi-view clustering using a flexible and optimal multi-graph fusion method (MVC/FOMF). Specifically, we first obtain the similarity graph of each view by using red the self-expressive method. Second, we fuse these graphs into a consensus graph whose column sum is constrained to s(0<s≤1), and we can adjust s to look for the best clustering performance. Third, we impose a rank constraint on the Laplacian matrix of the consensus graph to learn the best clustering structure. Finally, all these steps are unified into a framework and the corresponding optimization procedure, which is based on the alternating multiplier method, is also designed. More importantly, the complexity of our algorithm is also lower than those of many representative algorithms. Compared with that of the state-of-the-art algorithms, our algorithm shows very encouraging performance on some datasets. The code can be found at https://github.com/wulala2233/MVCFOMF.

Multi-view unsupervised complementary feature selection with multi-order similarity learning

Graph construction, an open and challenging problem, is of great significance for multi-view unsupervised feature selection. So far, many graph construction methods, such as distance-based (the local structure) and self-reconstruction-based (the global structure), have been devised to serve the feature selection task. Although these methods have achieved some improvements, they fail to utilize high-order neighbor information, let alone exploit the neighbor information of different orders, to improve the feature selection task. In this paper, we propose a new insight to construct graphs that can accommodate multi-order neighbor information for selecting the relevant features. Besides, we observe that existing methods adopts the general information fusion strategy in multi-view learning, e.g. fusing graphs, without taking into account the unique characteristics of the feature selection task. Therefore, the proposed method seeks to project multi-view data onto a shared latent representation, which explores the complementarity tailored to the feature selection task at the feature level. A simple yet effective algorithm is designed to solve the optimization problem of the objective function. Extensive clustering experiments demonstrate the superiority of our method over state-of-the-art ones.

A structural consensus representation learning framework for multi-view clustering

Learning a structural consensus representation is crucial for various multi-view tasks, such as multi-view clustering, multi-view classification, etc. However, this task is challenging due to the inconsistent structure problem caused by the view-level bias of multi-view data. To address this issue, we propose a structural consensus representation learning (SCRL) framework, which contains two cascading representation training processes, to learn and refine structural consensus representation. A Consensual Joint Multi-AutoEncoder is developed which estimates a consensual structure shared among all views and learns each view representation guided by the consensual structure in a unique process. By applying EM (Expectation–Maximization)-style optimization, the view representations and the consensual structure are optimized iteratively. Then, we devise a Hybrid Contrastive Refining Net, which contains two contrastive refining components to fine-tune the learnt representations by further eliminating inconsistent view representations within the same view and from different views. The proposed SCRL framework is able to debias the learning of view representations and provide structural consensus representations for multi-view clustering. Extensive experiments and analysis on several real datasets show the effectiveness of our proposed SCRL framework.

Revisiting multi-view learning: A perspective of implicitly heterogeneous Graph Convolutional Network

Graph Convolutional Network (GCN) has become a hotspot in graph-based machine learning due to its powerful graph processing capability. Most of the existing GCN-based approaches are designed for single-view data. In numerous practical scenarios, data is expressed through multiple views, rather than a single view. The ability of GCN to model homogeneous graphs is indisputable, while it is insufficient in facing the heterophily property of multi-view data. In this paper, we revisit multi-view learning to propose an implicit heterogeneous graph convolutional network that efficiently captures the heterogeneity of multi-view data while exploiting the powerful feature aggregation capability of GCN. We automatically assign optimal importance to each view when constructing the meta-path graph. High-order cross-view meta-paths are explored based on the obtained graph, and a series of graph matrices are generated. Combining graph matrices with learnable global feature representation to obtain heterogeneous graph embeddings at various levels. Finally, in order to effectively utilize both local and global information, we introduce a graph-level attention mechanism at the meta-path level that allocates private information to each node individually. Extensive experimental results convincingly support the superior performance of the proposed method compared to other state-of-the-art approaches.

Accelerated Partially Shared Dictionary Learning With Differentiable Scale-Invariant Sparsity for Multi-View Clustering.

Multiview dictionary learning (DL) is attracting attention in multiview clustering due to the efficient feature learning ability. However, most existing multiview DL algorithms are facing problems in fully utilizing consistent and complementary information simultaneously in the multiview data and learning the most precise representation for multiview clustering because of gaps between views. This article proposes an efficient multiview DL algorithm for multiview clustering, which uses the partially shared DL model with a flexible ratio of shared sparse coefficients to excavate both consistency and complementarity in the multiview data. In particular, a differentiable scale-invariant function is used as the sparsity regularizer, which considers the absolute sparsity of coefficients as the l0 norm regularizer but is continuous and differentiable almost everywhere. The corresponding optimization problem is solved by the proximal splitting method with extrapolation technology; moreover, the proximal operator of the differentiable scale-invariant regularizer can be derived. The synthetic experiment results demonstrate that the proposed algorithm can recover the synthetic dictionary well with reasonable convergence time costs. Multiview clustering experiments include six real-world multiview datasets, and the performances show that the proposed algorithm is not sensitive to the regularizer parameter as the other algorithms. Furthermore, an appropriate coefficient sharing ratio can help to exploit consistent information while keeping complementary information from multiview data and thus enhance performances in multiview clustering. In addition, the convergence performances show that the proposed algorithm can obtain the best performances in multiview clustering among compared algorithms and can converge faster than compared multiview algorithms mostly.