Multi-view Learning Framework Research Articles

Multi-view graph contrastive learning for social recommendation.

With the rapid popularity of online social media, recommendation systems have increasingly harnessed social relations to enhance user-item interactions and mitigate the data sparsity issue. Beyond social connections, the semantic relatedness among items has emerged as a crucial factor in comprehending their inherent connections. In this work, we propose a novel Multi-view Contrastive learning framework for Social Recommendation, named MultiCSR. This framework adaptively incorporates user social networks and item knowledge graphs into modeling users preferences within recommendation systems. To facilitate the alignment of different views, we introduce a dedicated multi-view contrastive learning process that extracts rich information from each view and foster mutual enhancement. Extensive experiments conducted on three real-world datasets demonstrate the effectiveness of our framework over representative recommendation methods. Furthermore, ablation studies offer a deeper understanding of the mechanisms underlying our framework.

Multi-view learning framework for predicting unknown types of cancer markers via directed graph neural networks fitting regulatory networks.

The discovery of diagnostic and therapeutic biomarkers for complex diseases, especially cancer, has always been a central and long-term challenge in molecular association prediction research, offering promising avenues for advancing the understanding of complex diseases. To this end, researchers have developed various network-based prediction techniques targeting specific molecular associations. However, limitations imposed by reductionism and network representation learning have led existing studies to narrowly focus on high prediction efficiency within single association type, thereby glossing over the discovery of unknown types of associations. Additionally, effectively utilizing network structure to fit the interaction properties of regulatory networks and combining specific case biomarker validations remains an unresolved issue in cancer biomarker prediction methods. To overcome these limitations, we propose a multi-view learning framework, CeRVE, based on directed graph neural networks (DGNN) for predicting unknown type cancer biomarkers. CeRVE effectively extracts and integrates subgraph information through multi-view feature learning. Subsequently, CeRVE utilizes DGNN to simulate the entire regulatory network, propagating node attribute features and extracting various interaction relationships between molecules. Furthermore, CeRVE constructed a comparative analysis matrix of three cancers and adjacent normal tissues through The Cancer Genome Atlas and identified multiple types of potential cancer biomarkers through differential expression analysis of mRNA, microRNA, and long noncoding RNA. Computational testing of multiple types of biomarkers for 72 cancers demonstrates that CeRVE exhibits superior performance in cancer biomarker prediction, providing a powerful tool and insightful approach for AI-assisted disease biomarker discovery.

A Multi-view Graph Contrastive Learning Framework for Cross-Domain Sequential Recommendation

A Multi-view Graph Contrastive Learning Framework for Cross-Domain Sequential Recommendation

MCMVDRP: a multi-channel multi-view deep learning framework for cancer drug response prediction.

Drug therapy remains the primary approach to treating tumours. Variability among cancer patients, including variations in genomic profiles, often results in divergent therapeutic responses to analogous anti-cancer drug treatments within the same cohort of cancer patients. Hence, predicting the drug response by analysing the genomic profile characteristics of individual patients holds significant research importance. With the notable progress in machine learning and deep learning, many effective methods have emerged for predicting drug responses utilizing features from both drugs and cell lines. However, these methods are inadequate in capturing a sufficient number of features inherent to drugs. Consequently, we propose a representational approach for drugs that incorporates three distinct types of features: the molecular graph, the SMILE strings, and the molecular fingerprints. In this study, a novel deep learning model, named MCMVDRP, is introduced for the prediction of cancer drug responses. In our proposed model, an amalgamation of these extracted features is performed, followed by the utilization of fully connected layers to predict the drug response based on the IC50 values. Experimental results demonstrate that the presented model outperforms current state-of-the-art models in performance.

Multi-View Graph Learning for Path-Level Aging-Aware Timing Prediction

As CMOS technology continues to scale down, the aging effect—known as negative bias temperature instability (NBTI)—has become increasingly prominent, gradually emerging as a key factor affecting device reliability. Accurate aging-aware static timing analysis (STA) at the early design phase is critical for establishing appropriate timing margins to ensure circuit reliability throughout the chip lifecycle. However, traditional aging-aware timing analysis methods, typically based on Simulation Program with Integrated Circuit Emphasis (SPICE) simulations or aging-aware timing libraries, struggle to balance prediction accuracy with computational cost. In this paper, we propose a multi-view graph learning framework for path-level aging-aware timing prediction, which combines the strengths of the spatial–temporal Transformer network (STTN) and graph attention network (GAT) models to extract the aging timing features of paths from both timing-sensitive and workload-sensitive perspectives. Experimental results demonstrate that our proposed framework achieves an average MAPE score of 3.96% and reduces the average MAPE by 5.8 times compared to FFNN and 2.2 times compared to PNA, while maintaining acceptable increases in processing time.

Divergence-guided disentanglement of view-common and view-unique representations for multi-view data

In the field of multi-view learning (MVL), it is crucial to extract both common (consistent) and unique (complementary) information across different views. While the focus has traditionally been on acquiring common information, there has been a recent shift towards exploring unique information as well. However, developing an MVL model that can simultaneously capture both common and unique information, thereby facilitating a comprehensive understanding of multi-view data, remains a significant challenge. To address this, we propose the Divergence-guided Multi-view Learning framework (DG-MVL), inspired by information-theoretic learning theory, specifically the generalized divergence measure. This framework employs multi-view autoencoders to disentangle the features obtained from each view into coarse common and unique components. By minimizing the divergence between the coarse common features learned from the common encoder of each view and maximizing the divergence between the coarse unique features from unique encoders simultaneously, we optimize the extraction of both common and unique information. Subsequently, we merge these features to generate a comprehensive and concise representation of the multi-view data, which can be easily utilized for various downstream tasks. We validate our framework on a synthetic multi-view dataset, demonstrating its effectiveness in disentangling common and unique information. Further experiments on various real-world datasets confirm the effectiveness of DG-MVL in capturing common and unique information from multi-view data, resulting in superior classification performance compared to existing methods. Code is available at https://github.com/LMFLRB/DG-MVL.git.

MolMVC: Enhancing molecular representations for drug-related tasks through multi-view contrastive learning.

Effective molecular representation is critical in drug development. The complex nature of molecules demands comprehensive multi-view representations, considering 1D, 2D, and 3D aspects, to capture diverse perspectives. Obtaining representations that encompass these varied structures is crucial for a holistic understanding of molecules in drug-related contexts. In this study, we introduce an innovative multi-view contrastive learning framework for molecular representation, denoted as MolMVC. Initially, we use a Transformer encoder to capture 1D sequence information and a Graph Transformer to encode the intricate 2D and 3D structural details of molecules. Our approach incorporates a novel attention-guided augmentation scheme, leveraging prior knowledge to create positive samples tailored to different molecular data views. To align multi-view molecular positive samples effectively in latent space, we introduce an adaptive multi-view contrastive loss (AMCLoss). In particular, we calculate AMCLoss at various levels within the model to effectively capture the hierarchical nature of the molecular information. Eventually, we pre-train the encoders via minimizing AMCLoss to obtain the molecular representation, which can be used for various down-stream tasks. In our experiments, we evaluate the performance of our MolMVC on multiple tasks, including molecular property prediction (MPP), drug-target binding affinity (DTA) prediction and cancer drug response (CDR) prediction. The results demonstrate that the molecular representation learned by our MolMVC can enhance the predictive accuracy on these tasks and also reduce the computational costs. Furthermore, we showcase MolMVC's efficacy in drug repositioning across a spectrum of drug-related applications. The code and pre-trained model are publicly available at https://github.com/Hhhzj-7/MolMVC.

A multi-view ensemble machine learning approach for 3D modeling using geological and geophysical data

Geophysical data are often integrated into geological data for 3D modeling of underground spaces. However, the existing single-view approach means it is difficult to adequately fuse the valid information between the two types of data, and the complexity of lithological decoding and classification is high. To address this issue, a multi-view ensemble machine learning (ML) framework is proposed. Initially, the original dataset of lithology prediction is constructed by aligning geological and geophysical data with different spatial scales. Next, the dataset is divided into three datasets of structural strength, density, and moisture content according to the lithology properties of the geophysical data. The proposed framework is then used to capture the lithologic characteristics under different views to achieve the prediction of lithologic labels. In this process, a self-attentive mechanism is used to adaptively fuse the valid information under each view. To validate the proposed framework, it is applied to a project in Jiaxing, Zhejiang Province, China. Compared with existing ML methods, the proposed multi-view ensemble ML framework improves modeling accuracy and constructs models with low uncertainty. The framework can be extended to other multi-source data fusion tasks across geoscience domains.

Advancing mRNA subcellular localization prediction with graph neural network and RNA structure.

The asymmetrical distribution of expressed mRNAs tightly controls the precise synthesis of proteins within human cells. This non-uniform distribution, a cornerstone of developmental biology, plays a pivotal role in numerous cellular processes. To advance our comprehension of gene regulatory networks, it is essential to develop computational tools for accurately identifying the subcellular localizations of mRNAs. However, considering multi-localization phenomena remains limited in existing approaches, with none considering the influence of RNA's secondary structure. In this study, we propose Allocator, a multi-view parallel deep learning framework that seamlessly integrates the RNA sequence-level and structure-level information, enhancing the prediction of mRNA multi-localization. The Allocator models equip four efficient feature extractors, each designed to handle different inputs. Two are tailored for sequence-based inputs, incorporating multilayer perceptron and multi-head self-attention mechanisms. The other two are specialized in processing structure-based inputs, employing graph neural networks. Benchmarking results underscore Allocator's superiority over state-of-the-art methods, showcasing its strength in revealing intricate localization associations. The webserver of Allocator is available at http://Allocator.unimelb-biotools.cloud.edu.au; the source code and datasets are available on GitHub (https://github.com/lifuyi774/Allocator) and Zenodo (https://doi.org/10.5281/zenodo.13235798).

A Framework of Multi-View Machine Learning for Biological Spectral Unmixing of Fluorophores with Overlapping Excitation and Emission Spectra.

The accuracy in assigning fluorophore identity and abundance, termed spectral unmixing, in biological fluorescence microscopy images remains challenging due to the unavoidable and significant overlap in emission spectra among fluorophores. In conventional laser scanning confocal spectral microscopy, fluorophore information is acquired by recording emission spectra with a single combination of discrete excitation wavelengths. As a matter of fact, organic fluorophores have not only unique emission spectral signatures but also have unique and characteristic excitation spectra. In this paper, we propose a generalized multi-view machine learning approach, which makes use of both excitation and emission spectra to greatly improve the accuracy in differentiating multiple highly overlapping fluorophores in a single image. By recording emission spectra of the same field with multiple combinations of excitation wavelengths, we obtain data representing these different views of the underlying fluorophore distribution in the sample. We then propose a framework of multi-view machine learning methods, which allows us to flexibly incorporate noise information and abundance constraints, to extract the spectral signatures of fluorophores from their reference images and to efficiently recover their corresponding abundances in unknown mixed images. Numerical experiments on simulated image data demonstrate the method's efficacy in improving accuracy, allowing for the discrimination of 100 fluorophores with highly overlapping spectra. Furthermore, validation on images of mixtures of fluorescently labeled E. coli demonstrates the power of the proposed multi-view strategy in discriminating fluorophores with spectral overlap in real biological images.

Multi-View Fusion Network-Based Gesture Recognition Using sEMG Data.

sEMG(surface electromyography) signals have been widely used in rehabilitation medicine in the past decades because of their non-invasive, convenient and informative features, especially in human action recognition, which has developed rapidly. However, the research on sparse EMG in multi-view fusion has made less progress compared to high-density EMG signals, and for the problem of how to enrich sparse EMG feature information, a method that can effectively reduce the information loss of feature signals in the channel dimension is needed. In this article, a novel IMSE (Inception-MaxPooling-Squeeze- Excitation) network module is proposed to reduce the loss of feature information during deep learning. Then, multiple feature encoders are constructed to enrich the information of sparse sEMG feature maps based on the multi-core parallel processing method in multi-view fusion networks, while SwT (Swin Transformer) is used as the classification backbone network. By comparing the feature fusion effects of different decision layers of the multi-view fusion network, it is experimentally obtained that the fusion of decision layers can better improve the classification performance of the network. In NinaPro DB1, the proposed network achieves 93.96% average accuracy in gesture action classification with the feature maps obtained in 300ms time window, and the maximum variation range of action recognition rate of individuals is less than 11.2%. The results show that the proposed framework of multi-view learning plays a good role in reducing individuality differences and augmenting channel feature information, which provides a certain reference for non-dense biosignal pattern recognition.

An unsupervised multi-view contrastive learning framework with attention-based reranking strategy for entity alignment

Entity alignment is a crucial task in knowledge graphs, aiming to match corresponding entities from different knowledge graphs. Due to the scarcity of pre-aligned entities in real-world scenarios, research focused on unsupervised entity alignment has become more popular. However, current unsupervised entity alignment methods suffer from a lack of informative entity guidance, hindering their ability to accurately predict challenging entities with similar names and structures. To solve these problems, we present an unsupervised multi-view contrastive learning framework with an attention-based reranking strategy for entity alignment, named AR-Align. In AR-Align, two kinds of data augmentation methods are employed to provide a complementary view for neighborhood and attribute, respectively. Next, a multi-view contrastive learning method is introduced to reduce the semantic gap between different views of the augmented entities. Moreover, an attention-based reranking strategy is proposed to rerank the hard entities through calculating their weighted sum of embedding similarities on different structures. Experimental results indicate that AR-Align outperforms most both supervised and unsupervised state-of-the-art methods on three benchmark datasets.

Learning from Feature and Global Topologies: Adaptive Multi-View Parallel Graph Contrastive Learning

To address the limitations of existing graph contrastive learning methods, which fail to adaptively integrate feature and topological information and struggle to efficiently capture multi-hop information, we propose an adaptive multi-view parallel graph contrastive learning framework (AMPGCL). It is an unsupervised graph representation learning method designed to generate task-agnostic node embeddings. AMPGCL constructs and encodes feature and topological views to mine feature and global topological information. To encode global topological information, we introduce an H-Transformer to decouple multi-hop neighbor aggregations, capturing global topology from node subgraphs. AMPGCL learns embedding consistency among feature, topology, and original graph encodings through a multi-view contrastive loss, generating semantically rich embeddings while avoiding information redundancy. Experiments on nine real datasets demonstrate that AMPGCL consistently outperforms thirteen state-of-the-art graph representation learning models in classification accuracy, whether in homophilous or non-homophilous graphs.

Orchard bird song recognition based on multi-view multi-level contrastive learning

In the harvest season, orchards are frequently plagued by birds, and thus significant fruit pecking can adversely affect both the fruit quality and yield. Recognizing bird songs is crucial for preventing damage caused by orchard birds because it can provide the basis for subsequent bird repellent efforts. However, the extensive effort required to annotate sound samples poses a significant challenge for supervised deep learning. In this paper, we propose a self-supervised multi-view learning framework based on multi-level contrasting (MV-MLC) for bird song recognition, which utilizes both time and spectrogram views as inputs. This framework leverages MLC to automatically learn representations from unlabeled data and a multi-scale feature extraction (MSFE) backbone network is employed to capture the temporal features of bird songs at different scales. The time-spectrogram consistency task in MLC learning facilitates semantic-level information exchange across multi-views, while the hierarchical contrastive learning task captures granularity-level information, thereby resulting in more robust contextual representations. In addition, embedding the shuffle attention module in MSFE facilitates mining of the spatial and channel dependencies of bird song features to further enhance the representation of features by the multi-scale network. We conducted extensive experiments using our self-built 10-class bird song data set (Orchard-birds) and the publicly available Birdsdata and Powdermill data sets. The experimental results demonstrated that MV-MLC performed better than state-of-the-art self-supervised models. In particular, MV-MLC obtained outstanding performance even with a small proportion of labeled data. The recognition accuracies based on the Orchard-birds and Birdsdata data sets were 99.40% and 92.67%, respectively, with macro F1-scores of 99.40% and 92.61%.

Multi-view contextual adaptation network for weakly supervised object detection in remote sensing images

ABSTRACT Weakly supervised learning plays a pivotal role in the field of object detection, i.e. Weakly supervised object detection (WSOD), significantly reducing annotation costs relying on image-level labels. However, WSOD exhibits certain limitations. Typically, they tend to identify the most easily recognizable local regions within targets, posing challenges in accurately delineating the boundaries of targets. Moreover, the presence of multiple instances of the same class in adjacent locations complicates the effective distinction between multiple objects within the same category. On the other hand, the complex backgrounds and dense distribution of targets in remote sensing images (RSI) further exacerbate the difficulty of weakly supervised detection. To address the above issues, we propose a model termed the Multi-View Contextual Adaptation Network (VCANet). Building on the classic Online Instance Classifier Refinement (OICR) framework, we propose to incorporate an contextual adaptation perception, within a multi-view learning framework, and integrate a pseudo-label filtering process. The contextual adaptation perception utilizes the surrounding environment information to enhance localization capabilities, guiding the model to prioritize target objects by referring to their spatially neighbouring pixels. Multi-view learning manufactures additional constraints from diverse perspectives, thereby revealing objects that might be overlooked due to the weak supervision in a single view. The pseudo-label filtering process eliminates inaccurate pseudo-labels by identifying reliable foregrounds to mitigate overlapping proposals during the label propagation. On challenging datasets NWPU VHR-10.v2 and DIOR, we achieve promising results with mAP of 62.3% and 28.2%, respectively, surpassing existing benchmarks.

A multi-view graph contrastive learning framework for deciphering spatially resolved transcriptomics data.

Spatially resolved transcriptomics data are being used in a revolutionary way to decipher the spatial pattern of gene expression and the spatial architecture of cell types. Much work has been done to exploit the genomic spatial architectures of cells. Such work is based on the common assumption that gene expression profiles of spatially adjacent spots are more similar than those of more distant spots. However, related work might not consider the nonlocal spatial co-expression dependency, which can better characterize the tissue architectures. Therefore, we propose MuCoST, a Multi-view graph Contrastive learning framework for deciphering complex Spatially resolved Transcriptomic architectures with dual scale structural dependency. To achieve this, we employ spot dependency augmentation by fusing gene expression correlation and spatial location proximity, thereby enabling MuCoST to model both nonlocal spatial co-expression dependency and spatially adjacent dependency. We benchmark MuCoST on four datasets, and we compare it with other state-of-the-art spatial domain identification methods. We demonstrate that MuCoST achieves the highest accuracy on spatial domain identification from various datasets. In particular, MuCoST accurately deciphers subtle biological textures and elaborates the variation of spatially functional patterns.

Liver fibrosis automatic diagnosis utilizing dense-fusion attention contrastive learning network.

Liver fibrosis poses a significant public health challenge given its elevated incidence and associated mortality rates. Diffusion-Weighted Imaging (DWI) serves as a non-invasive diagnostic tool for supporting the identification of liver fibrosis. Deep learning, as a computer-aided diagnostic technology, can assist in recognizing the stage of liver fibrosis by extracting abstract features from DWI images. However, gathering samples is often challenging, posing a common dilemma in previous research. Moreover, previous studies frequently overlooked the cross-comparison information and latent connections among different DWI parameters. Thus, it is becoming a challenge to identify effective DWI parameters and dig potential features from multiple categories in a dataset with limited samples. A self-defined Multi-view Contrastive Learning Network is developed to automatically classify multi-parameter DWI images and explore synergies between different DWI parameters. A Dense-fusion Attention Contrastive Learning Network (DACLN) is designed and used to recognize DWI images. Concretely, a multi-view contrastive learning framework is constructed to train and extract features from raw multi-parameter DWI. Besides, a Dense-fusion module is designed to integrate feature and output predicted labels. We evaluated the performance of the proposed model on a set of real clinical data and analyzed the interpretability by Grad-CAM and annotation analysis, achieving average scores of 0.8825, 0.8702, 0.8933, 0.8727, and 0.8779 for accuracy, precision, recall, specificity and F-1 score. Of note, the experimental results revealed that IVIM-f, CTRW-β, and MONO-ADC exhibited significant recognition ability and complementarity. Our method achieves competitive accuracy in liver fibrosis diagnosis using the limited multi-parameter DWI dataset and finds three types of DWI parameters with high sensitivity for diagnosing liver fibrosis, which suggests potential directions for future research.

A multi-view representation learning framework for commonsense knowledge bases

Commonsense knowledge bases play an essential role in a wide range of natural language processing tasks. This paper studies the problem of representation learning for commonsense knowledge bases to effectively incorporate their knowledge into numerical models. Most existing knowledge base representation learning methods are difficult to apply to commonsense knowledge bases since they are much sparser than general knowledge bases. Hence, in this paper, we propose a novel method for commonsense knowledge base representation learning. Specifically, we first model the nodes from multiple views, including word/phase information, context information, and graph information. Then, we design a scoring function to measure whether the commonsense triplets are established through relation representation learning. We conduct extensive experiments on two tasks and the results show that our proposed model outperforms other knowledge base representation learning methods.

DiSMVC: a multi-view graph collaborative learning framework for measuring disease similarity.

Exploring potential associations between diseases can help in understanding pathological mechanisms of diseases and facilitating the discovery of candidate biomarkers and drug targets, thereby promoting disease diagnosis and treatment. Some computational methods have been proposed for measuring disease similarity. However, these methods describe diseases without considering their latent multi-molecule regulation and valuable supervision signal, resulting in limited biological interpretability and efficiency to capture association patterns. In this study, we propose a new computational method named DiSMVC. Different from existing predictors, DiSMVC designs a supervised graph collaborative framework to measure disease similarity. Multiple bio-entity associations related to genes and miRNAs are integrated via cross-view graph contrastive learning to extract informative disease representation, and then association pattern joint learning is implemented to compute disease similarity by incorporating phenotype-annotated disease associations. The experimental results show that DiSMVC can draw discriminative characteristics for disease pairs, and outperform other state-of-the-art methods. As a result, DiSMVC is a promising method for predicting disease associations with molecular interpretability. Datasets and source codes are available at https://github.com/Biohang/DiSMVC.

An Optimal Edge-weighted Graph Semantic Correlation Framework for Multi-view Feature Representation Learning

In this article, we present an optimal edge-weighted graph semantic correlation (EWGSC) framework for multi-view feature representation learning. Different from most existing multi-view representation methods, local structural information and global correlation in multi-view feature spaces are exploited jointly in the EWGSC framework, leading to a new and high-quality multi-view feature representation. Specifically, a novel edge-weighted graph model is first conceptualized and developed to preserve local structural information in each of the multi-view feature spaces. Then, the explored structural information is integrated with a semantic correlation algorithm, labeled multiple canonical correlation analysis (LMCCA), to form a powerful platform for effectively exploiting local and global relations across multi-view feature spaces jointly. We then theoretically verified the relation between the upper limit on the number of projected dimensions and the optimal solution to the multi-view feature representation problem. To validate the effectiveness and generality of the proposed framework, we conducted experiments on five datasets of different scales, including visual-based (University of California Irvine (UCI) iris database, Olivetti Research Lab (ORL) face database, and Caltech 256 database), text-image-based (Wiki database), and video-based (Ryerson Multimedia Lab (RML) audio-visual emotion database) examples. The experimental results show the superiority of the proposed framework on multi-view feature representation over state-of-the-art algorithms.