Multi-view Learning Research Articles

Keywords-enhanced Contrastive Learning Model for travel recommendation

Travel recommendation aims to infer travel intentions of users by analyzing their historical behaviors on Online Travel Agencies (OTAs). However, crucial keywords in clicked travel product titles, such as destination and itinerary duration, indicating tourists’ intentions, are often overlooked. Additionally, most previous studies only consider stable long-term user interests or temporary short-term user preferences, making the recommendation performance unreliable. To mitigate these constraints, this paper proposes a novel Keywords-enhanced Contrastive Learning Model (KCLM). KCLM simultaneously implements personalized travel recommendation and keywords generation tasks, integrating long-term and short-term user preferences within both tasks. Furthermore, we design two kinds of contrastive learning tasks for better user and travel product representation learning. The preference contrastive learning aims to bridge the gap between long-term and short-term user preferences. The multi-view contrastive learning focuses on modeling the coarse-grained commonality between clicked products and their keywords. Extensive experiments are conducted on two tourism datasets and a large-scale e-commerce dataset. The experimental results demonstrate that KCLM achieves substantial gains in both metrics compared to the best-performing baseline methods. Specifically, HR@20 improved by 5.79%–14.13%, MRR@20 improved by 6.57%–18.50%. Furthermore, to have an intuitive understanding of the keyword generation by the KCLM model, we provide a case study for several randomized examples.

Deep dual incomplete multi-view multi-label classification via label semantic-guided contrastive learning

Multi-view multi-label learning (MVML) aims to train a model that can explore the multi-view information of the input sample to obtain its accurate predictions of multiple labels. Unfortunately, a majority of existing MVML methods are based on the assumption of data completeness, making them useless in practical applications with partially missing views or some uncertain labels. Recently, many approaches have been proposed for incomplete data, but few of them can handle the case of both missing views and labels. Moreover, these few existing works commonly ignore potentially valuable information about unknown labels or do not sufficiently explore latent label information. Therefore, in this paper, we propose a label semantic-guided contrastive learning method named LSGC for the dual incomplete multi-view multi-label classification problem. Concretely, LSGC employs deep neural networks to extract high-level features of samples. Inspired by the observation of exploiting label correlations to improve the feature discriminability, we introduce a graph convolutional network to effectively capture label semantics. Furthermore, we introduce a new sample-label contrastive loss to explore the label semantic information and enhance the feature representation learning. For missing labels, we adopt a pseudo-label filling strategy and develop a weighting mechanism to explore the confidently recovered label information. We validate the framework on five standard datasets and the experimental results show that our method achieves superior performance in comparison with the state-of-the-art methods.

TABLE: Time-aware Balanced Multi-view Learning for stock ranking

Stock ranking is a significant and challenging problem. In recent years, the use of multi-view data, such as price and tweet, for stock ranking has gained considerable attention in the research field. Most existing methods are performed in (some of) the 3 steps: 1) view-specific representation learning; 2) cross-view representation interaction; 3) multi-view representation fusion. Although these methods make breakthroughs in stock ranking, they often treat all views equally. This neglects the unbalanced phenomenon in multi-view stock data, i.e., the dimension of the text view may be extremely big compared with those of other views; the price view exhibits standard and high-quality data, whereas the text view contains noise and has irregular time intervals. To solve this, we propose a Time-Aware Balanced multi-view LEarning (TABLE) method. TABLE method consists of a view-specific learning stage and a multi-view fusion stage. In the first stage, we aim to improve the quality of the low-quality text view. We achieve this by attenuating the negative impact of irrelevant texts using a hierarchical temporal attention mechanism that captures text correlations. Additionally, we explicitly model the time irregularities between sequential texts. In the fusion stage, we address the dimensions unbalance problem by establishing a multi-view decision fusion paradigm by weighted averaging the view-specific stock predictions. These weights are dynamic and determined based on the quality discrepancy between the views. Finally, we obtain the optimal stock ranking list by optimizing the point-wise regression loss and the ranking-aware loss. We empirically compare TABLE method with state-of-the-art baselines using the publicly available dataset, S&P500. The experimental results demonstrate that TABLE method outperforms the baseline methods in terms of accuracy and investment revenue.

A multi-view ensemble machine learning approach for 3D modeling using geological and geophysical data

Geophysical data are often integrated into geological data for 3D modeling of underground spaces. However, the existing single-view approach means it is difficult to adequately fuse the valid information between the two types of data, and the complexity of lithological decoding and classification is high. To address this issue, a multi-view ensemble machine learning (ML) framework is proposed. Initially, the original dataset of lithology prediction is constructed by aligning geological and geophysical data with different spatial scales. Next, the dataset is divided into three datasets of structural strength, density, and moisture content according to the lithology properties of the geophysical data. The proposed framework is then used to capture the lithologic characteristics under different views to achieve the prediction of lithologic labels. In this process, a self-attentive mechanism is used to adaptively fuse the valid information under each view. To validate the proposed framework, it is applied to a project in Jiaxing, Zhejiang Province, China. Compared with existing ML methods, the proposed multi-view ensemble ML framework improves modeling accuracy and constructs models with low uncertainty. The framework can be extended to other multi-source data fusion tasks across geoscience domains.

Deep spatial–spectral difference network with heterogeneous feature mutual learning for sea fog detection

Multispectral remote sensing image-based sea fog detection (SFD) is both important and challenging. Deep learning methods for SFD have become mainstream due to their powerful nonlinear learning capabilities and flexibility. However, existing methods have not fully utilized the physical difference priors in multispectral images (MSI) for SFD, making it difficult to skillfully capture the shape and appearance characteristics of sea fog, thus leading to uncertainties in SFD. We propose the spatial–spectral difference network (S2DNet), a deep encoding–decoding framework that merges inter-spectral and intra-spectral heterogeneous difference features. Specifically, inspired by physics-based difference threshold methods, we developed a physics-inspired inter-spectral difference module (PIDM) that combines feature-level difference with deep neural networks to capture the shape characteristics of sea fog. We designed the intra-spectral difference module (ISDM) using difference convolution to represent sea fog’s fine-grained and dynamic appearance information. Furthermore, inspired by multi-view learning, we propose heterogeneous feature mutual learning (HFML) that seeks robust representations by focusing on semantically invariant aspects within heterogeneous difference features, adapting to the dynamic nature of sea fog. HFML is achieved through global feature mutual learning using an adversarial procedure and local feature mutual learning supported by a novel information-theoretic objective function that links maximizing statistical correlation with expectation maximization. Experiments on two SFD datasets show that integrating physical difference priors into deep learning improves SFD. In both continuous temporal and high spatial resolution SFD tasks, S2DNet outperforms existing advanced deep learning methods. Moreover, S2DNet demonstrates stronger robustness under degraded remote sensing image conditions, highlighting its potential usefulness and practicality in real-world applications.

Attention-based stackable graph convolutional network for multi-view learning

In multi-view learning, graph-based methods like Graph Convolutional Network (GCN) are extensively researched due to effective graph processing capabilities. However, most GCN-based methods often require complex preliminary operations such as sparsification, which may bring additional computation costs and training difficulties. Additionally, as the number of stacking layers increases in most GCN, over-smoothing problem arises, resulting in ineffective utilization of GCN capabilities. In this paper, we propose an attention-based stackable graph convolutional network that captures consistency across views and combines attention mechanism to exploit the powerful aggregation capability of GCN to effectively mitigate over-smoothing. Specifically, we introduce node self-attention to establish dynamic connections between nodes and generate view-specific representations. To maintain cross-view consistency, a data-driven approach is devised to assign attention weights to views, forming a common representation. Finally, based on residual connectivity, we apply an attention mechanism to the original projection features to generate layer-specific complementarity, which compensates for the information loss during graph convolution. Comprehensive experimental results demonstrate that the proposed method outperforms other state-of-the-art methods in multi-view semi-supervised tasks.

A novel multi-view contrastive learning for herb recommendation

A novel multi-view contrastive learning for herb recommendation

MOSDNET: A multi-omics classification framework using simplified multi-view deep discriminant representation learning and dynamic edge GCN with multi-task learning

The integration of multi-omics data offers a robust approach to understanding the complexity of diseases by combining information from various biological levels, such as genomics, transcriptomics, proteomics, and metabolomics. This integrated approach is essential for a comprehensive understanding of disease mechanisms and for developing more effective diagnostic and therapeutic strategies. Nevertheless, most current methodologies fail to effectively extract both shared and specific representations from omics data. This study introduces MOSDNET, a multi-omics classification framework that effectively extracts shared and specific representations. This framework leverages Simplified Multi-view Deep Discriminant Representation Learning (S-MDDR) and Dynamic Edge GCN (DEGCN) to enhance the accuracy and efficiency of disease classification. Initially, MOSDNET utilizes S-MDDR to establish similarity and orthogonal constraints for extracting these representations, which are then concatenated to integrate the multi-omics data. Subsequently, MOSDNET constructs a comprehensive view of the sample data by employing patient similarity networks. By incorporating similarity networks into DEGCN, MOSDNET learns intricate network structures and node representations, which enables superior classification outcomes. MOSDNET is trained through a multitask learning approach, effectively leveraging the complementary knowledge from both the data integration and classification components. After conducting extensive comparative experiments, we have conclusively demonstrated that MOSDNET outperforms leading state-of-the-art multi-omics classification models in terms of classification accuracy. Simultaneously, we employ MOSDNET to identify pivotal biomarkers within the multi-omics data, providing insights into disease etiology and progression.

A Multi-View Deep Learning Approach for Enhanced Student Academic Performance Prediction

A Multi-View Deep Learning Approach for Enhanced Student Academic Performance Prediction

Embedded feature fusion for multi-view multi-label feature selection

With the explosive growth of data sources, multi-view multi-label learning (MVML) has garnered significant attention. However, the task of selecting informative features in MVML becomes more challenging as the dimensionality increase. Existing methods often extract information separately from the consensus part and the complementary part, potentially leading to noise attributed to ambiguous segmentation. In this paper, we propose an embedded feature selection model that combines with two aspects, which are the feature fusion between views and feature enhancement. Firstly, we calculate the adaptive weight of each view based on the local structure relations, and integrate it into one unified feature matrix. Subsequently, the mapping between unified feature matrix and ground-truth label matrix is established. Furthermore, a regularizer for the feature weight of each view is constructed to emphasize its characteristic, respectively. As a result, the relationship for inter-view and intra-view has been simultaneously considered, preserving comprehensive information of features by minimizing the difference between two types of feature weight. Experimental results demonstrate the superior performance of our method in coping with feature selection.

Advancing mRNA subcellular localization prediction with graph neural network and RNA structure.

The asymmetrical distribution of expressed mRNAs tightly controls the precise synthesis of proteins within human cells. This non-uniform distribution, a cornerstone of developmental biology, plays a pivotal role in numerous cellular processes. To advance our comprehension of gene regulatory networks, it is essential to develop computational tools for accurately identifying the subcellular localizations of mRNAs. However, considering multi-localization phenomena remains limited in existing approaches, with none considering the influence of RNA's secondary structure. In this study, we propose Allocator, a multi-view parallel deep learning framework that seamlessly integrates the RNA sequence-level and structure-level information, enhancing the prediction of mRNA multi-localization. The Allocator models equip four efficient feature extractors, each designed to handle different inputs. Two are tailored for sequence-based inputs, incorporating multilayer perceptron and multi-head self-attention mechanisms. The other two are specialized in processing structure-based inputs, employing graph neural networks. Benchmarking results underscore Allocator's superiority over state-of-the-art methods, showcasing its strength in revealing intricate localization associations. The webserver of Allocator is available at http://Allocator.unimelb-biotools.cloud.edu.au; the source code and datasets are available on GitHub (https://github.com/lifuyi774/Allocator) and Zenodo (https://doi.org/10.5281/zenodo.13235798).

A Framework of Multi-View Machine Learning for Biological Spectral Unmixing of Fluorophores with Overlapping Excitation and Emission Spectra.

The accuracy in assigning fluorophore identity and abundance, termed spectral unmixing, in biological fluorescence microscopy images remains challenging due to the unavoidable and significant overlap in emission spectra among fluorophores. In conventional laser scanning confocal spectral microscopy, fluorophore information is acquired by recording emission spectra with a single combination of discrete excitation wavelengths. As a matter of fact, organic fluorophores have not only unique emission spectral signatures but also have unique and characteristic excitation spectra. In this paper, we propose a generalized multi-view machine learning approach, which makes use of both excitation and emission spectra to greatly improve the accuracy in differentiating multiple highly overlapping fluorophores in a single image. By recording emission spectra of the same field with multiple combinations of excitation wavelengths, we obtain data representing these different views of the underlying fluorophore distribution in the sample. We then propose a framework of multi-view machine learning methods, which allows us to flexibly incorporate noise information and abundance constraints, to extract the spectral signatures of fluorophores from their reference images and to efficiently recover their corresponding abundances in unknown mixed images. Numerical experiments on simulated image data demonstrate the method's efficacy in improving accuracy, allowing for the discrimination of 100 fluorophores with highly overlapping spectra. Furthermore, validation on images of mixtures of fluorescently labeled E. coli demonstrates the power of the proposed multi-view strategy in discriminating fluorophores with spectral overlap in real biological images.

Partially shared federated multiview learning

With the diversification of data representations, the field of multiview learning has attracted the attention of many researchers. However, multiview data is stored in different devices in practical applications, and existing multiview learning methods fail to apply directly. Moreover, the data transmission process is often accompanied by the risk of privacy leakage. Therefore, this paper proposes a multiview learning method based on a vertical federated learning framework called partially shared federated multiview learning method (PSFedML) to handle this issue. Our proposed method uses a partial sharing approach that can simultaneously utilize consistent and complementary information between views to learn latent representations. In addition, the transmission of common latent representations between local clients and the central server avoids the risk of privacy leakage. We also give a corresponding iterative optimization algorithm. Experiments on real-world datasets demonstrate the efficiency and superiority of PSFedML.

Deep Double Incomplete Multi-View Multi-Label Learning With Incomplete Labels and Missing Views.

View missing and label missing are two challenging problems in the applications of multi-view multi-label classification scenery. In the past years, many efforts have been made to address the incomplete multi-view learning or incomplete multi-label learning problem. However, few works can simultaneously handle the challenging case with both the incomplete issues. In this article, we propose a new incomplete multi-view multi-label learning network to address this challenging issue. The proposed method is composed of four major parts: view-specific deep feature extraction network, weighted representation fusion module, classification module, and view-specific deep decoder network. By, respectively, integrating the view missing information and label missing information into the weighted fusion module and classification module, the proposed method can effectively reduce the negative influence caused by two such incomplete issues and sufficiently explore the available data and label information to obtain the most discriminative feature extractor and classifier. Furthermore, our method can be trained in both supervised and semi-supervised manners, which has important implications for flexible deployment. Experimental results on five benchmarks in supervised and semi-supervised cases demonstrate that the proposed method can greatly enhance the classification performance on the difficult incomplete multi-view multi-label classification tasks with missing labels and missing views.

Consistent and specific multi-view multi-label learning with correlation information

In multi-view multi-label (MVML) learning, each sample is represented by several heterogeneous distinct feature representations while associated with a set of class labels simultaneously. To achieve MVML learning, most of the existing methods contribute to the recovery of a consistent subspace, i.e., a shared feature representation, among multiple views. Nevertheless, each view has its inherent specific properties used in the discrimination process of labels. These methods lose sight in the specific information exploitation, and therefore are easily trapped in a sub-optimal result. In this study, we present an optimization framework CSVL to solve the learning problem. The main technical contribution in CSVL is a formulation for MVML learning while consistent subspace across views, specific subspace for each view, and the correlations among labels are taken into account. Specifically, consistent subspace is recovered by imposing a low-rank constraint among multiple views, and specific subspace of each view is extra generated with Frobenius norm. To further improve model generalization capability, we preserve both feature manifolds from multiple views and label correlations from multiple labels. Extensive experiments on 7 benchmark datasets show that our proposal CSVL has the advantages in MVML learning.

Multi-Objective Software Defect Prediction via Multi-Source Uncertain Information Fusion and Multi-Task Multi-View Learning

Multi-Objective Software Defect Prediction via Multi-Source Uncertain Information Fusion and Multi-Task Multi-View Learning

Multi-View Fusion Network-Based Gesture Recognition Using sEMG Data.

sEMG(surface electromyography) signals have been widely used in rehabilitation medicine in the past decades because of their non-invasive, convenient and informative features, especially in human action recognition, which has developed rapidly. However, the research on sparse EMG in multi-view fusion has made less progress compared to high-density EMG signals, and for the problem of how to enrich sparse EMG feature information, a method that can effectively reduce the information loss of feature signals in the channel dimension is needed. In this article, a novel IMSE (Inception-MaxPooling-Squeeze- Excitation) network module is proposed to reduce the loss of feature information during deep learning. Then, multiple feature encoders are constructed to enrich the information of sparse sEMG feature maps based on the multi-core parallel processing method in multi-view fusion networks, while SwT (Swin Transformer) is used as the classification backbone network. By comparing the feature fusion effects of different decision layers of the multi-view fusion network, it is experimentally obtained that the fusion of decision layers can better improve the classification performance of the network. In NinaPro DB1, the proposed network achieves 93.96% average accuracy in gesture action classification with the feature maps obtained in 300ms time window, and the maximum variation range of action recognition rate of individuals is less than 11.2%. The results show that the proposed framework of multi-view learning plays a good role in reducing individuality differences and augmenting channel feature information, which provides a certain reference for non-dense biosignal pattern recognition.

Evidential dissonance measure in robust multi-view classification to resist adversarial attack

Multi-view learning is effective in improving data classification accuracy through integrating information from multiple sources. To guarantee the reliability of multi-view classification, the trusted multi-view classification methods have been explored. However, extant trusted multi-view classification methods are still vulnerable to the low-quality views containing adversarial samples. This vulnerability arises from a challenge in accurately assessing the quality of data views that are subjected to adversarial attacks. To address this issue, we propose a robust multi-view classification method with dissonance measure of adversarial samples. Specifically, the proposed method utilizes the evidential dissonance measure in subjective logic to assess the quality of data views when encountering adversarial attacks. Based on the dissonance measure, we further propose a strategy of dissonance-aware belief integration for multi-view information fusion and construct the inter-view evidential gradient penalty in the learning objective of multi-view classification to improve the model robustness to resist adversarial samples. Experiments on diverse multi-view datasets confirm the reliability and robustness of the proposed multi-view classification method.

An unsupervised multi-view contrastive learning framework with attention-based reranking strategy for entity alignment

Entity alignment is a crucial task in knowledge graphs, aiming to match corresponding entities from different knowledge graphs. Due to the scarcity of pre-aligned entities in real-world scenarios, research focused on unsupervised entity alignment has become more popular. However, current unsupervised entity alignment methods suffer from a lack of informative entity guidance, hindering their ability to accurately predict challenging entities with similar names and structures. To solve these problems, we present an unsupervised multi-view contrastive learning framework with an attention-based reranking strategy for entity alignment, named AR-Align. In AR-Align, two kinds of data augmentation methods are employed to provide a complementary view for neighborhood and attribute, respectively. Next, a multi-view contrastive learning method is introduced to reduce the semantic gap between different views of the augmented entities. Moreover, an attention-based reranking strategy is proposed to rerank the hard entities through calculating their weighted sum of embedding similarities on different structures. Experimental results indicate that AR-Align outperforms most both supervised and unsupervised state-of-the-art methods on three benchmark datasets.

Incomplete multi-view partial multi-label classification via deep semantic structure preservation

Recent advances in multi-view multi-label learning are often hampered by the prevalent challenges of incomplete views and missing labels, common in real-world data due to uncertainties in data collection and manual annotation. These challenges restrict the capacity of the model to fully utilize the diverse semantic information of each sample, posing significant barriers to effective learning. Despite substantial scholarly efforts, many existing methods inadequately capture the depth of semantic information, focusing primarily on shallow feature extractions that fail to maintain semantic consistency. To address these shortcomings, we propose a novel Deep semantic structure-preserving (SSP) model that effectively tackles both incomplete views and missing labels. SSP innovatively incorporates a graph constraint learning (GCL) scheme to ensure the preservation of semantic structure throughout the feature extraction process across different views. Additionally, the SSP integrates a pseudo-labeling self-paced learning (PSL) strategy to address the often-overlooked issue of missing labels, enhancing the classification accuracy while preserving the distribution structure of data. The SSP model creates a unified framework that synergistically employs GCL and PSL to maintain the integrity of semantic structural information during both feature extraction and classification phases. Extensive evaluations across five real datasets demonstrate that the SSP method outperforms existing approaches, including lrMMC, MVL-IV, MvEL, iMSF, iMvWL, NAIML, and DD-IMvMLC-net. It effectively mitigates the impacts of data incompleteness and enhances semantic representation fidelity.