Multi-view Learning Research Articles

Multiview representation learning for identification of novel cancer genes and their causative biological mechanisms.

Tumorigenesis arises from the dysfunction of cancer genes, leading to uncontrolled cell proliferation through various mechanisms. Establishing a complete cancer gene catalogue will make precision oncology possible. Although existing methods based on graph neural networks (GNN) are effective in identifying cancer genes, they fall short in effectively integrating data from multiple views and interpreting predictive outcomes. To address these shortcomings, an interpretable representation learning framework IMVRL-GCN is proposed to capture both shared and specific representations from multiview data, offering significant insights into the identification of cancer genes. Experimental results demonstrate that IMVRL-GCN outperforms state-of-the-art cancer gene identification methods and several baselines. Furthermore, IMVRL-GCN is employed to identify a total of 74 high-confidence novel cancer genes, and multiview data analysis highlights the pivotal roles of shared, mutation-specific, and structure-specific representations in discriminating distinctive cancer genes. Exploration of the mechanisms behind their discriminative capabilities suggests that shared representations are strongly associated with gene functions, while mutation-specific and structure-specific representations are linked to mutagenic propensity and functional synergy, respectively. Finally, our in-depth analyses of these candidates suggest potential insights for individualized treatments: afatinib could counteract many mutation-driven risks, and targeting interactions with cancer gene SRC is a reasonable strategy to mitigate interaction-induced risks for NR3C1, RXRA, HNF4A, and SP1.

An integrative multi-context Mendelian randomization method for identifying risk genes across human tissues

Mendelian randomization (MR) provides valuable assessments of the causal effect of exposure on outcome, yet the application of conventional MR methods for mapping risk genes encounters new challenges. One of the issues is the limited availability of expression quantitative trait loci (eQTLs) as instrumental variables (IVs), hampering the estimation of sparse causal effects. Additionally, the often context- or tissue-specific eQTL effects challenge the MR assumption of consistent IV effects across eQTL and GWAS data. To address these challenges, we propose a multi-context multivariable integrative MR framework, mintMR, for mapping expression and molecular traits as joint exposures. It models the effects of molecular exposures across multiple tissues in each gene region, while simultaneously estimating across multiple gene regions. It uses eQTLs with consistent effects across more than one tissue type as IVs, improving IV consistency. A major innovation of mintMR involves employing multi-view learning methods to collectively model latent indicators of disease relevance across multiple tissues, molecular traits, and gene regions. The multi-view learning captures the major patterns of disease relevance and uses these patterns to update the estimated tissue relevance probabilities. The proposed mintMR iterates between performing a multi-tissue MR for each gene region and joint learning the disease-relevant tissue probabilities across gene regions, improving the estimation of sparse effects across genes. We apply mintMR to evaluate the causal effects of gene expression and DNA methylation for 35 complex traits using multi-tissue QTLs as IVs. The proposed mintMR controls genome-wide inflation and offers insights into disease mechanisms.

Multi-view heterogeneous graph learning with compressed hypergraph neural networks

Multi-view learning is an emerging field of multi-modal fusion, which involves representing a single instance using multiple heterogeneous features to improve compatibility prediction. However, existing graph-based multi-view learning approaches are implemented on homogeneous assumptions and pairwise relationships, which may not adequately capture the complex interactions among real-world instances. In this paper, we design a compressed hypergraph neural network from the perspective of multi-view heterogeneous graph learning. This approach effectively captures rich multi-view heterogeneous semantic information, incorporating a hypergraph structure that simultaneously enables the exploration of higher-order correlations between samples in multi-view scenarios. Specifically, we introduce efficient hypergraph convolutional networks based on an explainable regularizer-centered optimization framework. Additionally, a low-rank approximation is adopted as hypergraphs to reformat the initial complex multi-view heterogeneous graph. Extensive experiments compared with several advanced node classification methods and multi-view classification methods have demonstrated the feasibility and effectiveness of the proposed method.

Learning from Feature and Global Topologies: Adaptive Multi-View Parallel Graph Contrastive Learning

To address the limitations of existing graph contrastive learning methods, which fail to adaptively integrate feature and topological information and struggle to efficiently capture multi-hop information, we propose an adaptive multi-view parallel graph contrastive learning framework (AMPGCL). It is an unsupervised graph representation learning method designed to generate task-agnostic node embeddings. AMPGCL constructs and encodes feature and topological views to mine feature and global topological information. To encode global topological information, we introduce an H-Transformer to decouple multi-hop neighbor aggregations, capturing global topology from node subgraphs. AMPGCL learns embedding consistency among feature, topology, and original graph encodings through a multi-view contrastive loss, generating semantically rich embeddings while avoiding information redundancy. Experiments on nine real datasets demonstrate that AMPGCL consistently outperforms thirteen state-of-the-art graph representation learning models in classification accuracy, whether in homophilous or non-homophilous graphs.

Minimum spanning tree clustering approach for effective feature partitioning in multi-view ensemble learning

Minimum spanning tree clustering approach for effective feature partitioning in multi-view ensemble learning

Transferring Adult-like Phase Images for Robust Multi-view Isointense Infant Brain Segmentation.

Accurate tissue segmentation of infant brain in magnetic resonance (MR) images is crucial for charting early brain development and identifying biomarkers. Due to ongoing myelination and maturation, in the isointense phase (6-9 months of age), the gray and white matters of infant brain exhibit similar intensity levels in MR images, posing significant challenges for tissue segmentation. Meanwhile, in the adult-like phase around 12 months of age, the MR images show high tissue contrast and can be easily segmented. In this paper, we propose to effectively exploit adult-like phase images to achieve robustmulti-view isointense infant brain segmentation. Specifically, in one way, we transfer adult-like phase images to the isointense view, which have similar tissue contrast as the isointense phase images, and use the transferred images to train an isointense-view segmentation network. On the other way, we transfer isointense phase images to the adult-like view, which have enhanced tissue contrast, for training a segmentation network in the adult-like view. The segmentation networks of different views form a multi-path architecture that performs multi-view learning to further boost the segmentation performance. Since anatomy-preserving style transfer is key to the downstream segmentation task, we develop a Disentangled Cycle-consistent Adversarial Network (DCAN) with strong regularization terms to accurately transfer realistic tissue contrast between isointense and adult-like phase images while still maintaining their structural consistency. Experiments on both NDAR and iSeg-2019 datasets demonstrate a significant superior performance of our method over the state-of-the-art methods.

Multiview Classification Through Learning From Interval-Valued Data.

The classification problem concerning crisp-valued data has been well resolved. However, interval-valued data, where all of the observations' features are described by intervals, are also a common data type in real-world scenarios. For example, the data extracted by many measuring devices are not exact numbers but intervals. In this article, we focus on a highly challenging problem called learning from interval-valued data (LIND), where we aim to learn a classifier with high performance on interval-valued observations. First, we obtain the estimation error bound of the LIND problem based on the Rademacher complexity. Then, we give the theoretical analysis to show the strengths of multiview learning on classification problems, which inspires us to construct a new algorithm called multiview interval information extraction (Mv-IIE) approach for improving classification accuracy on interval-valued data. The experiment comparisons with several baselines on both synthetic and real-world datasets illustrate the superiority of the proposed framework in handling interval-valued data. Moreover, we describe an application of Mv-IIE that we can prevent data privacy leakage by transforming crisp-valued (raw) data into interval-valued data.

Cyclic Refiner: Object-Aware Temporal Representation Learning for Multi-view 3D Detection and Tracking

AbstractWe propose a unified object-aware temporal learning framework for multi-view 3D detection and tracking tasks. Having observed that the efficacy of the temporal fusion strategy in recent multi-view perception methods may be weakened by distractors and background clutters in historical frames, we propose a cyclic learning mechanism to improve the robustness of multi-view representation learning. The essence is constructing a backward bridge to propagate information from model predictions (e.g., object locations and sizes) to image and BEV features, which forms a circle with regular inference. After backward refinement, the responses of target-irrelevant regions in historical frames would be suppressed, decreasing the risk of polluting future frames and improving the object awareness ability of temporal fusion. We further tailor an object-aware association strategy for tracking based on the cyclic learning model. The cyclic learning model not only provides refined features, but also delivers finer clues (e.g., scale level) for tracklet association. The proposed cycle learning method and association module together contribute a novel and unified multi-task framework. Experiments on nuScenes show that the proposed model achieves consistent performance gains over baselines of different designs (i.e., dense query-based BEVFormer, sparse query-based SparseBEV and LSS-based BEVDet4D) on both detection and tracking evaluation. Codes and models will be released.

Difficult Airway Assessment Based on Multi-View Metric Learning.

The preoperative assessment of difficult airways is of great significance in the practice of anesthesia intubation. In recent years, although a large number of difficult airway recognition algorithms have been investigated, defects such as low recognition accuracy and poor recognition reliability still exist. In this paper, we propose a Dual-Path Multi-View Fusion Network (DMF-Net) based on multi-view metric learning, which aims to predict difficult airways through multi-view facial images of patients. DMF-Net adopts a dual-path structure to extract features by grouping the frontal and lateral images of the patients. Meanwhile, a Multi-Scale Feature Fusion Module and a Hybrid Co-Attention Module are designed to improve the feature representation ability of the model. Consistency loss and complementarity loss are utilized fully for the complementarity and consistency of information between multi-view data. Combined with Focal Loss, information bias is effectively avoided. Experimental validation illustrates the effectiveness of the proposed method, with the accuracy, specificity, sensitivity, and F1 score reaching 77.92%, 75.62%, 82.50%, and 71.35%, respectively. Compared with methods such as clinical bedside screening tests and existing artificial intelligence-based methods, our method is more accurate and reliable and can provide a reliable auxiliary tool for clinical healthcare personnel to effectively improve the accuracy and reliability of preoperative difficult airway assessments. The proposed network can help to identify and assess the risk of difficult airways in patients before surgery and reduce the incidence of postoperative complications.

Multi-view hypergraph regularized Lp norm least squares twin support vector machines for semi-supervised learning

In recent years, multi-view semi-supervised learning has gradually become a popular research direction. The classic binary classification methods in this field are multi-view Laplacian support vector machines (MvLapSVM) and multi-view Laplacian twin support vector machines (MvLapTSVM), which extend semi-supervised support vector machine to multi-view learning. Nevertheless, similar to the majority of SVM-based multi-view methods, the above methods are two-view methods that cannot fully leverage the information from all views and are constructed based on the L2 norm. Additionally, in semi-supervised graph learning, the quality of the graph often has a significant impact on the results. Therefore, we propose a novel multi-view hypergraph regularized Lp norm least squares twin support vector machines (MvHGLpLSTSVM) that can handle general multi-view data for semi-supervised learning. It extends hypergraph learning to multi-view learning and combines Lp norm to further explore the manifold structure and embedded geometric information of multi-view data. By using equality constraints, we design a simple and effective iterative algorithm. In the classification of six multi-view datasets, we compare the proposed method with some other state-of-the-art methods, and the results show that the proposed method is effective.

Anchor-guided global view reconstruction for multi-view multi-label feature selection

In multi-view multi-label learning (MVML), the accuracy of feature weights is pivotal for establishing feature order. However, conventional MVML methods often struggle with integrating distinct information from multiple views effectively, leading to unclear segmentation and potential noise introduction. To address this challenge, this paper proposes an anchor-based latent representation method for global view learning in MVML. Specifically, we encode inherent information from each view to derive a candidate multi-view representation. Anchors extracted from both the candidate view and the global view are then constrained to approximate equality in the latent space. Furthermore, a carefully designed view matrix serves as supplement, seamlessly integrated into the reconstruction process to augment information. The convergence of results is subsequently validated using multiplicative update rules. Experimental findings showcase the superior performance of our proposed method across various multi-view datasets.

Variational Distillation for Multi-View Learning.

Information Bottleneck (IB) provides an information-theoretic principle for multi-view learning by revealing the various components contained in each viewpoint. This highlights the necessity to capture their distinct roles to achieve view-invariance and predictive representations but remains under-explored due to the technical intractability of modeling and organizing innumerable mutual information (MI) terms. Recent studies show that sufficiency and consistency play such key roles in multi-view representation learning, and could be preserved via a variational distillation framework. But when it generalizes to arbitrary viewpoints, such strategy fails as the mutual information terms of consistency become complicated. This paper presents Multi-View Variational Distillation (MV 2D), tackling the above limitations for generalized multi-view learning. Uniquely, MV 2D can recognize useful consistent information and prioritize diverse components by their generalization ability. This guides an analytical and scalable solution to achieving both sufficiency and consistency. Additionally, by rigorously reformulating the IB objective, MV 2D tackles the difficulties in MI optimization and fully realizes the theoretical advantages of the information bottleneck principle. We extensively evaluate our model on diverse tasks to verify its effectiveness, where the considerable gains provide key insights into achieving generalized multi-view representations under a rigorous information-theoretic principle.

Pedestrian 3D Shape Understanding for Person Re-Identification via Multi-View Learning

Pedestrian 3D Shape Understanding for Person Re-Identification via Multi-View Learning

Orchard bird song recognition based on multi-view multi-level contrastive learning

In the harvest season, orchards are frequently plagued by birds, and thus significant fruit pecking can adversely affect both the fruit quality and yield. Recognizing bird songs is crucial for preventing damage caused by orchard birds because it can provide the basis for subsequent bird repellent efforts. However, the extensive effort required to annotate sound samples poses a significant challenge for supervised deep learning. In this paper, we propose a self-supervised multi-view learning framework based on multi-level contrasting (MV-MLC) for bird song recognition, which utilizes both time and spectrogram views as inputs. This framework leverages MLC to automatically learn representations from unlabeled data and a multi-scale feature extraction (MSFE) backbone network is employed to capture the temporal features of bird songs at different scales. The time-spectrogram consistency task in MLC learning facilitates semantic-level information exchange across multi-views, while the hierarchical contrastive learning task captures granularity-level information, thereby resulting in more robust contextual representations. In addition, embedding the shuffle attention module in MSFE facilitates mining of the spatial and channel dependencies of bird song features to further enhance the representation of features by the multi-scale network. We conducted extensive experiments using our self-built 10-class bird song data set (Orchard-birds) and the publicly available Birdsdata and Powdermill data sets. The experimental results demonstrated that MV-MLC performed better than state-of-the-art self-supervised models. In particular, MV-MLC obtained outstanding performance even with a small proportion of labeled data. The recognition accuracies based on the Orchard-birds and Birdsdata data sets were 99.40% and 92.67%, respectively, with macro F1-scores of 99.40% and 92.61%.

A Multi-View Deep Learning Model for Thyroid Nodules Detection and Characterization in Ultrasound Imaging.

Thyroid Ultrasound (US) is the primary method to evaluate thyroid nodules. Deep learning (DL) has been playing a significant role in evaluating thyroid cancer. We propose a DL-based pipeline to detect and classify thyroid nodules into benign or malignant groups relying on two views of US imaging. Transverse and longitudinal US images of thyroid nodules from 983 patients were collected retrospectively. Eighty-one cases were held out as a testing set, and the rest of the data were used in five-fold cross-validation (CV). Two You Look Only Once (YOLO) v5 models were trained to detect nodules and classify them. For each view, five models were developed during the CV, which was ensembled by using non-max suppression (NMS) to boost their collective generalizability. An extreme gradient boosting (XGBoost) model was trained on the outputs of the ensembled models for both views to yield a final prediction of malignancy for each nodule. The test set was evaluated by an expert radiologist using the American College of Radiology Thyroid Imaging Reporting and Data System (ACR-TIRADS). The ensemble models for each view achieved a mAP0.5 of 0.797 (transverse) and 0.716 (longitudinal). The whole pipeline reached an AUROC of 0.84 (CI 95%: 0.75-0.91) with sensitivity and specificity of 84% and 63%, respectively, while the ACR-TIRADS evaluation of the same set had a sensitivity of 76% and specificity of 34% (p-value = 0.003). Our proposed work demonstrated the potential possibility of a deep learning model to achieve diagnostic performance for thyroid nodule evaluation.

Multi-view contextual adaptation network for weakly supervised object detection in remote sensing images

ABSTRACT Weakly supervised learning plays a pivotal role in the field of object detection, i.e. Weakly supervised object detection (WSOD), significantly reducing annotation costs relying on image-level labels. However, WSOD exhibits certain limitations. Typically, they tend to identify the most easily recognizable local regions within targets, posing challenges in accurately delineating the boundaries of targets. Moreover, the presence of multiple instances of the same class in adjacent locations complicates the effective distinction between multiple objects within the same category. On the other hand, the complex backgrounds and dense distribution of targets in remote sensing images (RSI) further exacerbate the difficulty of weakly supervised detection. To address the above issues, we propose a model termed the Multi-View Contextual Adaptation Network (VCANet). Building on the classic Online Instance Classifier Refinement (OICR) framework, we propose to incorporate an contextual adaptation perception, within a multi-view learning framework, and integrate a pseudo-label filtering process. The contextual adaptation perception utilizes the surrounding environment information to enhance localization capabilities, guiding the model to prioritize target objects by referring to their spatially neighbouring pixels. Multi-view learning manufactures additional constraints from diverse perspectives, thereby revealing objects that might be overlooked due to the weak supervision in a single view. The pseudo-label filtering process eliminates inaccurate pseudo-labels by identifying reliable foregrounds to mitigate overlapping proposals during the label propagation. On challenging datasets NWPU VHR-10.v2 and DIOR, we achieve promising results with mAP of 62.3% and 28.2%, respectively, surpassing existing benchmarks.

Privacy preservation-based federated learning with uncertain data

Federated learning (FL) belongs to distributed machine learning. It allows data information sharing between users while protecting their data privacy at the same time. However, in many real-world scenarios, the data collected by client devices may be affected by noise in the working environment, leading to the decreased accuracy and confidence. Therefore, it is necessary to take measures to reduce data uncertainty in order to enhance the performance of FL algorithms. Traditional FL methods encounter challenges in handling uncertain data, which motivates the introduction of multi-view learning in this paper which designs an FL model suitable for highly variable data characteristics. We first achieve information complementarity among data views while ensuring the consistency in data representation. Furthermore, we quantify data uncertainty using the reachable region of data noise, thereby improving model robustness. To maintain data privacy between clients, we design an adaptive Kalman filter-based differential protection security protocol. Clients use the protocol to process local data and upload it to the master server, which returns the updated model parameters to the clients. The experimental results demonstrate the effectiveness of the federated learning model proposed in this paper.

Imputation of missing values in multi-view data

Data for which a set of objects is described by multiple distinct feature sets (called views) is known as multi-view data. When missing values occur in multi-view data, all features in a view are likely to be missing simultaneously. This may lead to very large quantities of missing data which, especially when combined with high-dimensionality, can make the application of conditional imputation methods computationally infeasible. However, the multi-view structure could be leveraged to reduce the complexity and computational load of imputation. We introduce a new imputation method based on the existing stacked penalized logistic regression (StaPLR) algorithm for multi-view learning. It performs imputation in a dimension-reduced space to address computational challenges inherent to the multi-view context. We compare the performance of the new imputation method with several existing imputation algorithms in simulated data sets and a real data application. The results show that the new imputation method leads to competitive results at a much lower computational cost, and makes the use of advanced imputation algorithms such as missForest and predictive mean matching possible in settings where they would otherwise be computationally infeasible.

Graph Contrastive Multi-view Learning: A Pre-training Framework for Graph Classification

Recent advancements in node and graph classification tasks can be attributed to the implementation of contrastive learning and similarity search. Despite considerable progress, these approaches present challenges. The integration of similarity search introduces an additional layer of complexity to the model. At the same time, applying contrastive learning to non-transferable domains or out-of-domain datasets results in less competitive outcomes. In this work, we propose maintaining domain specificity for these tasks, which has demonstrated the potential to improve performance by eliminating the need for additional similarity searches. We adopt a fraction of domain-specific datasets for pre-training purposes, generating augmented pairs that retain structural similarity to the original graph, thereby broadening the number of views. This strategy involves a comprehensive exploration of optimal augmentations to devise multi-view embeddings. An evaluation protocol, which focuses on error minimization, accuracy enhancement, and overfitting prevention, guides this process to learn inherent, transferable structural representations that span diverse datasets. We combine pre-trained embeddings and the source graph as a beneficial input, leveraging local and global graph information to enrich downstream tasks. Furthermore, to maximize the utility of negative samples in contrastive learning, we extend the training mechanism during the pre-training stage. Our method consistently outperforms comparative baseline approaches in comprehensive experiments conducted on benchmark graph datasets of varying sizes and characteristics, establishing new state-of-the-art results.

Multi-view discriminative edge heterophily contrastive learning network for attributed graph anomaly detection

Attributed graph anomaly detection aims to identify abnormal nodes that significantly differ from most nodes in terms of their attribute or structure. Recent graph contrastive learning methods, which follow an augmenting-contrasting learning scheme, have shown promising results. However, these methods still have two key limitations. Firstly, the contrastive views generated through random perturbation may not conform to abnormal patterns. Secondly, the inability to effectively discriminate heterophilic edges in anomalous networks can adversely impact the detection performance. To this end, we propose a Multi-view discriminative Edge heterophiLy cOntrastive learning Network (MELON) for attributed graph anomaly detection. Our approach addresses the aforementioned limitations by integrating human knowledge of different anomaly patterns for data augmentation and designing an edge discriminator to identify edge homophily/heterophily in an unsupervised manner. In addition, a dual-channel encoder is also designed to capture representative representations of the nodes from the discriminated homophilic/heterophilic edges. We then use a decoder to reconstruct the original network from the node representations learned by the dual-channel encoder and rank nodes according to their reconstruction error as the anomaly metric. Extensive experiments on four public benchmark datasets demonstrate that our proposed method outperforms state-of-the-art baselines.