Graph Learning Research Articles

Incomplete multi-view clustering via confidence graph completion based tensor decomposition

In recent years, extensive incomplete multi-view clustering models have been proposed to solve the problem of real-world multi-view data with missing views. However, they still have the following two drawbacks: (1) They ignore the harmful effects of outliers in data and the noise generated during graph recovery. (2) They do not fully explore the high-order relationships among views. To this end, we design a novel graph learning model called confidence graph completion based tensor decomposition (CGCTD) for incomplete multi-view clustering. Specifically, we use the confidence graphs to guide the learning of the complete graphs, which reduces the detrimental effects of outliers and missing samples in the data. Then, we stack the complete graphs of each view into an original tensor to explore high-order relationships and correlations between views. To reduce the negative effects of noise, we decompose the original tensor into an essential tensor and a noise tensor. The essential tensor is introduced to recover the accurate affinity graphs, and the noise tensor aims to model the noise contained in the corrupted graphs. Furthermore, we impose the tensor Schatten p-norm constraint on the essential tensor, which can enhance the low-rank property of the graphs and explore the similarity structure between views. Through extensive experiments on eight benchmark datasets, we demonstrate that the proposed CGCTD outperforms several existing state-of-the-art methods.

Multi-task heterogeneous graph learning on electronic health records

Learning electronic health records (EHRs) has received emerging attention because of its capability to facilitate accurate medical diagnosis. Since the EHRs contain enriched information specifying complex interactions between entities, modeling EHRs with graphs is shown to be effective in practice. The EHRs, however, present a great degree of heterogeneity, sparsity, and complexity, which hamper the performance of most of the models applied to them. Moreover, existing approaches modeling EHRs often focus on learning the representations for a single task, overlooking the multi-task nature of EHR analysis problems and resulting in limited generalizability across different tasks. In view of these limitations, we propose a novel framework for EHR modeling, namely MulT-EHR (Multi-Task EHR), which leverages a heterogeneous graph to mine the complex relations and model the heterogeneity in the EHRs. To mitigate the large degree of noise, we introduce a denoising module based on the causal inference framework to adjust for severe confounding effects and reduce noise in the EHR data. Additionally, since our model adopts a single graph neural network for simultaneous multi-task prediction, we design a multi-task learning module to leverage the inter-task knowledge to regularize the training process. Extensive empirical studies on MIMIC-III and MIMIC-IV datasets validate that the proposed method consistently outperforms the state-of-the-art designs in four popular EHR analysis tasks — drug recommendation, and predictions of the length of stay, mortality, and readmission. Thorough ablation studies demonstrate the robustness of our method upon variations to key components and hyperparameters.

A simple multi-constraint fusion based semi-supervised non-negative matrix decomposition for image clustering

Semi-supervised non-negative matrix decomposition (SNMF) is a highly interpretable image clustering algorithm. We typically incorporate graph learning to enhance the model’s representation ability, while adding positive and negative labeling information and pairwise constraints to prevent overfitting and improve the discriminative performance. However, embedding positive and negative labels and pairwise constraints simultaneously in the SNMF model is challenging due to their dimension mismatch. Thus We propose a method named multi-constraint fusion based semi-supervised non-negative matrix decomposition (MCFSNMF), which effectively solves the above problem by constructing symmetric labeling matrices embedded with positive and negative labeling information and fusing them with pairwise constraint matrices. In addition, SNMF is essentially a feature extraction method, which cannot obtain clustering results directly. Therefore, we combine the label propagation algorithm to achieve feature learning and label assignment interaction, which not only avoids the impact of clustering post-processing operations but also enhances the representation of labeled data relative to unlabeled data. We compare multiple SNMF algorithms on six image datasets and show that the proposed algorithm can effectively enhance the clustering performance and validate the algorithm’s convergence.

Bridging Data Complexity with GATNet for Learning in Interconnected Electronic Medical Records Graphs

Bridging Data Complexity with GATNet for Learning in Interconnected Electronic Medical Records Graphs

Dual temporal memory network with high-order spatio-temporal graph learning for video object segmentation

Typically, Video Object Segmentation (VOS) always has the semi-supervised setting in the testing phase. The VOS aims to track and segment one or several target objects in the following frames in the sequence, only given the ground-truth segmentation mask in the initial frame. A fundamental issue in VOS is how to best utilize the temporal information to improve the accuracy. To address the aforementioned issue, we provide an end-to-end framework that simultaneously extracts long-term and short-term historical sequential information to current frame as temporal memories. The integrated temporal architecture consists of a short-term and a long-term memory modules. Specifically, the short-term memory module leverages a high-order graph-based learning framework to simulate the fine-grained spatial–temporal interactions between local regions across neighboring frames in a video, thereby maintaining the spatio-temporal visual consistency on local regions. Meanwhile, to relieve the occlusion and drift issues, the long-term memory module employs a Simplified Gated Recurrent Unit (S-GRU) to model the long evolutions in a video. Furthermore, we design a novel direction-aware attention module to complementarily augment the object representation for more robust segmentation. Our experiments on three mainstream VOS benchmarks, containing DAVIS 2017, DAVIS 2016, and Youtube-VOS, demonstrate that our proposed solution provides a fair tradeoff performance between both speed and accuracy.

Regulation-aware graph learning for drug repositioning over heterogeneous biological network

Drug repositioning (DR) is crucial for identifying new disease indications for existing drugs and enhancing their clinical utility. Despite the effectiveness of various artificial intelligence techniques in discovering novel drug-disease associations (DDAs), many algorithms primarily focus on incorporating biological knowledge of drugs and diseases into DDA networks, often overlooking the rich connectivity patterns inherent in heterogeneous biological networks. In this study, we leveraged diverse connectivity patterns to gain new insights into the regulatory mechanisms of drugs acting on target proteins in diseases. We defined a set of meta-paths to reveal different regulatory mechanisms, each corresponding to distinct connectivity patterns. For each meta-path, we constructed a regulation graph through random-walk sampling of its instances in the network and obtained drug and disease embeddings through regulation-aware graph representation learning. Subsequently, we proposed a novel multi-view attention mechanism to enhance drug and disease representations. The task of predicting DDAs was accomplished using the XGBoost classifier based on the final representations of drugs and diseases. The experimental results demonstrated the superior performance of our method, RGLDR, on three benchmark datasets under ten-fold cross-validation, outperforming state-of-the-art DR algorithms across several evaluation metrics. Furthermore, case studies on two diseases indicated that RGLDR is a promising DR tool that leverages meaningful connectivity patterns for improved efficacy.

A novel fine-grained assembly sequence planning method based on knowledge graph and deep reinforcement learning

In the assembly sequence planning (ASP) of aviation products, recalibration of components or sufficient space to assemble subsequent components are critical factors for ensuring product quality. To address this need, a fine-grained ASP (FASP) is defined to take assembly operations as units to plan sequences. Lots of operations have complex sequence constraints that are attended unequally in the FASP. A method based on knowledge graph (KG) and deep reinforcement learning is proposed to plan assembly operations. Firstly, continuous and discrete procedures are defined, and a quantitative characterization method is presented to deduce complex constraints objectively. Then, a dynamic KG is designed to establish and update the information model mainly composed of constraints. Finally, a labeled degree centrality algorithm (LDCA) considers edge labels to minimize the number of assembly tool changes and assembly direction changes for sequences. An improved deep Q-network (IDQN) introduces a convolutional layer to extract local features of technical requirements for planning procedures more efficiently. A helicopter structure assembly is used to verify the effectiveness of the proposed method. The improved algorithms have better performance in solving speed, sequence quality, and convergence speed than ordinary ASP methods, respectively. The fine-grained assembly sequence is more reasonable and feasible by comparing it with the ordinary sequence.

Unsupervised feature selection based on bipartite graph and low-redundant regularization

Unsupervised feature selection (UFS) has attracted increasing attention because of the difficulty and high cost of obtaining data labels. Since the ignorance of redundancy between features and the use of a fixed similarity matrix, current UFS algorithms do not perform well. Moreover, because of the high computational complexity, existing UFS algorithms usually cannot handle large-scale data. A novel UFS method is proposed to solve these problems through bipartite graph learning with low-redundant regularization (BGLR). Firstly, BGLR uses a variance-based de-correlation anchor selection method to ensure that selected anchors can be evenly distributed across classes. Secondly, BGLR builds an adaptive bipartite graph by using selected anchors and original data in the projection space. L2,0-norm constraint is applied to the projection matrix. Thus, the feature subset can be obtained directly. Finally, a regularization term is applied to control the redundancy of selected features. Moreover, BGLR is solved by an iterative method, which can converge quickly. Experiments are performed on public datasets in comparison with some relevant methods. The experimental results demonstrate the BGLR’s effectiveness.

Emotion recognition of EEG signals based on contrastive learning graph convolutional model.

Electroencephalogram (EEG) signals offer invaluable insights into the complexities of emotion generation within the brain. Yet, the variability in EEG signals across individuals presents a formidable obstacle for empirical implementations. Our research addresses these challenges innovatively, focusing on the commonalities within distinct subjects' EEG data.We introduce a novel approach named Contrastive Learning Graph Convolutional Network (CLGCN). This method captures the distinctive features and crucial channel nodes related to individuals' emotional states. Specifically, CLGCN merges the dual benefits of Contrastive Learning's synchronous multisubject data learning and the Graph Convolutional Network's proficiency in deciphering brain connectivity matrices.Understanding multifaceted brain functions and their information interchange processes is realized as CLGCN generates a standardized brain network learning matrix during a dataset's learning process. Our model significantly streamlines the retraining process for new subjects, requiring only 5% of the initial sample size for fine-tuning to attain a remarkable 92.8% accuracy rate. Additionally, our model has undergone extensive testing on the DEAP and SEED datasets, demonstrating the effectiveness of our model.&#xD.

Graph classification using high-difference-frequency subgraph embedding

With the rapid growth of big data analysis, graphs have become an important data structure in relationship extraction and learning. However, the complexity of graph structure increases the difficulty of downstream learning tasks, e.g., graph classification. For example, some traditional graph classification methods rely on the attributes of nodes and edges; the attributes may be incomplete or missing in large-scale graph datasets, e.g., online social networks and knowledge graphs. Focusing on the node-level features overlooks the global characteristics, leading to decreased classification accuracy. This paper proposes a novel graph classification method based on the subgraph-level feature the high-difference-frequency subgraph. We measure the difference in subgraph appearing frequency and use the high difference frequency subgraphs as features in graph learning and classification. The experiments demonstrate that the proposed method outperforms the state-of-the-art graph classification methods.

TaylorNet: A novel approach for spectral filter learning on graph data

Graphs are indispensable data structures used to model interconnected nodes and their relationships. However, devising filters for graph data is challenging. Previous approaches, including BernNet, utilized Bernstein polynomials to approximate spectral graph convolutions but suffered from lengthy training times compared to alternatives. To address this, we propose TaylorNet, a novel graph neural network built upon rigorous mathematical principles and theoretical foundations. TaylorNet provides an efficient approach for learning arbitrary graph spectral filters. Our methodology revolves around utilizing Kth order Taylor polynomials to estimate filters on the normalized Laplacian operator of graphs. By carefully setting the coefficients of these polynomials, we achieve effective filter design. Additionally, TaylorNet personalizes filter learning for specific graphs by leveraging observed graph data and signals to determine these coefficients. Extensive experimentation demonstrates TaylorNet’s ability to learn arbitrary spectral filters, surpassing current state-of-the-art methods, particularly for handling large graphs. Its exceptional performance extends to real-world graph modeling tasks, highlighting its versatility and applicability. For interested researchers, the complete implementation of TaylorNet is available at https://github.com/12chen20/TaylorNet.

A stage-driven construction algorithm of undirected independence graph for Bayesian network structure learning

A stage-driven construction algorithm of undirected independence graph for Bayesian network structure learning

Optimal sensor placement for leak location in water distribution networks based on graph learning algorithm

Abstract The effective monitoring of urban water distribution networks (WDNs) relies heavily on pressure sensor placement. Nevertheless, a WDN may consist of hundreds of nodes, and it is not economically feasible to install sensors at each node. Therefore, how to identify an optimal location for sensor placement becomes a crucial issue. We use graph signal processing to analyze the pressure signals and introduce a framework for optimal sensor placement specifically designed for pressure signals. To address the limitation that pressure signals cannot be sampled directly, we propose a method to convert the signal into a band-limited signal that meets the requirements. Central to the method is learning a graph Fourier operator, and the effectiveness of the proposed method is proved theoretically. The graph Fourier operator enables the pressure data to become a smooth graph signal with variations in its topology. In addition, we design a graph filter based on the energy of the signal and obtain a band-limited signal that meets the requirements. To ensure the selection of representative nodes, we use a noise-robust graph sampling method to obtain the sensor node. Our method is further evaluated using the pressure data from Anytown versus Net3, showing strong performance in leak identification and signal reconstruction capabilities.

Graph and text multi-modal representation learning with momentum distillation on Electronic Health Records

The emergence and widespread adoption of electronic health records (EHR) in modern healthcare systems has generated a large amount of data with the potential to significantly improve patient outcomes. However, extracting meaningful insights from EHR is challenging due to their extensive volume and inherent complexity. Leveraging diverse modalities of data enhances the exploration of patient data representation. Nevertheless, current research on multi-modal relationships between concealed graph structures within medical codes and unstructured text overlooks inherent disparities and inconsistencies. To address the significant gap, this study introduces a pretraining approach named graph and text multi-modal representation learning with momentum distillation on Electronic Health Records (GTMMRL). This approach tackles the challenge of noisy and unreliable labels using a large open-source EHR dataset, MIMIC-III, for pretraining. Five carefully selected proxy tasks are employed, each guiding the student model’s learning through pseudo-targets generated by the teacher model, which is an exponential moving average of the student model, thereby mitigating overfitting tendencies. We evaluate the performance of our model in five downstream tasks, each addressing real-world challenges in EHR data analysis. Results showcase that our method achieves state-of-the-art performance in various downstream tasks, highlighting the model’s critical role in enhancing the quality of insights extracted from complex EHR data. The code is available at https://github.com/deepEMR/GTMMRL.

From data to action in flood forecasting leveraging graph neural networks and digital twin visualization

Forecasting floods encompasses significant complexity due to the nonlinear nature of hydrological systems, which involve intricate interactions among precipitation, landscapes, river systems, and hydrological networks. Recent efforts in hydrology have aimed at predicting water flow, floods, and quality, yet most methodologies overlook the influence of adjacent areas and lack advanced visualization for water level assessment. Our contribution is two-fold: firstly, we introduce a graph neural network model (LocalFLoodNet) equipped with a graph learning module to capture the interconnections of water systems and the connectivity between stations to predict future water levels. Secondly, we develop a simulation prototype offering visual insights for decision-making in disaster prevention and policy-making. This prototype visualizes predicted water levels and facilitates data analysis using decades of historical information. Focusing on the Greater Montreal Area (GMA), particularly Terrebonne, Quebec, Canada, we apply LocalFLoodNet and prototype to demonstrate a comprehensive method for assessing flood impacts. By utilizing a digital twin of Terrebonne, our simulation tool allows users to interactively modify the landscape and simulate various flood scenarios, thereby providing valuable insights into preventive strategies. This research aims to enhance water level prediction and evaluation of preventive measures, setting a benchmark for similar applications across different geographic areas.

A New Benchmark: Clinical Uncertainty and Severity Aware Labeled Chest X-Ray Images with Multi-Relationship Graph Learning.

Chest radiography, commonly known as CXR, is frequently utilized in clinical settings to detect cardiopulmonary conditions. However, even seasoned radiologists might offer different evaluations regarding the seriousness and uncertainty associated with observed abnormalities. Previous research has attempted to utilize clinical notes to extract abnormal labels for training deep-learning models in CXR image diagnosis. However, these methods often neglected the varying degrees of severity and uncertainty linked to different labels. In our study, we initially assembled a comprehensive new dataset of CXR images based on clinical textual data, which incorporated radiologists' assessments of uncertainty and severity. Using this dataset, we introduced a multi-relationship graph learning framework that leverages spatial and semantic relationships while addressing expert uncertainty through a dedicated loss function. Our research showcases a notable enhancement in CXR image diagnosis and the interpretability of the diagnostic model, surpassing existing state-of-the-art methodologies. The dataset address of disease severity and uncertainty we extracted is: https://physionet.org/content/cad-chest/1.0/.

Learning to Allocate Time-Bound and Dynamic Tasks to Multiple Robots Using Covariant Attention Neural Networks

Abstract In various applications of multi-robotics in disaster response, warehouse management, and manufacturing, tasks that are known a priori and tasks added during run time need to be assigned efficiently and without conflicts to robots in the team. This multi-robot task allocation (MRTA) process presents itself as a combinatorial optimization (CO) problem that is usually challenging to be solved in meaningful timescales using typical (mixed)integer (non)linear programming tools. Building on a growing body of work in using graph reinforcement learning to learn search heuristics for such complex CO problems, this paper presents a new graph neural network architecture called the covariant attention mechanism (CAM). CAM can not only generalize but also scale to larger problems than that encountered in training, and handle dynamic tasks. This architecture combines the concept of covariant compositional networks used here to embed the local structures in task graphs, with a context module that encodes the robots’ states. The encoded information is passed onto a decoder designed using multi-head attention mechanism. When applied to a class of MRTA problems with time deadlines, robot ferry range constraints, and multi-trip settings, CAM surpasses a state-of-the-art graph learning approach based on the attention mechanism, as well as a feasible random-walk baseline across various generalizability and scalability tests. Performance of CAM is also found to be at par with a high-performing non-learning baseline called BiG-MRTA, while noting up to a 70-fold improvement in decision-making efficiency over this baseline.

A multi-view mask contrastive learning graph convolutional neural network for age estimation

A multi-view mask contrastive learning graph convolutional neural network for age estimation

Consensus local graph for multiple kernel clustering

Due to the ability to represent the relationship among data, graph has received extensive attention in clustering field. As illustrated by existing studies, learning graph in kernel space is an effective way to capture the structure of data. Among kernel-based methods, multiple kernel learning (MKL) is increasingly popular since it can automatically utilize the complementary information contained in base kernels. Although many existing MKL-based graph learning algorithms have promising learning abilities, we observe that they (1) put too much attention on learning the consensus kernel which probably contains lots of redundant information and loses the diversity of base kernels; (2) ignore the local structure of the representations in multiple kernel spaces. To eliminate the bad effect of these issues, we propose a novel graph learning method, termed consensus local graph based on MKL (CLGMKL), for clustering. In CLGMKL, a low-rank kernel matrix is used to extract the discriminative information of each base kernel and a local graph is constructed to capture the local structure contained in each new kernel. Then CLGMKL combines these local graphs in a self-weighed way. Since the above processes are associated with each other, we jointly optimize them to obtain the overall optimality. An effective learning scheme with proved convergence is developed to optimize the combined objective function. Finally, extensive experiments on some popular datasets are conducted to test the effectiveness of the presented method. As illustrated, CLGMKL is more competitive than the state-of-the-art algorithms.

Unveiling the unseen potential of graph learning through MLPs: Effective graph learners using propagation-embracing MLPs

Recent studies attempted to utilize multilayer perceptrons (MLPs) to solve semi-supervised node classification on graphs, by training a student MLP by knowledge distillation (KD) from a teacher graph neural network (GNN). While previous studies have focused mostly on training the student MLP by matching the output probability distributions between the teacher and student models during KD, it has not been systematically studied how to inject the structural information in an explicit and interpretable manner. Inspired by GNNs that separate feature transformation T and propagation Π, we re-frame the KD process as enabling the student MLP to explicitly learn both T and Π. Although this can be achieved by applying the inverse propagation Π−1 before distillation from the teacher GNN, it still comes with a high computational cost from large matrix multiplications during training. To solve this problem, we propose Propagate & Distill (P&D), which propagates the output of the teacher GNN before KD and can be interpreted as an approximate process of the inverse propagation Π−1. Through comprehensive evaluations using real-world benchmark datasets, we demonstrate the effectiveness of P&D by showing further performance boost of the student MLP.