Graph Representation Research Articles

Wasserstein Discriminant Dictionary Learning for Graph Representation.

Mining discriminative graph topological information plays an important role in promoting graph representation ability. However, it suffers from two main issues: (1) the difficulty/complexity of computing global inter-class/intra-class scatters, commonly related to mean and covariance of graph samples, for discriminant learning; (2) the huge complexity and variety of graph topological structure that is rather challenging to robustly characterize. In this paper, we propose the Wasserstein Discriminant Dictionary Learning (WDDL) framework to achieve discriminant learning on graphs with robust graph topology modeling, and hence facilitate graph-based pattern analysis tasks. Considering the difficulty of calculating global inter-class/intra-class scatters, a reference set of graphs (aka graph dictionary) is first constructed by generating representative graph samples (aka graph keys) with expressive topological structure. Then, a Wasserstein Graph Representation (WGR) process is proposed to project input graphs into a succinct dictionary space through the graph dictionary lookup. To further achieve discriminant graph learning, a Wasserstein discriminant loss (WD-loss) is defined on the graph dictionary, in which the graph keys are optimizable, to make the intra-class keys more compact and inter-class keys more dispersed. Hence, the calculation of global Wasserstein metric (W-metric) centers can be bypassed. For sophisticated topology mining in the WGR process, a joint-Wasserstein graph embedding module is constructed to model both between-node and between-edge relationships across inputs and graph keys by encapsulating both the Wasserstein metric (between cross-graph nodes) and proposed novel Kron-Gromov-Wasserstein (KGW) metric (between cross-graph adjacencies). Specifically, the KGW-metric comprehensively characterizes the cross-graph connection patterns with the Kronecker operation, then adaptively captures those salient patterns through connection pooling. To evaluate the proposed framework, we study two graph-based pattern analysis problems, i.e. graph classification and cross-modal retrieval, with the graph dictionary flexibly adjusted to cater to these two tasks. Extensive experiments are conducted to comprehensively compare with existing advanced methods, as well as dissect the critical component of our proposed architecture. The experimental results validate the effectiveness of the WDDL framework.

HiMolformer: Integrating graph and sequence representations for predicting liver microsome stability with SMILES

In the initial stages of drug discovery or pre-clinical studies, understanding the metabolic stability of new molecules is crucial. Recently, research on pre-trained deep learning for molecular property prediction has been actively progressing, with various models being made open-source. However, most of these models rely on either 2D graph or 1D sequence for training, and the representation varies depending on the data format used. Consequently, combining multiple representations can broaden the scope of learning and may potentially be a manageable and most effective method to enhance performance.Therefore, we propose a novel hybrid model for predicting metabolic stability, which integrates representations from both graph-based and sequence-based models pre-trained for molecular features. This approach utilizes the combined strengths of 2D topological and 1D sequential information of molecules. HiMol, a graph-based graph neural network (GNN) model, and Molformer, a sequence-based Transformer model, were selected for integration, thus we named it HiMolformer. HiMolformer demonstrated superior performance compared to other models. We also focus on regression task for prediction with a empirical dataset from Korea Chemical Bank (KCB), comprising 3,498 molecules with mouse liver microsome (MLM) and human liver microsome (HLM) data obtained from actual metabolic reaction experiments. To the best of our knowledge, it is the first attempt to develop MLM and HLM prediction models using regression with a single SMILES input. The source code of this model is available at https://github.com/YUNSEOKWOO/HiMolformer.

Pair Then Relation: Pair-Net for Panoptic Scene Graph Generation.

Panoptic Scene Graph (PSG) is a challenging task in Scene Graph Generation (SGG) that aims to create a more comprehensive scene graph representation using panoptic segmentation instead of boxes. Compared to SGG, PSG has several challenging problems: pixel-level segment outputs and full relationship exploration (It also considers thing and stuff relation). Thus, current PSG methods have limited performance, which hinders downstream tasks or applications. This work aims to design a novel and strong baseline for PSG. To achieve that, we first conduct an in-depth analysis to identify the bottleneck of the current PSG models, finding that inter-object pair-wise recall is a crucial factor that was ignored by previous PSG methods. Based on this and the recent query-based frameworks, we present a novel framework: Pair then Relation (Pair-Net), which uses a Pair Proposal Network (PPN) to learn and filter sparse pair-wise relationships between subjects and objects. Moreover, we also observed the sparse nature of object pairs for both. Motivated by this, we design a lightweight Matrix Learner within the PPN, which directly learns pair-wised relationships for pair proposal generation. Through extensive ablation and analysis, our approach significantly improves upon leveraging the segmenter solid baseline. Notably, our method achieves over 10% absolute gains compared to our baseline, PSGFormer.

SaaN 2L-GRL: Two-Level Graph Representation Learning Empowered With Subgraph-as-a-Node

SaaN 2L-GRL: Two-Level Graph Representation Learning Empowered With Subgraph-as-a-Node

U-Match: Exploring Hierarchy-Aware Local Context for Two-View Correspondence Learning.

Rejecting outlier correspondences is one of the critical steps for successful feature-based two-view geometry estimation, and contingent heavily upon local context exploration. Recent advances focus on devising elaborate local context extractors whereas typically adopting explicit neighborhood relationship modeling at a specific scale, which is intrinsically flawed and inflexible, because 1) severe outliers often populated in putative correspondences and 2) the uncertainty in the distribution of inliers and outliers make the network incapable of capturing adequate and reliable local context from such neighborhoods, therefore resulting in the failure of pose estimation. This prospective study proposes a novel network called U-Match that has the flexibility to enable implicit local context awareness at multiple levels, naturally circumventing the aforementioned issues that plague most existing studies. Specifically, to aggregate multi-level local context implicitly, a hierarchy-aware graph representation module is designed to flexibly encode and decode hierarchical features. Moreover, considering that global context always works collaboratively with local context, an orthogonal local-and-global information fusion module is presented to integrate complementary local and global context in a redundancy-free manner, thus yielding compact feature representations to facilitate correspondence learning. Thorough experimentation across relative pose estimation, homography estimation, visual localization, and point cloud registration affirms U-Match's remarkable capabilities.

A Survey on Graph Neural Networks and Graph Transformers in Computer Vision: A Task-Oriented Perspective.

Graph Neural Networks (GNNs) have gained momentum in graph representation learning and boosted the state of the art in a variety of areas, such as data mining (e.g., social network analysis and recommender systems), computer vision (e.g., object detection and point cloud learning), and natural language processing (e.g., relation extraction and sequence learning), to name a few. With the emergence of Transformers in natural language processing and computer vision, graph Transformers embed a graph structure into the Transformer architecture to overcome the limitations of local neighborhood aggregation while avoiding strict structural inductive biases. In this paper, we present a comprehensive review of GNNs and graph Transformers in computer vision from a task-oriented perspective. Specifically, we divide their applications in computer vision into five categories according to the modality of input data, i.e., 2D natural images, videos, 3D data, vision + language, and medical images. In each category, we further divide the applications according to a set of vision tasks. Such a task-oriented taxonomy allows us to examine how each task is tackled by different GNN-based approaches and how well these approaches perform. Based on the necessary preliminaries, we provide the definitions and challenges of the tasks, in-depth coverage of the representative approaches, as well as discussions regarding insights, limitations, and future directions.

Graph Representation Learning Based on Cognitive Spreading Activations

Graph Representation Learning Based on Cognitive Spreading Activations

Unsupervised Graph Representation Learning Beyond Aggregated View

Unsupervised Graph Representation Learning Beyond Aggregated View

Deep reinforcement learning-based spatio-temporal graph neural network for solving job shop scheduling problem

AbstractThe job shop scheduling problem (JSSP) is a well-known NP-hard combinatorial optimization problem that focuses on assigning tasks to limited resources while adhering to certain constraints. Currently, deep reinforcement learning (DRL)-based solutions are being widely used to solve the JSSP by defining the problem structure on disjunctive graphs. Some of the proposed approaches attempt to leverage the structural information of the JSSP to capture the dynamics of the environment without considering the time dependency within the JSSP. However, learning graph representations only from the structural relationship of nodes results in a weak and incomplete representation of these graphs which does not provide an expressive representation of the dynamics in the environment. In this study, unlike existing frameworks, we defined the JSSP as a dynamic graph to explicitly consider the time-varying aspect of the JSSP environment. To this end, we propose a novel DRL framework that captures both the spatial and temporal attributes of the JSSP to construct rich and complete graph representations. Our DRL framework introduces a novel attentive graph isomorphism network (Attentive-GIN)-based spatial block to learn the structural relationship and a temporal block to capture the time dependency. Additionally, we designed a gated fusion block that selectively combines the learned representations from the two blocks. We trained the model using the proximal policy optimization algorithm of reinforcement learning. Experimental results show that our trained model exhibits significant performance enhancement compared to heuristic dispatching rules and learning-based solutions for both randomly generated datasets and public benchmarks.

Point-of-interest recommendation based on the spatial-temporal graph

Purpose Point-of-interest (POI) recommendation techniques play a crucial role in mitigating information overload and delivering tailored services. To address limitations in conventional POI recommendation systems, constrained by sparse user-POI interactions and incomplete consideration of temporal dynamics, POI recommendation based on the spatial-temporal graph (STG-POI) is proposed. Design/methodology/approach Spatial-temporal sequence graphs from geographical locations and user interaction history data are constructed, which are used to mine spatial-temporal sequence information. Using the data filtered by the band-pass filter, graph neural networks with distance-awareness and sequence-awareness are applied to capture high-order spatial-temporal connections within diverse graph topologies. The model leverages contrastive learning for self-supervised disentanglement of graph representations, providing self-supervised signals for sequential and geographical intent perception, thereby achieving more precise POI personalization. Findings Compared to the baseline model GSTN, experiments on the Foursquare and Gowalla data sets reveal that STG-POI improves testing AUC by 2.0%, 2.1%, 2.0% and decreases logloss by 1.9%, 3.3%, 0.3%, respectively. These results indicate the model’s effectiveness in capturing spatial-temporal information, surpassing mainstream POI recommendation baseline models. Originality/value This approach constructs a dual graph from user interaction data, harnessing sequential and geographical information as self-supervised signals. It yields decoupled representations of these influences, offering a comprehensive insight into user behaviors and preferences within location-based social networks, thus enhancing recommendation accuracy and interpretability. This approach addresses the challenge in graph convolutional network where only rough and smooth features are conducive to recommendation by using band-pass filters to significantly reduce computational complexity, thereby enhancing recommendation speed by filtering out noise data that does not contribute to recommendation performance. Experimental results indicate that this model surpasses current mainstream approaches in POI recommendation tasks, effectively integrating both geographical and temporal features.

Predicting Protein-Ligand Binding Affinity Using Fusion Model of Spatial-Temporal Graph Neural Network and 3D Structure-Based Complex Graph.

The investigation of molecular interactions between ligands and their target molecules is becoming more significant as protein structure data continues to develop. In this study, we introduce PLA-STGCNnet, a deep fusion spatial-temporal graph neural network designed to study protein-ligand interactions based on the 3D structural data of protein-ligand complexes. Unlike 1D protein sequences or 2D ligand graphs, the 3D graph representation offers a more precise portrayal of the complex interactions between proteins and ligands. Research studies have shown that our fusion model, PLA-STGCNnet, outperforms individual algorithms in accurately predicting binding affinity. The advantage of a fusion model is the ability to fully combine the advantages of multiple different models and improve overall performance by combining their features and outputs. Our fusion model shows satisfactory performance on different data sets, which proves its generalization ability and stability. The fusion-based model showed good performance in protein-ligand affinity prediction, and we successfully applied the model to drug screening. Our research underscores the promise of fusion spatial-temporal graph neural networks in addressing complex challenges in protein-ligand affinity prediction. The Python scripts for implementing various model components are accessible at https://github.com/ligaili01/PLA-STGCN.

Graph Representation Learning Enhanced Semi-Supervised Feature Selection

Feature selection is a key step in machine learning by eliminating features that are not related to the modeling target to create reliable and interpretable models. By exploring the potential complex correlations among features of unlabeled data, recently introduced self-supervision-enhanced feature selection greatly reduces the reliance on the labeled samples. However, they are generally based on the autoencoder with sample-wise self-supervision, which can hardly exploit the relations among samples. To address this limitation, this article proposes graph representation learning enhanced semi-supervised feature selection (G-FS) which performs feature selection based on the discovery and exploitation of the non-Euclidean relations among features and samples by translating unlabeled “plain” tabular data into a bipartite graph. A self-supervised edge prediction task is designed to distill rich information on the graph into low-dimensional embeddings, which remove redundant features and noise. Guided by the condensed graph representation, we propose a batch attention feature weight generation mechanism that generates more robust weights according to batch-based selection patterns rather than individual samples. The results show that G-FS achieves significant performance edges in 14 datasets compared to twelve state-of-the-art baselines, including two recent self-supervised baselines. The source code is public available at https://github.com/Icannotnamemyselff/G-FS_Graph_enhacned_feature_selection .

Drug Sensitivity Prediction Based on Multi-stage Multi-modal Drug Representation Learning.

Accurate prediction of anticancer drug responses is essential for developing personalized treatment plans in order to improve cancer patient survival rates and reduce healthcare costs. To this end, we propose a drug sensitivity prediction model based on multi-stage multi-modal drug representations (ModDRDSP) to reflect the properties of drugs more comprehensively, and to better model the complex interactions between cells and drugs. Specifically, we adopt the SMILES representation learning method based on the deep hierarchical bi-directional GRU network (DSBiGRU) and the molecular graph representation learning method based on the deep message-crossing network (DMCN) for the multi-modal information of drugs. Additionally, we integrate the multi-omics information of cell lines based on a convolutional neural network (CNN). Finally, we use an ensemble deep forest algorithm for the prediction of drug sensitivity. After validation, the ModDRDSP shows impressive performance which outperforms the four current industry-leading models. More importantly, ablation experiments demonstrate the validity of each module of the proposed model, and case studies show the good results of ModDRDSP for predicting drug sensitivity, further establishing the superiority of ModDRDSP in terms of performance.

Demonstrative Learning for Human-Agent Knowledge Transfer

Demonstrative learning in virtual reality (VR) is a pivotal learning strategy for knowledge transfer for embodied agents. While existing studies have extensively explored agents’ knowledge transfer through self-demonstrative learning (SDL) or teacher-demonstrative learning (TDL), there has been limited focus on a system that integrates both paradigms. This paper proposes a comprehensive system that combines the SDL paradigm with the TDL paradigm in VR from a top-down perspective. The system involves using directed probabilistic graphs (DPGs) for knowledge representation, constructing detectors and actuators based on object fluents and atomic actions, representing knowledge acquired from both learning paradigms on the DPGs, and incorporating knowledge integration and visualization. Through system evaluation, we show the advantages of integrating two demonstrative learning paradigms, including increased learning efficiency, mitigating demonstrator's deficiencies, and more logical task execution. The study also reveals that a dynamically decreasing fusion factor with the learning progresses, performs well across different correct percentages of teacher demonstrations. Additionally, we show a more decentralized demonstration in the middle of the agent's learning progress maximizing learning efficiency when demonstrating only a few atomic actions. Finally, we assess the system's generalizability and delineate its boundaries. With the ongoing development and the increasing availability of human data in VR, we anticipate that our system can be applied to future scenarios of efficient human-agent knowledge transfer in human-agent symbiosis.

AWA-GCN: Enhancing Chinese sentiment analysis with a novel GCN model for Triplet and Quadruplet Extraction at SIGHAN 2024 dimABSA Task

This study presents the AWA-GCN (Attention-Weighted Affine with Graph Convolutional Networks) model, a new method designed exclusively for the SIGHAN 2024 Chinese Dimensional Aspect-Based Sentiment Analysis (dimABSA) Task. This test requires participants to identify sentiment triples and sentiment quadruples from textual data. The data comprises aspects, sentiment views, categories, and intensities across valence-arousal dimensions. Our model differs from existing models by utilizing a complex multi-layer attention mechanism within a GCN architecture, instead of relying on pipeline techniques or Grid Tagging Schemes (GTS). This design successfully captures the intricate interconnections among many sentiment components. The AWA-GCN model is very innovative in its ability to effectively handle quadruple extraction, which is a complex problem that has not been handled by traditional methods. Our approach enhances the understanding of aspect-sentiment interactions by including sophisticated techniques like word-level attention and semantic graph representations. The AWA- GCN model outperforms previous baselines in precision, recall, and F1 score on the dimABSA dataset, as demonstrated by empirical assessments. Additionally, the model exhibits significant enhancements in capturing the dimensional features of sentiment expressions. The results validate the models exceptional ability to handle the complex nuances necessary for successful sentiment analysis in simple Chinese.

Efficient structure-informed featurization and property prediction of ordered, dilute, and random atomic structures

Structure-informed materials informatics is a rapidly evolving discipline of materials science relying on the featurization of atomic structures or configurations to construct vector, voxel, graph, graphlet, and other representations useful for machine learning prediction of properties, fingerprinting, and generative design. This work discusses how current featurizers typically perform redundant calculations and how their efficiency could be improved by considering (1) fundamentals of crystallographic (orbits) equivalency to optimize ordered structures and (2) representation-dependent equivalency to optimize dilute, doped, and defect structures with broken symmetry. It also discusses and contrasts ways of (3) approximating random solid solutions occupying arbitrary lattices under such representations. Efficiency improvements discussed in this work were implemented within ▪ or python toolset for Structure-Informed Property and Feature Engineering with Neural Networks developed by authors since 2019 and shown to increase performance from 2 to 10 times for typical inputs. Throughout this work, the authors explicitly discuss how these advances can be applied to different kinds of similar tools in the community.

Baseline-free assisted lamb wave-based damage detection in CFRP composites using graph convolutional networks and Transformer models

Carbon fiber-reinforced polymer (CFRP) composites are widely used in aerospace but are susceptible to damage, threatening structural safety. Thus, developing an effective structural health monitoring system is essential. This study presents CFRP-former, a novel deep learning framework that detects damage in CFRP composites by constructing spatio-temporal representation graphs of Lamb waves (LWs). The CFRP-former comprises three key modules: graph convolutional neural network (GCNN), a multi-layer perceptron (MLP), and a Transformer Encoder. The GCNN models the topological relationships between LW nodes and sensors. The MLP reduces data dimensionality while preserving key information for subsequent analysis. Meanwhile, the Transformer Encoder, employing multi-head attention mechanisms, captures global time-series patterns in the LW signals. Additionally, the CFRP-former framework includes a parallel module that separately predicts both the size and location of damage. Experiments using only four sensors show that CFRP-former achieves high accuracy in detecting and quantifying damage, with robustness confirmed through ablation tests.

Fusing semantic and syntactic information for aspect sentiment triplet extraction

Aspect Sentiment Triplet Extraction (ASTE) aims to extract aspect terms, sentiment polarity and opinion terms explaining the reason for the sentiment from a sentence in the form of triplets. Many existing studies model the context by graph neural networks to learn relevant information from the generated graphs. However, some sentences may have syntactic errors or lack significant grammar, which may lead to poor results on the dataset of the model. In this paper, we propose the Fusing Semantic and Syntactic Information for Aspect Sentiment Triplet Extraction (FSSI) model, which incorporates both syntactic structure and semantic relevance in the context. Specifically, we construct a syntactic graph convolutional network to obtain comprehensive syntactic structure information and a semantic graph convolutional network to obtain global semantic relevance of sentences using the self-attention mechanism. Furthermore, we concatenate the graph representations generated by the two graph convolution networks to obtain the final enhanced representation. Finally, we apply an effective inference strategy to extract triplets. Extensive experimental results on benchmark datasets show that our model outperforms state-of-the-art approaches.

SPORT: A Subgraph Perspective on Graph Classification with Label Noise

Graph neural networks (GNNs) have achieved great success recently on graph classification tasks using supervised end-to-end training. Unfortunately, extensive noisy graph labels could exist in the real world because of the complicated processes of manual graph data annotations, which may significantly degrade the performance of GNNs. Therefore, we investigate the problem of graph classification with label noise, which is demanding because of the complex graph representation learning issue and serious memorization of noisy samples. In this work, we present a novel approach called S ubgra p h Set Netw or k with Sample Selection and Consis t ency Learning (SPORT) for this problem. To release the overfitting of GNNs, SPORT proposes to characterize each graph as a set of subgraphs generated by certain predefined stratagems, which can be viewed as samples from its underlying semantic distribution in graph space. Then we develop an equivariant network to encode the subgraph set with the consideration of the symmetry group. To further release the influences of noisy examples, we leverage the predictions of subgraphs to measure the likelihood of a sample being clean or noisy, followed by effective label updating. In addition, we propose a joint loss to advance the model generalizability by introducing consistency regularization. Comprehensive experiments on a wide range of graph classification datasets demonstrate the effectiveness of our SPORT. Specifically, SPORT outperforms the most competing baseline by up to 6.4%.

Efficient generation of grids and traversal graphs in compositional spaces towards exploration and path planning

AbstractDiverse disciplines across science and engineering deal with problems related to compositions, which exist in non-Euclidean simplex spaces, rendering many standard tools inaccurate or inefficient. This work explores such spaces conceptually in the context of materials discovery, quantifies their computational feasibility, and implements several essential methods specific to simplex spaces through a new high-performance open-source library . Most significantly, we derive and implement an algorithm for constructing a novel n-dimensional simplex graph data structure, containing all discretized compositions and possible neighbor-to-neighbor transitions. Critically, no distance or neighborhood calculations are performed, instead leveraging pure combinatorics and order in procedurally generated simplex grids, keeping the algorithm $${\mathcal{O}}(N)$$ O ( N ) , with minimal memory, enabling rapid construction of graphs with billions of transitions in seconds. Additionally, we demonstrate how such graph representations can be combined to homogeneously express complex path-planning problems, while facilitating efficient deployment of existing high-performance gradient descent, graph traversal, and other optimization algorithms.