Hierarchical Graph Representation Research Articles

GeoHi-GNN: Geometry-aware hierarchical graph representation learning for normal estimation

Normal estimation has been one of the key tasks in point cloud analysis, while it is challenging when facing with severe noises or complex regions. The challenges mainly come from the selection of supporting points for estimation, that is, improper selections of points and points' scale will lead to insufficient information, loss of details, etc. To this end, this paper proposes one feature-centric fitting scheme, GeoHi-GNN, by learning geometry-aware hierarchical graph representation for fitting weights estimation. The main functional module is the continuously conducted Hierarchically Geometric-aware (HG) module, consisting of two core operations, namely, the graph node construction (GNC) and the geometric-aware dynamic graph convolution (GDGC). GNC aims to aggregate the feature information onto a smaller number of nodes, providing global-to-local information while avoiding the interferences from noises in larger scales. With these nodes distributed in different scales, GDGC dynamically updates the node features regarding to both intrinsic feature and extrinsic geometric information. Finally, the hierarchical graphical features are cascaded to estimate the weights for supporting points in the surface fitting. Through the extensive experiments and comprehensive comparisons with the state-of-the-arts, our scheme has exhibited many attractive advantages such as being geometry-aware and robust, empowering further applications like more accurate surface reconstruction.

Image Manipulation Detection With Cascade Hierarchical Graph Representation

Image Manipulation Detection With Cascade Hierarchical Graph Representation

Haisor : Human-aware Indoor Scene Optimization via Deep Reinforcement Learning

3D scene synthesis facilitates and benefits many real-world applications. Most scene generators focus on making indoor scenes plausible via learning from training data and leveraging extra constraints such as adjacency and symmetry. Although the generated 3D scenes are mostly plausible with visually realistic layouts, they can be functionally unsuitable for human users to navigate and interact with furniture. Our key observation is that human activity plays a critical role and sufficient free space is essential for human-scene interactions. This is exactly where many existing synthesized scenes fail—the seemingly correct layouts are often not fit for living. To tackle this, we present a human-aware optimization framework Haisor for 3D indoor scene arrangement via reinforcement learning, which aims to find an action sequence to optimize the indoor scene layout automatically. Based on the hierarchical scene graph representation, an optimal action sequence is predicted and performed via Deep Q-Learning with Monte Carlo Tree Search (MCTS), where MCTS is our key feature to search for the optimal solution in long-term sequences and large action space. Multiple human-aware rewards are designed as our core criteria of human-scene interaction, aiming to identify the next smart action by leveraging powerful reinforcement learning. Our framework is optimized end-to-end by giving the indoor scenes with part-level furniture layout including part mobility information. Furthermore, our methodology is extensible and allows utilizing different reward designs to achieve personalized indoor scene synthesis. Extensive experiments demonstrate that our approach optimizes the layout of 3D indoor scenes in a human-aware manner, which is more realistic and plausible than original state-of-the-art generator results, and our approach produces superior smart actions, outperforming alternative baselines.

HKTSG: A Hierarchical Knowledge-guided Traffic Scene Graph Representation Learning Framework for Intelligent Vehicles

HKTSG: A Hierarchical Knowledge-guided Traffic Scene Graph Representation Learning Framework for Intelligent Vehicles

Hierarchical graph representation learning with multi-granularity features for anti-cancer drug response prediction.

Patients with the same type of cancer often respond differently to identical drug treatments due to unique genomic traits. Accurately predicting a patient's response to drug is crucial in guiding treatment decisions, alleviating patient suffering, and improving cancer prognosis. Current computational methods utilize deep learning models trained on extensive drug screening data to predict anti-cancer drug responses based on features of cell lines and drugs. However, the interaction between cell lines and drugs is a complex biological process involving interactions across various levels, from internal cellular and drug structures to the external interactions among different molecules.To address this complexity, we propose a novel Hierarchical graph representation Learning with Multi-Granularity features (HLMG) algorithm for predicting anti-cancer drug responses. The HLMG algorithm combines features at two granularities: the overall gene expression and pathway substructures of cell lines, and the overall molecular fingerprints and substructures of drugs. Subsequently, it constructs a heterogeneous graph including cell lines, drugs, known cell line-drug responses, and the associations between similar cell lines and similar drugs. Through a graph convolutional network model, the HLMG learns the final cell line and drug representations by aggregating features of their multi-level neighbor in the heterogeneous graph. The multi-level neighbors consist of the node self, directly related drugs/cell lines, and indirectly related similar drugs/cell lines. Finally, a linear correlation coefficient decoder is employed to reconstruct the cell line-drug correlation matrix to predict anti-cancer drug responses. Our model was tested on the Genomics of Drug Sensitivity in Cancer (GDSC) and the Cancer Cell Line Encyclopedia (CCLE) databases. Results indicate that HLMG outperforms other state-of-the-art methods in accurately predicting anti-cancer drug responses.

A Hierarchical Graph V-Net With Semi-Supervised Pre-Training for Histological Image Based Breast Cancer Classification.

Numerous patch-based methods have recently been proposed for histological image based breast cancer classification. However, their performance could be highly affected by ignoring spatial contextual information in the whole slide image (WSI). To address this issue, we propose a novel hierarchical Graph V-Net by integrating 1) patch-level pre-training and 2) context-based fine-tuning, with a hierarchical graph network. Specifically, a semi-supervised framework based on knowledge distillation is first developed to pre-train a patch encoder for extracting disease-relevant features. Then, a hierarchical Graph V-Net is designed to construct a hierarchical graph representation from neighboring/similar individual patches for coarse-to-fine classification, where each graph node (corresponding to one patch) is attached with extracted disease-relevant features and its target label during training is the average label of all pixels in the corresponding patch. To evaluate the performance of our proposed hierarchical Graph V-Net, we collect a large WSI dataset of 560 WSIs, with 30 labeled WSIs from the BACH dataset (through our further refinement), 30 labeled WSIs and 500 unlabeled WSIs from Yunnan Cancer Hospital. Those 500 unlabeled WSIs are employed for patch-level pre-training to improve feature representation, while 60 labeled WSIs are used to train and test our proposed hierarchical Graph V-Net. Both comparative assessment and ablation studies demonstrate the superiority of our proposed hierarchical Graph V-Net over state-of-the-art methods in classifying breast cancer from WSIs. The source code and our annotations for the BACH dataset have been released at https://github.com/lyhkevin/Graph-V-Net.

HG-CAD: Hierarchical Graph Learning for Material Prediction and Recommendation in Computer-Aided Design

Abstract To support intelligent computer-aided design (CAD), we introduce a machine learning architecture, namely HG-CAD, that recommends assembly body material through joint learning of body- and assembly-level features using a hierarchical graph representation. Specifically, we formulate the material prediction and recommendation process as a node-level classification task over a novel hierarchical graph representation of CAD models, with a low-level graph capturing the body geometry, a high-level graph representing the assembly topology, and a batch-level mask randomization enabling contextual awareness. This enables our network to aggregate geometric and topological features from both the body and assembly levels, leading to competitive performance. Qualitative and quantitative evaluation of the proposed architecture on the Fusion 360 Gallery Assembly Dataset demonstrates the feasibility of our approach, outperforming selected computer vision and human baselines while showing promise in application scenarios. The proposed HG-CAD architecture that unifies the processing, encoding, and joint learning of multi-modal CAD features indicates the potential to serve as a recommendation system for design automation and a baseline for future work.

A Structure-aware Hierarchical Graph-based Multiple Instance Learning Framework for pT Staging in Histopathological Image.

Pathological primary tumor (pT) stage focuses on the infiltration degree of the primary tumor to surrounding tissues, which relates to the prognosis and treatment choices. The pT staging relies on the field-of-views from multiple magnifications in the gigapixel images, which makes pixel-level annotation difficult. Therefore, this task is usually formulated as a weakly supervised whole slide image (WSI) classification task with the slide-level label. Existing weakly-supervised classification methods mainly follow the multiple instance learning paradigm, which takes the patches from single magnification as the instances and extracts their morphological features independently. However, they cannot progressively represent the contextual information from multiple magnifications, which is critical for pT staging. Therefore, we propose a structure-aware hierarchical graph-based multi-instance learning framework (SGMF) inspired by the diagnostic process of pathologists. Specifically, a novel graph-based instance organization method is proposed, namely structure-aware hierarchical graph (SAHG), to represent the WSI. Based on that, we design a novel hierarchical attention-based graph representation (HAGR) network to capture the critical patterns for pT staging by learning cross-scale spatial features. Finally, the top nodes of SAHG are aggregated by a global attention layer for bag-level representation. Extensive studies on three large-scale multi-center pT staging datasets with two different cancer types demonstrate the effectiveness of SGMF, which outperforms state-of-the-art up to 5.6% in the F1 score.

IHerd: an integrative hierarchical graph representation learning framework to quantify network changes and prioritize risk genes in disease.

Different genes form complex networks within cells to carry out critical cellular functions, while network alterations in this process can potentially introduce downstream transcriptome perturbations and phenotypic variations. Therefore, developing efficient and interpretable methods to quantify network changes and pinpoint driver genes across conditions is crucial. We propose a hierarchical graph representation learning method, called iHerd. Given a set of networks, iHerd first hierarchically generates a series of coarsened sub-graphs in a data-driven manner, representing network modules at different resolutions (e.g., the level of signaling pathways). Then, it sequentially learns low-dimensional node representations at all hierarchical levels via efficient graph embedding. Lastly, iHerd projects separate gene embeddings onto the same latent space in its graph alignment module to calculate a rewiring index for driver gene prioritization. To demonstrate its effectiveness, we applied iHerd on a tumor-to-normal GRN rewiring analysis and cell-type-specific GCN analysis using single-cell multiome data of the brain. We showed that iHerd can effectively pinpoint novel and well-known risk genes in different diseases. Distinct from existing models, iHerd's graph coarsening for hierarchical learning allows us to successfully classify network driver genes into early and late divergent genes (EDGs and LDGs), emphasizing genes with extensive network changes across and within signaling pathway levels. This unique approach for driver gene classification can provide us with deeper molecular insights. The code is freely available at https://github.com/aicb-ZhangLabs/iHerd. All other relevant data are within the manuscript and supporting information files.

HimGNN: a novel hierarchical molecular graph representation learning framework for property prediction.

Accurate prediction of molecular properties is an important topic in drug discovery. Recent works have developed various representation schemes for molecular structures to capture different chemical information in molecules. The atom and motif can be viewed as hierarchical molecular structures that are widely used for learning molecular representations to predict chemical properties. Previous works have attempted to exploit both atom and motif to address the problem of information loss in single representation learning for various tasks. To further fuse such hierarchical information, the correspondence between learned chemical features from different molecular structures should be considered. Herein, we propose a novel framework for molecular property prediction, called hierarchical molecular graph neural networks (HimGNN). HimGNN learns hierarchical topology representations by applying graph neural networks on atom- and motif-based graphs. In order to boost the representational power of the motif feature, we design a Transformer-based local augmentation module to enrich motif features by introducing heterogeneous atom information in motif representation learning. Besides, we focus on the molecular hierarchical relationship and propose a simple yet effective rescaling module, called contextual self-rescaling, that adaptively recalibrates molecular representations by explicitly modelling interdependencies between atom and motif features. Extensive computational experiments demonstrate that HimGNN can achieve promising performances over state-of-the-art baselines on both classification and regression tasks in molecular property prediction.

Cognitive Path Planning with Spatial Memory Distortion.

Human path-planning operates differently from deterministic AI-based path-planning algorithms due to the decay and distortion in a human's spatial memory and the lack of complete scene knowledge. Here, we present a cognitive model of path-planning that simulates human-like learning of unfamiliar environments, supports systematic degradation in spatial memory, and distorts spatial recall during path-planning. We propose a Dynamic Hierarchical Cognitive Graph (DHCG) representation to encode the environment structure by incorporating two critical spatial memory biases during exploration: categorical adjustment and \sequence order effect. We then extend the '`Fine-To-Coarse'' (FTC), the most prevalent path-planning heuristic, to incorporate spatial uncertainty during recall through the DHCG. We conducted a lab-based Virtual Reality (VR) experiment to validate the proposed cognitive path-planning model and made three observations: (1) a statistically significant impact of sequence order effect on participants' route-choices, (2) approximately three hierarchical levels in the DHCG according to participants' recall data, and (3) similar trajectories and significantly similar wayfinding performances between participants and simulated cognitive agents on identical path-planning tasks. Furthermore, we performed two detailed simulation experiments with different FTC variants on a Manhattan-style grid. Experimental results demonstrate that the proposed cognitive path-planning model successfully produces human-like paths and can capture human wayfinding's complex and dynamic nature, which traditional AI-based path-planning algorithms cannot capture.

Association Hierarchical Representation Learning for Plant-Wide Process Monitoring by Using Multilevel Knowledge Graph

In order to satisfy safety requirements of plant-wide processes, distributed process monitoring methods are often used. However, few of them consider the problem on building multi-level knowledge blocks and the associations between different blocks, all of which are beneficial for plant-wide process monitoring in avoiding information conflict and even to improve monitoring accuracy. To handle these issues, a plant-wide process monitoring method is proposed, which is based on hierarchical graph representation learning with differentiable pooling by using multi-level knowledge graph (MLKG). Specifically, MLKG consists of devices level, subprocess level, process level, etc. Each level has numerous blocks (nodes) which is firstly constructed by priori-knowledge on monitoring variables to calculate the status of key components, such as the Hotelling's statistics. And then normalized mutual information (NMI) is used to obtain the associations between monitoring variables, and the status of each block on each level can be updated. Based on this method, MLKG can be completely constructed. In order to consider the association information of hierarchical representations of MLKG, the hierarchical graph representation learning is used to achieve plant-wide process monitoring. Results of case study on practical cobalt and nickel removal from zinc solution demonstrate the effectiveness and applicability of this method.

Learning Hierarchical Spatial-Temporal Graph Representations for Robust Multivariate Industrial Anomaly Detection

Multivariate time series anomaly detection is one of the most indispensable yet troublesome links in complex industrial processes. The main challenge lies in discovering the representative patterns for collective or contextual anomalies among interconnected sensory data streams, which has been largely hampered by inefficient spatial-temporal feature extraction and suboptimal decision criteria under the scarcity of positive training samples. This article goes beyond the common limitations of the existing methods, and novelly proposes Hierarchical Spatial-Temporal grAph Representation (HiSTAR). It processes the data with strong structural inductive biases through latent spatial-temporal graph modeling, yet requiring no pre-defined topological priors for the sensor network. A discriminative decision boundary is constructed by learning hierarchical normality-enclosing hyperspheres on the produced graph-structure representations. In this way, HiSTAR not only presents superior anomaly detection performance, but also provides consistent anomaly localization results. The efficacy of the proposed method is experimentally corroborated through three industrial case studies.

Air Traffic Complexity Evaluation with Hierarchical Graph Representation Learning

Air traffic management (ATM) relies on the running condition of the air traffic control sector (ATCS), and assessing whether it is overloaded is crucial for efficiency and safety for the entire aviation industry. Previous approaches to evaluating air traffic complexity in a sector were mostly based on aircraft operational status and lacked comprehensiveness of characterization and were less adaptable in real situations. To settle these issues, a deep learning technique grounded on complex networks was proposed, employing the flight conflict network (FCN) to generate an air traffic situation graph (ATSG), with the air traffic control instruction (ATCOI) received by each aircraft included as an extra node attribute to increase the accuracy of the evaluation. A pooling method with a graph neural network (GNN) was used to analyze the graph-structured air traffic information and produce the sector complexity rank automatically. The model Hierarchical Graph Representing Learning (HGRL) was created to build comprehensive feature representations which involve two parts: graph structure coarsening and graph attribute learning. Structure coarsening reduced the feature map size by choosing an adaptive selection of nodes, while attribute coarsening selected key nodes in the graph-level representation. The experimental findings of a real dataset from the Chinese aviation industry reveal that our proposed model exceeds prior methods in its ability to extract critical information from an ATSG. Moreover, our work could be applied in the two main types of sectors and without extra factor calculations to determine the complexity of the airspace.

Hierarchical graph multi-agent reinforcement learning for traffic signal control

Multi-agent reinforcement learning (MARL) is a promising algorithm for traffic signal control (TSC), and graph neural networks make a further improvement on its learning capacity. However, these state-of-the-art algorithms adopt fine-granularity information (i.e., current step-states) to get traffic-network embeddings, but ignore great-granularity information (e.g., previous multiple step-states); In addition, these algorithms are optimized by loss functions in the viewpoint of MARL, but ignore the correlation between input information and output embeddings. This paper proposes a Hierarchical Graph Multi-agent Mutual Information (HG-M2I) algorithm for TSC. Specifically, HG-M2I algorithm is developed based on: 1) Multi-granularity fusion employs a proposed Hierarchical Graph representation learning (HG) algorithm, which fuses both current step-states of multiple agents and previous multiple step-states of each agent, thus facilitates to produce optimal final embeddings; 2) Joint optimization employs a proposed Multi-agent Mutual Information (M2I) framework, which measures the correlation between input step-states and output embeddings by maximizing mutual information, the corresponding mutual-information and MARL loss jointly optimize the whole algorithm, thus facilitates to derive optimal traffic-signal policies. HG-M2I is compared against these state-of-the-art algorithms on synthetic-road and real-road datasets. Experimental results not only show the superior performance of HG-M2I over other compared algorithms, but also illustrate its advantageous transferability.

Fragment-based deep molecular generation using hierarchical chemical graph representation and multi-resolution graph variational autoencoder.

Graph generative models have recently emerged as an interesting approach to construct molecular structures atom-by-atom or fragment-by-fragment. In this study, we adopt the fragment-based strategy and decompose each input molecule into a set of small chemical fragments. In drug discovery, a few drug molecules are designed by replacing certain chemical substituents with their bioisosteres or alternative chemical moieties. This inspires us to group decomposed fragments into different fragment clusters according to their local structural environment around bond-breaking positions. In this way, an input structure can be transformed into an equivalent three-layer graph, in which individual atoms, decomposed fragments, or obtained fragment clusters act as graph nodes at each corresponding layer. We further implement a prototype model, named multi-resolution graph variational autoencoder (MRGVAE), to learn embeddings of constituted nodes at each layer in a fine-to-coarse order. Our decoder adopts a similar but conversely hierarchical structure. It first predicts the next possible fragment cluster, then samples an exact fragment structure out of the determined fragment cluster, and sequentially attaches it to the preceding chemical moiety. Our proposed approach demonstrates comparatively good performance in molecular evaluation metrics compared with several other graph-based molecular generative models. The introduction of the additional fragment cluster graph layer will hopefully increase the odds of assembling new chemical moieties absent in the original training set and enhance their structural diversity. We hope that our prototyping work will inspire more creative research to explore the possibility of incorporating different kinds of chemical domain knowledge into a similar multi-resolution neural network architecture.

Hierarchical graph representation for unsupervised crop row detection in images

Crop row detection is an important aspect of smart farming. An accurate method of crop row detection ensures better navigation of the robot in the field as well as accurate weed control between rows, and crop monitoring. Recent deep learning approaches have emerged as the indispensable methods for most computer vision tasks. Therefore, crop row detection using neural networks is currently the most frequently studied approach. However, these methods require a large amount of labeled data, which is not suitable for many situations such as agriculture, where labeling data is tedious and expensive. Unsupervised techniques such as graph-based represent a promising alternative to tackle such a problem. Indeed, the crop rows represent relationships between plants (relative position, co-occurrence...), that could be integrated as structured information with an unsupervised approach. However, little attention has been paid to graph-based techniques for crop row structures representation. In this paper we propose a new method based on hierarchical approach and unsupervised graph representation for crop row detection. The idea is to transform the field into a structured data where each plant can be linked to its neighbors according to their spatial relationships. Then, with a hierarchical clustering and a criterion of exploring the nodes of the graph we extract subgraphs that represent the aligned structures. We show that the graph matching technique can discard inconsistent structures such as weed aggregations. The results obtained show the validity of the proposed method, with higher performances.

Hierarchical graph representation learning for the prediction of drug-target binding affinity

Computationally predicting drug-target binding affinity (DTA) has attracted increasing attention due to its benefit for accelerating drug discovery. Currently, numerous deep learning-based prediction models have been proposed, often with a biencoder architecture that commonly focuses on how to extract expressive representations for drugs and targets but overlooks modeling explicit drug-target interactions. However, known DTA can provide underlying knowledge about how the drugs interact with targets that is beneficial for predictive accuracy. In this paper, we propose a novel hierarchical graph representation learning model for DTA prediction, named HGRL-DTA. The main contribution of our model is to establish a hierarchical graph learning architecture to integrate the coarse- and fine-level information from an affinity graph and drug/target molecule graphs, respectively, in a well-designed coarse-to-fine manner. In addition, we design a similarity-based representation inference method to infer coarse-level information when it is unavailable for new drugs or targets under the cold start scenario. Comprehensive experimental results under four scenarios across two benchmark datasets indicate that HGRL-DTA outperforms the state-of-the-art models in almost all cases.

GraMMy: Graph representation learning based on micro–macro analysis

Graph Neural Networks (GNNs) are robust variants of deep network models, typically designed to learn from graph-structured data. Despite the recent advancement of GNNs, the basic message passing scheme of learning often holds back these models in effectively capturing the influence of the nodes from higher order neighbourhood. Further, the state-of-the-art approaches mostly ignore the contextual significance of the paths through which the message/information propagates to a node. In order to deal with these two issues, we propose GraMMY as a novel framework for hierarchical semantics-driven graph representation learning based on Micro-Macro analysis. The key idea here is to study the graph structure from different levels of abstraction, which not only provides an opportunity for flexible flow of information from both local and higher-order neighbours but also helps in more concretely capturing how information travels within various hierarchical structures of the graph. We incorporate the knowledge gained from micro and macro level semantics into the embedding of a node and use this to perform graph classification. Experimentations on four bio-informatics and two social datasets exhibit the superiority of GraMMy over state-of-the-art GNN-based graph classifiers.

CommPOOL: An interpretable graph pooling framework for hierarchical graph representation learning

Recent years have witnessed the emergence and flourishing of hierarchical graph pooling neural networks (HGPNNs) which are effective graph representation learning approaches for graph level tasks such as graph classification. However, current HGPNNs do not take full advantage of the graph’s intrinsic structures (e.g., community structure). Moreover, the pooling operations in existing HGPNNs are difficult to be interpreted. In this paper, we propose a new interpretable graph pooling framework — CommPOOL, that can capture and preserve the hierarchical community structure of graphs in the graph representation learning process. Specifically, the proposed community pooling mechanism in CommPOOL utilizes an unsupervised approach for capturing the inherent community structure of graphs in an interpretable manner. CommPOOL is a general and flexible framework for hierarchical graph representation learning that can further facilitate various graph-level tasks. Evaluations on five public benchmark datasets and one synthetic dataset demonstrate the superior performance of CommPOOL in graph representation learning for graph classification compared to the state-of-the-art baseline methods, and its effectiveness in capturing and preserving the community structure of graphs.