Beyond GNNs: a methodological benchmark of feature efficiency for link prediction in sparse developer networks

LightGCAN: A lightweight graph convolutional attention network for user preference modeling and personalized recommendation

The n-octanol/buffer solution distribution coefficient at pH = 7.4 (log D7.4) is an indicator of lipophilicity, and it influences a wide variety of absorption, distribution, metabolism, excretion, and toxicity (ADMET) properties and druggability of compounds. In log D7.4 prediction, graph neural networks (GNNs) can uncover subtle structure-property relationships (SPRs) by automatically extracting features from molecular graphs that facilitate the learning of SPRs, but their performances are often limited by the small size of available datasets. Herein, we present a transfer learning strategy called pretraining on computational data and then fine-tuning on experimental data (PCFE) to fully exploit the predictive potential of GNNs. PCFE works by pretraining a GNN model on 1.71 million computational log D data (low-fidelity data) and then fine-tuning it on 19,155 experimental log D7.4 data (high-fidelity data). The experiments for three GNN architectures (graph convolutional network (GCN), graph attention network (GAT), and Attentive FP) demonstrated the effectiveness of PCFE in improving GNNs for log D7.4 predictions. Moreover, the optimal PCFE-trained GNN model (cx-Attentive FP, Rtest2 = 0.909) outperformed four excellent descriptor-based models (random forest (RF), gradient boosting (GB), support vector machine (SVM), and extreme gradient boosting (XGBoost)). The robustness of the cx-Attentive FP model was also confirmed by evaluating the models with different training data sizes and dataset splitting strategies. Therefore, we developed a webserver and defined the applicability domain for this model. The webserver (http://tools.scbdd.com/chemlogd/) provides free log D7.4 prediction services. In addition, the important descriptors for log D7.4 were detected by the Shapley additive explanations (SHAP) method, and the most relevant substructures of log D7.4 were identified by the attention mechanism. Finally, the matched molecular pair analysis (MMPA) was performed to summarize the contributions of common chemical substituents to log D7.4, including a variety of hydrocarbon groups, halogen groups, heteroatoms, and polar groups. In conclusion, we believe that the cx-Attentive FP model can serve as a reliable tool to predict log D7.4 and hope that pretraining on low-fidelity data can help GNNs make accurate predictions of other endpoints in drug discovery.

Marine and coastal ecosystems (MCEs) are vital to human well-being, playing a significant role in climate regulation, carbon sequestration, while protecting coastal areas from sea level rise and erosion. However, these ecosystems are increasingly threatened by the combined effects of anthropogenic stressors (e.g., pollution) and climate change-related pressures (e.g., rising sea temperatures and ocean acidification).  Cumulative impacts arising from this complex interplay threaten MCEs' ability to deliver critical ecosystem services, compromising their health and resilience.Machine Learning (ML) has emerged as a valuable tool for assessing ecological conditions under multiple pressures. Algorithms like Random Forest (RF) and Support Vector Machine (SVM) have demonstrated their effectiveness in identifying patterns and predicting changes in ecosystem health. However, these models often fail to account for spatial dependencies between data points, which are crucial for understanding the interconnected nature of marine environments. Graph Neural Networks (GNNs), a more recent advancement in ML, overcome this limitation by explicitly modelling spatial relationships, making them highly suitable for analysing complex MCE dynamics.This study explores the application of GNN-based models to assess the impact of multiple pressures on seagrass ecosystems in the Italian coastal areas. To this aim, a comprehensive dataset was constructed, including key variables influencing seagrass health, such as nutrient concentrations, temperature, and salinity, derived from open-source platforms (e.g., Copernicus CMEMS, EMODnet). Data were synthesized into a 4km raster grid, with each pixel representing seagrass presence or absence. GNNs were constructed by considering each pixel as a node and connecting it to neighbouring pixels to capture spatial relationships. Experiments evaluated different GNN architectures, such as Graph Convolutional Networks (GCNs) and Graph Attention Networks (GATs), alongside traditional ML models like RF, SVM, and Multi-Layer Perceptron.The results showed that GNNs outperformed traditional models in terms of F1-score and accuracy, particularly in spatially complex scenarios. Traditional models often misclassified regions with intricate spatial dependencies, such as boundaries between seagrass patches, whereas GNNs demonstrated superior capability in leveraging spatial context. Despite these advantages, the study faced challenges due to the limited availability of high-resolution, temporal datasets, constraining the full exploration of dynamic ecosystem processes. However, by addressing the challenge of spatial resolution in ecological data, GNNs represents a transformative approach to understanding ocean dynamics. Their integration into a Digital Twin of the Ocean has the potential to transform ecosystem management and significantly advance coastal resilience efforts. This framework would enable detailed simulations and predictions of processes like ocean currents, extreme weather events, and the cumulative impacts of climate change and human activities. Moreover, the combination of GNNs and Digital Twins would provide deeper insights into the complex interplay of factors shaping marine and coastal ecosystems ecological state and processes and their resilience overall. This synergy empowers scientists and policymakers with actionable intelligence, fostering effective decision-making and the development of strategies to mitigate ocean hazards, while safeguarding biodiversity and enhancing the resilience of coastal communities. As future efforts move towards incorporating high-resolution data, this integrated approach holds promise for advancing the sustainable management of MCEs globally.

Recently, there has been growing interest in hyperspectral images (HSIs) classification tasks, with both Graph Neural Networks (GNN) and Convolutional Neural Networks (CNN) proving to be effective means of analysis. GNN can better capture the spatial structure of HSIs in large target irregular regions through superpixel segmentation, while CNN can refine classification tasks by processing pixel-level features in small target regular regions. However, neither GNN nor CNN models alone can simultaneously consider superpixel-level and pixel-level features to cover both large and small target regions. To fully utilize the strengths of GNN and CNN, we propose a novel model called the Graph-Polarized Fusion Network (GPF). The GPF consists of two branches: the Fusion Graph Neural Network (FGNN) classifier in the GNN branch conducts feature learning on large, irregular target regions using both Graph Convolutional Network (GCN) and Graph Attention Network (GAT) as feature extraction operators. The features are integrated using three aggregators, namely Min, Max, and Weighted Add, followed by updating the nodes through 2D convolutional layers. The Polarized Neural Network (PNN) classifier of the CNN branch primarily works on small, target regular regions using Polarized Self-Attention (PSA) to conduct high-resolution processing on the two dimensions of space and channel without increasing time loss. Additionally, GPF employs residual connections to extract features from long distances and multi-angles. It also uses weighted fusion to integrate the superpixel-level and pixel-level features obtained from the two branches. Rigorous experiments on five real datasets demonstrate that GPF can fully mine the latent features of HSIs, achieving competitive results compared with other state-of-the-art methods.

Graph mining has proven to be extremely useful in analysing features and properties of real-world graphs. This enables a number of tasks including the prediction and evaluation of how information varies with changes in the link structure, generating and building models to extract properties such as link prediction, node classification and recommendation, cluster and community detection, etc. Deep learning techniques have become the prevalent approach for graph mining as they are able to take advantage of the increasing availability of graph data. Traditional deep learning models pre-process the graph data by mapping the nodes to a real vector, which can desecrate important node relationships in the graphs. Training on such pre-processed structures with the traditional deep learning models may present extremely unstable and inaccurate results. To overcome these issues, Graph Neural Network (GNN) was introduced to handle the non-Euclidean structure of the graphs for many of the classifier and regressor tasks. However, state-of-the-art GNNs require a large amount of training data, which incur an enormous burden of labelled data annotations for supervised learning. Also, GNNs vastly suffer from degradation with the increasing number of depth layers leading to oversmoothing during neighbourhood feature aggregation. The aim of this research is to overcome the above-mentioned challenges faced by GNNs for graph mining. We propose a method for graph data mining as a regression problem for the estimation of Eigenvector Centrality in graphs with a GNN based approach in a completely unsupervised learning environment. To achieve this, we define an Encoder-Decoder based model architecture called CUL. We show that even when trained on a small number of datasets, the model performed at least on par with the supervised methods in terms of accuracy with different types of embedding schemes like Graph Convolutional Network (GCN), GraphSAGE, and Graph Attention Network (GAT). This is reinforced by the model performance based on top-$\mathcal{N}\%$ accuracy metric, on real-world and synthetically generated datasets. CUL demonstrates at least on par or even a superior performance over existing methods. Next, we focus on the applications of GNNs in the domain of continual learning. We propose a model called GCL that learns across a sequence of tasks for node-classification in graphs under task-incremental and class-incremental settings. GCL comprises of a Reinforced Controller/RLC and an expandable GNN framework called Child Network/CN, along with a memory buffer to store the node samples. We compared our method against state-of-the-art methods in regards to average accuracy and average forgetting on four datasets with different GNNs. Our model performed better than the other methods i.e., it showed higher accuracy values as well as lower forgetting rates across the tasks in both task-incremental and class-incremental settings. Finally, we propose a method for improving few-shot node classification on graphs on any generic GNN backbone framework. We propose an uncertainty-based estimator which is modeled using a GNN that maps the scalar discrete probabilities of a GNN-classifier outputs to a continuous probability distribution. We demonstrate that the few-shot node classification accuracy improves by testing on different graph datasets. We also focus on bridging gaps between different few-shot learning methods for node classification for graphs. The performance variance between the methods is analysed and the pros/cons of each of the architectures are highlighted. This analysis aids in understanding the performance differentiator and development of better architectures for few-shot learning on graph node classification.

Graph Neural Networks (GNNs) have emerged as powerful tools for analyzing structured data, particularly in domains where relationships and interactions between entities are key. By leveraging the inherent graph structure in datasets, GNNs excel in capturing complex dependencies and patterns that traditional neural networks might miss. This advantage is especially pronounced in the field of computational biology, where the intricate connections between biological entities play a crucial role. In this context, Our work explores the application of GNNs to single-cell RNA sequencing (scRNA-seq) data, a domain characterized by complex and heterogeneous relationships. By extracting ligand-receptor (L-R) associations from LIANA and constructing Cell-Cell association networks with varying edge homophily ratios, based on L-R information, we enhance the biological relevance and accuracy of depicting cellular communication pathways. While standard GNN models like Graph Convolutional Networks (GCN), GraphSAGE, Graph Attention Networks (GAT), and MixHop often assume homophily (similar nodes are more likely to be connected), this assumption does not always hold in biological networks. To address this, we explore advanced graph neural network methods, such as Graph Convolutional Networks and Gated Bi-Kernel GNNs (GBK-GNN), that are specifically designed to handle heterophilic data. Our study spans across six diverse datasets, enabling a thorough comparison between heterophily-aware GNNs and traditional homophily-assuming models, including Multi-Layer Perceptrons, which disregards graph structure entirely. Our findings highlight the importance of considering data-specific characteristics in GNN applications, demonstrating that heterophily-focused methods can effectively decipher the complex patterns within scRNA-seq data. By integrating multi-omics data, including gene expression profiles and L-R interactions, we pave the way for more accurate and insightful analyses in computational biology, offering a more comprehensive understanding of cellular environments and interactions.

Graph Neural Networks (GNNs) have become the dominant deep learning model for learning on graph-structured data, enabling breakthroughs in fields ranging from bioinformatics to social network analysis. Yet, as any deep learning model they suffer from ‘black box’ syndrome. Their decisions making process remains largely unknown. In this work, we want to advance our understanding of GNN explainability. We evaluate the performance and stability of GNNExplainer, a widely used post-hoc interpretability method, on the simple task of random graph classification. Using three very different GNN architectures, Graph Convolutional Networks, Graph Attention Networks, and Graph Isomorphism Networks, we examine the explainability of models trained to distinguish between Erdős-Rényi and Barabási-Albert random graphs, as well as between dk-randomized variants of four real-world networks. Our results show that despite the models achieving perfect classification accuracy, feature importance values identified by GNNExplainer exhibit substantial variability across architectures, hyperparameters, and random seed values. Moreover, the extracted explanations often fail to align with theoretical expectations based on established graph properties, such as degree distributions and degree correlations. These findings indicate that explanations produced by GNNExplainer are highly model-, configuration-, and seed value-dependent, challenging its reliability for deriving general insights into GNN decision mechanisms. Our work highlights fundamental limitations in the current generation of GNN explanations using GNNExplainer and suggests the need for more stable, theoretically grounded approaches to explainability in graph-based learning.

Deep learning methods have been successfully applied to the tasks of predicting functional genomic elements such as histone marks, transcriptions factor binding sites, non-B DNA structures, and regulatory variants. Initially convolutional neural networks (CNN) and recurrent neural networks (RNN) or hybrid CNN-RNN models appeared to be the methods of choice for genomic studies. With the advance of machine learning algorithms other deep learning architectures started to outperform CNN and RNN in various applications. Graph neural network (GNN) applications improved the prediction of drug effects, disease associations, protein-protein interactions, protein structures and their functions. The performance of GNN is yet to be fully explored in genomics. Earlier we developed DeepZ approach in which deep learning model is trained on information both from sequence and omics data. Initially this approach was implemented with CNN and RNN but is not limited to these classes of neural networks. In this study we implemented the DeepZ approach by substituting RNN with GNN. We tested three different GNN architectures - Graph Convolutional Network (GCN), Graph Attention Network (GAT) and inductive representation learning network GraphSAGE. The GNN models outperformed current state-of the art RNN model from initial DeepZ realization. Graph SAGE showed the best performance for the small training set of human Z-DNA ChIP-seq data while Graph Convolutional Network was superior for specific curaxin-induced mouse Z-DNA data that was recently reported. Our results show the potential of GNN applications for the task of predicting genomic functional elements based on DNA sequence and omics data.Availability and implementation–The code is freely available at https://github.com/MrARVO/GraphZ.

Graph Neural Networks (GNNs) have shown superb performance in handling networked data, mainly attributed to their message passing and convolution process across neighbors. For most literature, the performance of GNNs is mainly reported based on noise-free data environments. No study has systematically evaluated GNNs’ performance under noise. In this article, we carry out an empirical study and theoretical analysis of four types of GNNs, including Graph Convolutional Networks (GCNs), Graph Attention Networks (GATs), Graph Contrastive Networks (GCL), and graph UniFilter under three types of noise, including attribute noise, structure noise, and label noise. Our study shows that GNNs behave tremendously differently in response to different types of noise. Overall, GAT is the most noise vulnerable and sensitive, whereas GCL is the most noise resilient. We further carry out theoretical analysis to explain the reason causing GAT to be sensitive to noise, and propose a solution to enhance its noise resilience. Our study brings in-depth firsthand knowledge of GNNs under noise for researchers and practitioners to better utilize GNNs in real-world applications.

Due to the homophily assumption in graph convolution networks (GCNs), a common consensus in the graph node classification task is that graph neural networks (GNNs) perform well on homophilic graphs but may fail on heterophilic graphs with many interclass edges. However, the previous interclass edges' perspective and related homo-ratio metrics cannot well explain the GNNs' performance under some heterophilic datasets, which implies that not all the interclass edges are harmful to GNNs. In this work, we propose a new metric based on the von Neumann entropy to reexamine the heterophily problem of GNNs and investigate the feature aggregation of interclass edges from an entire neighbor identifiable perspective. Moreover, we propose a simple yet effective Conv-Agnostic GNN framework (CAGNNs) to enhance the performance of most GNNs on the heterophily datasets by learning the neighbor effect for each node. Specifically, we first decouple the feature of each node into the discriminative feature for downstream tasks and the aggregation feature for graph convolution (GC). Then, we propose a shared mixer module to adaptively evaluate the neighbor effect of each node to incorporate the neighbor information. The proposed framework can be regarded as a plug-in component and is compatible with most GNNs. The experimental results over nine well-known benchmark datasets indicate that our framework can significantly improve performance, especially for the heterophily graphs. The average performance gain is 9.81%, 25.81%, and 20.61% compared with graph isomorphism network (GIN), graph attention network (GAT), and GCN, respectively. Extensive ablation studies and robustness analysis further verify the effectiveness, robustness, and interpretability of our framework. Code is available at https://github.com/JC-202/CAGNN.

Edge features contain important information about graphs. However, current state-of-the-art neural network models designed for graph learning, e.g. graph convolutional networks (GCN) and graph attention networks (GAT), adequately utilize edge features, especially multi-dimensional edge features. In this paper, we build a new framework for a family of new graph neural network models that can more sufficiently exploit edge features, including those of undirected or multi-dimensional edges. The proposed framework can consolidate current graph neural network models; e.g. graph convolutional networks (GCN) and graph attention networks (GAT). The proposed framework and new models have the following novelties: First, we propose to use doubly stochastic normalization of graph edge features instead of the commonly used row or symmetric normalization approches used in current graph neural networks. Second, we construct new formulas for the operations in each individual layer so that they can handle multi-dimensional edge features. Third, for the proposed new framework, edge features are adaptive across network layers. As a result, our proposed new framework and new models can exploit a rich source of graph information. We apply our new models to graph node classification on several citation networks, whole graph classification, and regression on several molecular datasets. Compared with the current state-of-the-art methods, i.e. GCNs and GAT, our models obtain better performance, which testify to the importance of exploiting edge features in graph neural networks.

Purpose Accurate prediction of construction costs is vital for effective project management and risk mitigation in the construction industry. This study aims to enhance the precision of construction cost forecasts by developing a hybrid model that integrates graph convolutional networks (GCN), graph attention networks (GAT) and multi-layer perceptrons (MLP). Design/methodology/approach A hybrid GCN-GAT-MLP approach is introduced to effectively capture the complex relationships and dependencies within construction cost data. GCN is utilized to extract local topological information, GAT employs an attention mechanism to adaptively weigh the influence of neighboring nodes and MLP captures global non-linear patterns. We also employed a gate network to dynamically allocate branch weights for feature fusion and integrated multiple loss functions, thereby enhancing the model’s generalization ability. The model’s performance is assessed by comparing it to traditional machine learning models and standalone graph neural networks (GNNs), using metrics such as root mean square error (RMSE), R², MAE and MAPE. Findings The proposed GCN-GAT-MLP hybrid model with a fused loss function exhibits outstanding predictive performance, achieving an average RMSE of 0.4490, R² of 0.9891, mean absolute error of 0.2640 and mean absolute percentage error of 0.0159 across ten independent runs using different random seeds. These metrics demonstrate substantial improvements over baseline machine learning models and standalone GNNs and highlight the superior performance of the fused loss function compared to standard loss functions. Results visualizations like density scatter plots and violin plots further confirm the model’s predictions closely align with actual values, with minimal errors, making it a robust tool for practical cost estimation. Research limitations/implications The proposed hybrid model enhances early-stage construction cost forecasting by addressing accuracy issues and incorporating structural characteristics. It supports project managers with reliable data for better decision-making, optimizes budgets and mitigates risks, while enabling stakeholders like investors and contractors to improve project outcomes through informed cost control. The model’s high accuracy and efficient training also boost resource allocation and productivity, promoting sustainable and intelligent construction practices. While effective for moderate sample sizes, the proposed hybrid model faces limitations including increased complexity with larger datasets, reduced interpretability of its predictions and challenges in efficiently processing real-time data updates. Originality/value This paper introduces a pioneering hybrid model that integrates GCN, GAT and MLP for construction cost prediction, filling a gap in the application of advanced GNNs within this field. This study innovatively employs a gate network to dynamically allocate branch weights for feature fusion and incorporates multi-loss integration, further enhancing prediction accuracy and improving the model’s generalization capability. This paper establishes a new standard for predictive modeling in construction cost estimation, with significant potential to influence future research and industry practices.

<sec><p indent="0mm">Due to the ongoing climate change, there has been extensive attention drawn to carbon capture and storage technologies, with the focus on applying ionic liquids as absorbents. Ionic liquids are widely regarded as promising in capturing CO₂, because of their unique chemical and physical properties, such as low vapor pressure, high chemical stability, and tunability. Notably, the binary mixtures of ionic liquids often outperform single-component ionic liquids in CO₂ absorption, which is attributed to the synergistic interaction between different ionic liquids. As a critical thermodynamic parameter, the Henry’s law constant (HLC) can be used to describe the solubility of a solute in a solvent under low concentration limits. Regarding CO₂ capture, the magnitude of the HLC is closely associated with the capacity of ionic liquids to absorb CO₂. When HLC decreases, solubility is improved, which means the solvent, particularly in the presence of ionic liquids, can capture a greater amount of CO₂ under a given pressure. Given a significant correlation between this enhanced absorption capacity and the efficiency of CO₂ capture, it is crucial to optimize the performance of ionic liquids in carbon capture. </sec><sec> Usually, it is a lengthy, labor-intensive, and costly process to determine HLC in the conventional ways, whether through experimental techniques or theoretical calculations. Experimental approaches require meticulous procedures and precise measurements, while theoretical calculations necessitate the consumption of extensive computational resources and time. Both methods are resource-intensive, limiting their scalability when applied to a wide range of materials. However, the advancement of machine learning technology makes it possible to predict these constants in an efficient and cost-effective way by using computational models. Trained on relevant data, machine learning models enable the rapid and accurate predictions of HLC, which significantly reduces the time cost and expense incurred by experimental or theoretical determinations. In addition to accelerating the process of identifying promising ionic liquids for CO₂ capture, this shift toward data-driven approaches also enhances the overall efficiency and practicality of carbon capture research and development. </sec><sec> In this study, an innovative framework is proposed to predict the HLC of CO₂ in ionic liquids by combining graph neural networks (GNNs) with machine learning (ML) techniques. Specifically, there are four types of GNNs used to generate molecular descriptors, namely Graph Convolutional Networks (GCN), Graph Attention Networks (GAT), Graph Isomorphism Networks (GIN), and Graph Sample and Aggregate Networks (GraphSAGE). These descriptors are integrated with various machine learning algorithms to systematically compare the performance of these combinations in HLC prediction, including K-Nearest Neighbors (KNN), Support Vector Regression (SVR), Random Forest (RF), Gradient Boosting Regression Trees (GBRT), and XGBoost (eXtreme Gradient Boosting). According to the research results, the combination of GNNs with tree-based algorithms, such as GBRT, XGBoost, and RF, leads to a significantly better performance relative to the combinations involving KNN and SVR. Among them, the GNN+GBRT model performs best on the training set, while the GNN+XGBoost model performs best in accuracy on the test set. Additionally, the molecular descriptors created by using the GCN model produce a slightly better overall performance than other GNN architectures, which evidences the high capacity of GCNs in processing complex molecular structure data. Therefore, they are applicable to build high-precision predictive models for CO₂ Henry’s law constant. </sec><sec> To sum up, the application of this GNN+ML framework in predicting CO₂ Henry’s law constant plays a vital role in optimizing CO₂ capture technologies. By improving the accuracy and efficiency of predictions, this approach contributes to the advancement of environmental technologies and sustainable engineering. </sec>

Sparse norm regularized attribute selection for graph neural networks

Graph neural networks (GNNs) have emerged as a powerful approach for modelling and learning from graph-structured data. Multiple fields have since benefitted enormously from the capabilities of GNNs, such as recommendation systems, social network analysis, drug discovery, and robotics. However, accelerating and efficiently processing GNNs require a unique approach that goes beyond conventional artificial neural network accelerators, due to the substantial computational and memory requirements of GNNs. The slowdown of scaling in CMOS platforms also motivates a search for alternative implementation substrates. In this paper, we present GHOST , the first silicon-photonic hardware accelerator for GNNs. GHOST efficiently alleviates the costs associated with both vertex-centric and edge-centric operations. It implements separately the three main stages involved in running GNNs in the optical domain, allowing it to be used for the inference of various widely used GNN models and architectures, such as graph convolution networks and graph attention networks. Our simulation studies indicate that GHOST exhibits at least 10.2 × better throughput and 3.8 × better energy efficiency when compared to GPU, TPU, CPU and multiple state-of-the-art GNN hardware accelerators.