Message Passing Research Articles

Common Neighbor Completion with Information Entropy for Link Prediction in Social Networks

Abstract Link prediction is essential for identifying hidden relationships within network data, with significant implications for fields such as social network analysis and bioinformatics. Traditional methods often overlook potential relationships among common neighbors, limiting their effectiveness in utilizing graph information fully. To address this, we introduce a novel approach, Common Neighbor Completion with Information Entropy (IECNC), which enhances model expressiveness by considering logical neighbor relationships. Our method integrates a dynamic node function with a Message Passing Neural Network (MPNN), focusing on first-order neighbors and employing set-based aggregation to improve missing link predictions. By combining the information entropy of probabilistic predictions of common neighbors with MPNN and leveraging information entropy to assess uncertainty in adjacent connections, our approach significantly enhances prediction accuracy. Experimental results demonstrate that our IECNC method achieves optimal performance across multiple datasets, surpassing existing techniques. Furthermore, visualizations confirm that our model effectively captures and accurately learns feature information from various categories, Demonstrating the method’s efficacy and adaptability.

Adaptive Neural Message Passing for Inductive Learning on Hypergraphs.

Graphs are the most ubiquitous data structures for representing relational datasets and performing inferences in them. They model, however, only pairwise relations between nodes and are not designed for encoding the higher-order relations. This drawback is mitigated by hypergraphs, in which an edge can connect an arbitrary number of nodes. Most hypergraph learning approaches convert the hypergraph structure to that of a graph and then deploy existing geometric deep learning methods. This transformation leads to information loss, and sub-optimal exploitation of the hypergraph's expressive power. We present HyperMSG, a novel hypergraph learning framework that uses a modular two-level neural message passing strategy to accurately and efficiently propagate information within each hyperedge and across the hyperedges. HyperMSG adapts to the data and task by learning an attention weight associated with each node's degree centrality. Such a mechanism quantifies both local and global importance of a node, capturing the structural properties of a hypergraph. HyperMSG is inductive, allowing inference on previously unseen nodes. Further, it is robust and outperforms state-of-the-art hypergraph learning methods on a wide range of tasks and datasets. Finally, we demonstrate the effectiveness of HyperMSG in learning multimodal relations through detailed experimentation on a challenging multimedia dataset.

Unitary Approximate Message Passing Detector for OTSM System Based on Walsh-Hadamard Transform in LEO Satellite Communications

Unitary Approximate Message Passing Detector for OTSM System Based on Walsh-Hadamard Transform in LEO Satellite Communications

PopCluster: A Population Genetics Model-Based Toolset for Simulating, Inferring and Visualising Individual Admixture and Population Structure.

In this computer note I introduce software, PopCluster, that implements a new likelihood method for unsupervised population structure analysis from marker data. To infer a coarse population structure, it assumes the mixture model and adopts a simulated annealing algorithm to make a maximum likelihood clustering analysis, partitioning the sampled individuals into a predefined number of clusters. To deduce a fine population structure, it further assumes the admixture model and employs an expectation maximisation algorithm to estimate individual admixture proportions. PopCluster has many features. First, it is one of just a couple of model-based methods that can handle both biallelic and multiallelic markers in the same framework. Second, it is the first population structure analysis method that uses both Message Passing Interface (MPI) and openMP to exploit multiple CPUs with both shared and distributed memories and has the capacity to handle genomic data with millions of individuals and millions of loci. Third, the algorithms for both mixture and admixture analyses are fast, rendering PopCluster favourably in computational efficiency over previous methods in analysing genomic data. Fourth, PopCluster is built for Windows, Linux and Mac platforms, and its Windows version has an integrated GUI that can conveniently visualise analysis results and facilitate data input. Fifth, its Windows version has a built-in simulation module designed to simulate genotype data under admixture, hybridization or migration models. PopCluster provides a valuable toolset for researchers to simulate, infer and visualise individual admixture and population genetic structure, hybridization and migration using marker data.

Parallelization of particle-based reaction-diffusion simulations using MPI.

Particle-based reaction-diffusion models offer a high-resolution alternative to the continuum reaction-diffusion approach, capturing the discrete and volume-excluding nature of molecules undergoing stochastic dynamics. These methods are thus uniquely capable of simulating explicit self-assembly of particles into higher-order structures like filaments, spherical cages, or heterogeneous macromolecular complexes, which are ubiquitous across living systems and in materials design. The disadvantage of these high-resolution methods is their increased computational cost. Here we present a parallel implementation of the particle-based NERDSS software using the Message Passing Interface (MPI) and spatial domain decomposition, achieving close to linear scaling for up to 96 processors in the largest simulation systems. The scalability of parallel NERDSS is evaluated for bimolecular reactions in 3D and 2D, for self-assembly of trimeric and hexameric complexes, and for protein lattice assembly from 3D to 2D, with all parallel test cases producing accurate solutions. We demonstrate how parallel efficiency depends on the system size, the reaction network, and the limiting timescales of the system, showing optimal scaling only for smaller assemblies with slower timescales. The formation of very large assemblies represents a challenge in evaluating reaction updates across processors, and here we restrict assembly sizes to below the spatial decomposition size. We provide the parallel NERDSS code open source, with detailed documentation for developers and extension to other particle-based reaction-diffusion software.

Unified Knowledge-Guided Molecular Graph Encoder with multimodal fusion and multi-task learning.

The remarkable success of Graph Neural Networks underscores their formidable capacity to assimilate multimodal inputs, markedly enhancing performance across a broad spectrum of domains. In the context of molecular modeling, considerable efforts have been made to enrich molecular representations by integrating data from diverse aspects. Nevertheless, current methodologies frequently compartmentalize geometric and semantic components, resulting in a fragmented approach that impairs the holistic integration of molecular attributes. This constrained scope limits the generalizability and efficacy of such models in downstream applications. A pivotal challenge lies in harmonizing heterogeneous data sources, particularly in addressing the inherent inconsistencies and sparsity within multimodal molecular datasets. To overcome these limitations, we present the Unified Knowledge-Guided Molecular Graph Encoder (UKGE), a groundbreaking framework that leverages heterogeneous graphs to unify the representation of diverse molecular modalities. Unlike prior methods, UKGE reconciles geometric and semantic features through the use of elemental knowledge graphs (KGs) and meta-path definitions by constructing Unified Molecular Graphs, enabling comprehensive and unified molecular representations. It employs an innovative Meta-Path Aware Message Passing mechanism within its molecular encoder, enhancing the integration of multimodal data. Additionally, a multi-task learning strategy balances data from different modalities, further enriching UKGE's capability to embed complex biological insights.Empirical evaluations highlight UKGE's excellence across tasks: DDI prediction achieves 96.91% ACC and 99.14% AUC in warm-start settings, with 83.15% ACC in cold-start scenarios. For CPI prediction, it reaches 0.644 CI on Davis and 0.659 on KIBA. In LBDD, it achieves 99.3% validity, 98.4% uniqueness, and 98.9% novelty, establishing UKGE as a state-of-the-art molecular modeling framework.

Advanced AI-Driven Prediction of Pregnancy-Related Adverse Drug Reactions.

Ensuring drug safety during pregnancy is critical due to the potential risks to both the mother and fetus. However, the exclusion of pregnant women from clinical trials complicates the assessment of adverse drug reactions (ADRs) in this population. This study aimed to develop and validate risk prediction models for pregnancy-related ADRs of drugs using advanced Machine Learning (ML) and Deep Learning (DL) techniques, leveraging real-world data from the FDA Adverse Event Reporting System. We explored three methods─Information Component, Reporting Odds Ratio, and 95% confidence interval of ROR─for classifying drugs into high-risk and low-risk categories. DL models, including Directed Message Passing Neural Networks (DMPNN), Graph Neural Networks, and Graph Convolutional Networks, were developed and compared to traditional ML models like Random Forest, Support Vector Machines, and XGBoost. Among these, the DMPNN model, which integrated molecular graph information and molecular descriptors, exhibited the highest predictive performance, particularly at the preferred term level. The model was validated against external data sets from SIDER and DailyMed, demonstrating strong generalizability. Additionally, the model was applied to assess the risk of 22 oral hypoglycemic drugs, and potential substructure alerts for pregnancy-related ADRs were identified. These findings suggest that the DMPNN model is a valuable tool for predicting ADRs in pregnant women, offering significant advancement in drug safety assessment and providing crucial insights for safer medication use during pregnancy.

Equilibria of large random Lotka-Volterra systems with vanishing species: a mathematical approach.

Ecosystems with a large number of species are often modelled as Lotka-Volterra dynamical systems built around a large interaction matrix with random part. Under some known conditions, a global equilibrium exists and is unique. In this article, we rigorously study its statistical properties in the large dimensional regime. Such an equilibrium vector is known to be the solution of a so-called Linear Complementarity Problem. We describe its statistical properties by designing an Approximate Message Passing (AMP) algorithm, a technique that has recently aroused an intense research effort in the fields of statistical physics, machine learning, or communication theory. Interaction matrices based on the Gaussian Orthogonal Ensemble, or following a Wishart distribution are considered. Beyond these models, the AMP approach developed in this article has the potential to describe the statistical properties of equilibria associated to more involved interaction matrix models.

Topology-aware fault diagnosis for microgrid clusters with diverse scenarios generated by digital twins

Microgrid clusters, characterized by their dynamically changing topologies and flexible operational modes ranging from grid-tied to autonomous functioning, present significant challenges for conventional protective measures. To address these complexities, this paper proposes a novel topology-aware fault diagnosis approach that integrates Message Passing Neural Networks (MPNNs) with a Graph-Lasso-based topology identification method. Our framework is designed to enhance the precision and adaptability of fault diagnosis in microgrid clusters, which are crucial for maintaining system stability. The proposed model demonstrates over 99% accuracy in fault localization and over 97% in fault type recognition, even under varying topological conditions and data loss scenarios. Through the application of digital twin technology, the framework not only improves the diagnostic capabilities but also supports the intelligent operation and maintenance of microgrid clusters, thereby contributing to the overall reliability and quality of their electrical systems.

An implicit GNN solver for Poisson-like problems

This paper presents Ψ-GNN, a novel Graph Neural Network (GNN) approach for solving the ubiquitous Poisson PDE problems on general unstructured meshes with mixed boundary conditions. By leveraging the Implicit Layer Theory, Ψ-GNN models an “infinitely” deep network, thus avoiding the empirical tuning of the number of required Message Passing layers to attain the solution. Its original architecture explicitly takes into account the boundary conditions, a critical pre-requisite for physical applications, and is able to adapt to any initially provided solution. Ψ-GNN is trained using a physics-informed loss, and the training process is stable by design. Furthermore, the consistency of the approach is theoretically proven, and its flexibility and generalization efficiency are experimentally demonstrated: the same learned model can accurately handle unstructured meshes of various sizes, as well as different boundary conditions. To the best of our knowledge, Ψ-GNN is the first physics-informed GNN-based method that can handle various unstructured domains, boundary conditions and initial solutions while also providing convergence guarantees.

Flow2GNN: Flexible Two-Way Flow Message Passing for Enhancing GNNs Beyond Homophily.

Message passing (MP) is crucial for effective graph neural networks (GNNs). Most local message-passing schemes have been shown to underperform on heterophily graphs due to the perturbation of updated representations caused by local redundant heterophily information. However, our experiment findings indicate that the distribution of heterophily information during MP can be disrupted by disentangling local neighborhoods. This finding can be applied to other GNNs, improving their performance on heterophily graphs in a more flexible manner compared to most heterophily GNNs with complex designs. This article proposes a new type of simple message-passing neural network called Flow2GNN. It uses a two-way flow message-passing scheme to enhance the ability of GNNs by disentangling and redistributing heterophily information in the topology space and the attribute space. Our proposed message-passing scheme consists of two steps in topology space and attribute space. First, we introduce a new disentangled operator with binary elements that disentangle topology information in-flow and out-flow between connected nodes. Second, we use an adaptive aggregation model that adjusts the flow amount between homophily and heterophily attribute information. Furthermore, we rigorously prove that disentangling in message-passing can reduce the generalization gap, offering a deeper understanding of how our model enhances other GNNs. The extensive experiment results show that the proposed model, Flow2GNN, not only outperforms state-of-the-art GNNs, but also helps improve the performance of other commonly used GNNs on heterophily graphs, including GCN, GAT, GCNII, and H2GCN, specifically for GCN, with up to a 25.88% improvement on the Wisconsin dataset.

MARS: Message Passing for Antenna and RF Chain Selection for Hybrid Beamforming in MIMO Communication Systems

MARS: Message Passing for Antenna and RF Chain Selection for Hybrid Beamforming in MIMO Communication Systems

Performance Study of an MRI Motion-Compensated Reconstruction Program on Intel CPUs, AMD EPYC CPUs, and NVIDIA GPUs

Motion-compensated image reconstruction enables new clinical applications of Magnetic Resonance Imaging (MRI), but it relies on computationally intensive algorithms. This study focuses on the Generalized Reconstruction by Inversion of Coupled Systems (GRICS) program, applied to the reconstruction of 3D images in cases of non-rigid or rigid motion. It uses hybrid parallelization with the MPI (Message Passing Interface) and OpenMP (Open Multi-Processing). For clinical integration, the GRICS needs to efficiently harness the computational resources of compute nodes. We aim to improve the GRICS’s performance without any code modification. This work presents a performance study of GRICS on two CPU architectures: Intel Xeon Gold and AMD EPYC. The roofline model is used to study the software–hardware interaction and quantify the code’s performance. For CPU–GPU comparison purposes, we propose a preliminary MATLAB–GPU implementation of the GRICS’s reconstruction kernel. We establish the roofline model of the kernel on two NVIDIA GPU architectures: Quadro RTX 5000 and A100. After the performance study, we propose some optimization patterns for the code’s execution on CPUs, first considering only the OpenMP implementation using thread binding and affinity and appropriate architecture-compilation flags and then looking for the optimal combination of MPI processes and OpenMP threads in the case of the hybrid MPI–OpenMP implementation. The results show that the GRICS performed well on the AMD EPYC CPUs, with an architectural efficiency of 52%. The kernel’s execution was fast on the NVIDIA A100 GPU, but the roofline model reported low architectural efficiency and utilization.

Approximate Message Passing with Rigorous Guarantees for Pooled Data and Quantitative Group Testing

Approximate Message Passing with Rigorous Guarantees for Pooled Data and Quantitative Group Testing

Multris: Functional Verification of Multiparty Message Passing in Separation Logic

We introduce Multris, a separation logic for verifying functional correctness of programs that combine multiparty message-passing communication with shared-memory concurrency. The foundation of our work is a novel concept of multiparty protocol consistency, which guarantees safe communication among a set of parties, provided each party adheres to its prescribed protocol. Our concept of protocol consistency is inspired by the bottom-up approach for multiparty session types. However, by considering it in the context of separation logic instead of a type system, we go further in terms of generality by supporting new notions of implicit transfer of knowledge and implicit transfer of resources. We develop tactics for automatically verifying protocol consistency and for reasoning about message-passing operations in Multris. We evaluate Multris on a range of examples, including the well-known two- and three-buyer protocols, as well as a new verification benchmark based on Chang and Roberts's ring leader election protocol. To ensure the reliability of our work, we prove soundness of Multris w.r.t. a low-level channel semantics using the Iris framework in Coq.

KAN-HyperMP: An Enhanced Fault Diagnosis Model for Rolling Bearings in Noisy Environments

Rolling bearings often produce non-stationary signals that are easily obscured by noise, particularly in high-noise environments, making fault detection a challenging task. To address this challenge, a novel fault diagnosis approach based on the Kolmogorov–Arnold Network-based Hypergraph Message Passing (KAN-HyperMP) model is proposed. The KAN-HyperMP model is composed of three key components: a neighbor feature aggregation block, a feature fusion block, and a KANLinear block. Firstly, the neighbor feature aggregation block leverages hypergraph theory to integrate information from more distant neighbors, aiding in the reduction of noise impact, even when nearby neighbors are severely affected. Subsequently, the feature fusion block combines the features of these higher-order neighbors with the target node’s own features, enabling the model to capture the complete structure of the hypergraph. Finally, the smoothness properties of B-spline functions within the Kolmogorov–Arnold Network (KAN) are employed to extract critical diagnostic features from noisy signals. The proposed model is trained and evaluated on the Southeast University (SEU) and Jiangnan University (JNU) Datasets, achieving accuracy rates of 99.70% and 99.10%, respectively, demonstrating its effectiveness in fault diagnosis under both noise-free and noisy conditions.

Hierarchical Graph Neural Network: A Lightweight Image Matching Model with Enhanced Message Passing of Local and Global Information in Hierarchical Graph Neural Networks

Graph Neural Networks (GNNs) have gained popularity in image matching methods, proving useful for various computer vision tasks like Structure from Motion (SfM) and 3D reconstruction. A well-known example is SuperGlue. Lightweight variants, such as LightGlue, have been developed with a focus on stacking fewer GNN layers compared to SuperGlue. This paper proposes the h-GNN, a lightweight image matching model, with improvements in the two processing modules, the GNN and matching modules. After image features are detected and described as keypoint nodes of a base graph, the GNN module, which primarily aims at increasing the h-GNN’s depth, creates successive hierarchies of compressed-size graphs from the base graph through a clustering technique termed SC+PCA. SC+PCA combines Principal Component Analysis (PCA) with Spectral Clustering (SC) to enrich nodes with local and global information during graph clustering. A dual non-contrastive clustering loss is used to optimize graph clustering. Additionally, four message-passing mechanisms have been proposed to only update node representations within a graph cluster at the same hierarchical level or to update node representations across graph clusters at different hierarchical levels. The matching module performs iterative pairwise matching on the enriched node representations to obtain a scoring matrix. This matrix comprises scores indicating potential correct matches between the image keypoint nodes. The score matrix is refined with a ‘dustbin’ to further suppress unmatched features. There is a reprojection loss used to optimize keypoint match positions. The Sinkhorn algorithm generates a final partial assignment from the refined score matrix. Experimental results demonstrate the performance of the proposed h-GNN against competing state-of-the-art (SOTA) GNN-based methods on several image matching tasks under homography, estimation, indoor and outdoor camera pose estimation, and 3D reconstruction on multiple datasets. Experiments also demonstrate improved computational memory and runtime, approximately 38.1% and 26.14% lower than SuperGlue, and an average of about 6.8% and 7.1% lower than LightGlue. Future research will explore the effects of integrating more recent simplicial message-passing mechanisms, which concurrently update both node and edge representations, into our proposed model.

A hierarchical GNN across semantic and topological domains for predicting circRNA-microRNA interactions

Identifying circRNA-microRNA interactions (CMI) is a significant biomedical issue in recent years. This problem provides insights into using circRNA as biomarkers, developing cancer therapies and producing cancer vaccines. Using computational methods for identification is a more time-efficient and cost-effective approach. In computational methods, using graphs to represent and explore the CMI is a mainstream approach. However, existing relevant methods do not achieve optimal results by utilizing both the semantic information extracted from sequences and the topological information extracted from graph structures. To address this issue, we propose HGLMALLM, a graph contrastive learning method that learns node representation crossing both the semantic domain generated via motif-aware pre-trained LLMs and the topological domain extracted from hierarchical graph structures. Our method effectively addresses the issue in existing Message Passing Neural Network (MPNN) method that edge components losing heterogeneity after multiple iterations. Moreover, this method utilizes the heterogeneity of graph which is extended from the traditional bipartite graph to heterogeneous through the semantic domain. Two commonly used datasets were partitioned based on the distribution of node degrees. Then, we benchmarked our method against existing methods. In the independent testing set evaluation, it achieved a 3 % and 1 % improvement on two datasets. Our method demonstrated the best stability in ten-fold cross-validation on the training set. A test conducted on the peripheral components reveals robust performance of our model. A dataset collected from real scenarios was used to demonstrate the strong predictive ability of our method for identifying unidentified CMI.

Accelerating Fortran codes: A method for integrating Coarray Fortran with CUDA Fortran and OpenMP

Fortran's prominence in scientific computing requires strategies to ensure both that legacy codes are efficient on high-performance computing systems, and that the language remains attractive for the development of new high-performance codes. Coarray Fortran (CAF), part of the Fortran 2008 standard introduced for parallel programming, facilitates distributed memory parallelism with a syntax familiar to Fortran programmers, simplifying the transition from single-processor to multi-processor coding. This research focuses on innovating and refining a parallel programming methodology that fuses the strengths of Intel Coarray Fortran, Nvidia CUDA Fortran, and OpenMP for distributed memory parallelism, high-speed GPU acceleration and shared memory parallelism respectively. We consider the management of pageable and pinned memory, CPU-GPU affinity in NUMA multiprocessors, and robust compiler interfacing with speed optimisation. We demonstrate our method through its application to a parallelised Poisson solver and compare the methodology, implementation, and scaling performance to that of the Message Passing Interface (MPI), finding CAF offers similar speeds with easier implementation. For new codes, this approach offers a faster route to optimised parallel computing. For legacy codes, it eases the transition to parallel computing, allowing their transformation into scalable, high-performance computing applications without the need for extensive re-design or additional syntax.

Long railway track modelling – A parallel computing approach

This paper presents the development of a dynamics model for long track sections. It is based on an established short track model that utilises the Finite Element Method to describe rails and block models to describe sleepers, ballast and subballast. By implementing a parallel computing method, this innovation enables the construction of a true long track model: by segmenting the long track into shorter segments that are easier to compute. The model facilitates simulations to be run in parallel, thereby permitting simultaneous calculations of various numerical track variables. The model employs a Message Passing Interface framework to seamlessly link the track segments, handling the flow of data among the computing cores designated to each subdivided section. This strategic framework allows the long track model with the capability to simulate tracks of virtually any length, with the only constraints being the available computational resources and time. The claimed contribution about modelling capability is verified using two case studies on a 6km-long track involving different practical and conceptual train operational scenarios: emergency braking and constant braking force with constant train speed. These case studies show the flexibility and scalability of the method and its capability to handle complex track dynamic systems.