Graph Computation Research Articles

Recommendation Unlearning via Influence Function

Recommendation unlearning is an emerging task to serve users for erasing unusable data ( e.g., some historical behaviors) from a well-trained recommender model. Existing methods process unlearning requests by fully or partially retraining the model after removing the unusable data. However, these methods are impractical due to the high computation cost of full retraining and the highly possible performance damage of partial training. In this light, a desired recommendation unlearning method should obtain a similar model as full retraining in a more efficient manner, i.e., achieving complete, efficient and harmless unlearning. In this work, we propose a new Influence Function-based Recommendation Unlearning (IFRU) framework, which efficiently updates the model without retraining by estimating the influence of the unusable data on the model via the influence function . In the light that recent recommender models use historical data for both the constructions of the optimization loss and the computational graph ( e.g., neighborhood aggregation), IFRU jointly estimates the direct influence of unusable data on optimization loss and the spillover influence on the computational graph to pursue complete unlearning. Furthermore, we propose an importance-based pruning algorithm to reduce the cost of the influence function. IFRU is harmless and applicable to mainstream differentiable models. Extensive experiments demonstrate that IFRU achieves more than 250 times acceleration compared to retraining-based methods with recommendation performance comparable to full retraining. Codes are available at https://github.com/baiyimeng/IFRU.

Automatically Differentiable Higher-Order Parabolic Equation for Real-Time Underwater Sound Speed Profile Sensing

This paper is dedicated to the acoustic inversion of the vertical sound speed profiles (SSPs) in the underwater marine environment. The method of automatic differentiation is applied for the first time in this context. Representing the finite-difference Padé approximation of the propagation operator as a computational graph allows for the analytical computation of the gradient with respect to the SSP directly within the numerical scheme. The availability of the gradient, along with the high computational efficiency of the numerical method used, enables rapid inversion of the SSP based on acoustic measurements from a hydrophone array. It is demonstrated that local optimization methods can be effectively used for real-time sound speed inversion. Comparative analysis with existing methods shows the significant superiority of the proposed method in terms of computation speed.

Exploring Mean Cordial Labeling Possible Combination Position of Vertices in Graphs: A Computational Approach

Objectives: We have developed the Python module of mean cordial labeling for the different types of graphs, like Path, Cycle, and Subdivision of Star graphs. We aim to identify the number of combination positions of vertices that satisfy the mean cordial labeling rules. We have found mathematical equations that describe the labeling behavior in these graphs. Methods: In this paper, a Python program was developed to find the mean cordial labeling of the Path, Cycle, and Subdivision of the Star graph. We have obtained the number of combination positions of vertex obeying the mean cordial labeling condition using this Python module. We have drawn the graph between the number of vertices and the number of combination positions of the vertex. Findings: We have found that the different graph types like Path, Cycle, and Subdivision of Star graph satisfy the mean cordial labeling condition. The result indicates that the number of combination positions of the Path and cycle graph fulfill the power equation Y=a*xb and the Subdivision of the Star graph meets the exponential growth equation Y=a*bx. Novelty: In this paper, a new method has been developed to find the number of combination position of varies graphs using the Python module. This computational approach will be useful to analyze the biological networking and protein-protein interaction study. Keywords: Mean Cordial, Path, Cycle, Subdivision of Star graph, Computational graph theory

STCO: Enhancing Training Efficiency via Structured Sparse Tensor Compilation Optimization

Network sparsification serves as an effective technique to accelerate Deep Neural Network (DNN) inference. However, existing sparsification techniques often rely on structured sparsity, which yields limited benefits. This is primarily due to the significant memory and computational overhead introduced by numerous sparse storage formats during address generation and gradient updates. Additionally, many of these solutions are tailored solely for the inference phase, neglecting the crucial training phase. In this paper, we introduce STCO, a novel Sparse Tensor Compilation Optimization technique that significantly enhances training efficiency through structured sparse tensor compilation. Central to STCO is the Tensorization-aware Index Entity (TIE) format, which effectively represents structured sparse tensors by eliminating redundant indices and minimizing storage overhead. The TIE format plays a pivotal role in the Address-carry flow (AC flow) pass, which optimizes the data layout at the computational graph level. This pass leverages the TIE format to enhance the efficiency of tensor representations, enabling more compact and efficient sparse tensor storage. Meanwhile, a shape inference pass utilizes the AC flow to derive optimized tensor shapes, further refining the performance of sparse tensor operations. Moreover, the Address-Carry TIE Flow dynamically tracks nonzero addresses, extending the benefits of sparse optimization to both forward and backward propagation. This seamless integration into the training pipeline enables a smooth transition to sparse tensor compilation without significant modifications to existing codebases. To further boost training performance, we implement an operator-level AC flow optimization pass tailored for structured sparse tensors. This pass generates efficient addresses, ensuring minimal computational overhead during sparse tensor operations. The flexibility of STCO allows it to be efficiently integrated into various frameworks or compilers, providing a robust solution for enhancing training efficiency with structured sparse tensors. Experiments demonstrated that STCO achieved impressive speedups of 3.64 ×, 5.43 ×, 4.89 ×, and 3.91 × when compared to state-of-the-art sparse formats on VGG16, ResNet-18, MobileNetV1, and MobileNetV2, respectively. These findings underscore the efficiency and superiority of our proposed approach in leveraging unstructured sparsity for Deep Neural Network inference acceleration.

PolyJuice: Detecting Mis-compilation Bugs in Tensor Compilers with Equality Saturation Based Rewriting

Tensor compilers are essential for deploying deep learning applications across various hardware platforms. While powerful, they are inherently complex and present significant challenges in ensuring correctness. This paper introduces PolyJuice, an automatic detection tool for identifying mis-compilation bugs in tensor compilers. Its basic idea is to construct semantically-equivalent computation graphs to validate the correctness of tensor compilers. The main challenge is to construct equivalent graphs capable of efficiently exploring the diverse optimization logic during compilation. We approach it from two dimensions. First, we propose arithmetic and structural equivalent rewrite rules to modify the dataflow of a tensor program. Second, we design an efficient equality saturation based rewriting framework to identify the most simplified and the most complex equivalent computation graphs for an input graph. After that, the outcome computation graphs have different dataflow and will likely experience different optimization processes during compilation. We applied it to five well-tested industrial tensor compilers, namely PyTorch Inductor, OnnxRuntime, TVM, TensorRT, and XLA, as well as two well-maintained academic tensor compilers, EinNet and Hidet. In total, PolyJuice detected 84 non-crash mis-compilation bugs, out of which 49 were confirmed with 20 fixed.

An end-to-end bi-objective approach to deep graph partitioning

Graphs are ubiquitous in real-world applications, such as computation graphs and social networks. Partitioning large graphs into smaller, balanced partitions is often essential, with the bi-objective graph partitioning problem aiming to minimize both the “cut” across partitions and the imbalance in partition sizes. However, existing heuristic methods face scalability challenges or overlook partition balance, leading to suboptimal results. Recent deep learning approaches, while promising, typically focus only on node-level features and lack a truly end-to-end framework, resulting in limited performance. In this paper, we introduce a novel method based on graph neural networks (GNNs) that leverages multilevel graph features and addresses the problem end-to-end through a bi-objective formulation. Our approach explores node-, local-, and global-level features, and introduces a well-bounded bi-objective function that minimizes the cut while ensuring partition-wise balance across all partitions. Additionally, we propose a GNN-based deep model incorporating a Hardmax operator, allowing the model to optimize partitions in a fully end-to-end manner. Experimental results on 12 datasets across various applications and scales demonstrate that our method significantly improves both partitioning quality and scalability compared to existing bi-objective and deep graph partitioning baselines.

Strategic allocation of landmarks to reduce uncertainty in indoor navigation

Indoor navigation systems often rely on verbal, turn-based route instructions. These can, at times, be ambiguous at complex decision points with multiple paths intersecting under angles that are not well distinguished by the turn grammar used. Landmarks can be included into turn instructions to reduce this ambiguity. Here, we propose an approach to optimize landmark allocation to improve the clarity of route instructions. This study assumes that landmark locations are constrained to a pre-determined set of slots. We select a minimum-size subset of the set of all slots and allocate it with landmarks, such that the navigation ambiguity is resolved. Our methodology leverages computational geometric analysis, graph algorithms, and optimization formulations to strategically incorporate landmarks into indoor route instructions. We propose a method to optimize landmark allocation in indoor navigation guidance systems, improving the clarity of route instructions at complex decision points that are inadequately served by turn-based instructions alone.

Few-shot image classification using graph neural network with fine-grained feature descriptors

Graph computation via Graph Neural Networks (GNNs) is emerging as a pivotal approach for addressing the challenges in image classification tasks. This paper introduces a novel strategy for image classification using minimal labeled data from the mini-ImageNet database. The primary contributions include the development of an innovative Fine-Grained Feature Descriptor (FGFD) module. Following this, the GNN is employed at a more granular level to enhance image classification efficiency. Additionally, ablation studies were conducted in conjunction with existing state-of-the-art systems for few-shot image classification. Comparative analyses were performed, and the simulation results demonstrate that the proposed method significantly improves classification accuracy over traditional few-shot image classification methods.

AutoGMap: Learning to Map Large-Scale Sparse Graphs on Memristive Crossbars.

The sparse representation of graphs has shown great potential for accelerating the computation of graph applications (e.g., social networks and knowledge graphs) on traditional computing architectures (CPU, GPU, or TPU). But, the exploration of large-scale sparse graph computing on processing-in-memory (PIM) platforms (typically with memristive crossbars) is still in its infancy. To implement the computation or storage of large-scale or batch graphs on memristive crossbars, a natural assumption is that a large-scale crossbar is demanded, but with low utilization. Some recent works question this assumption; to avoid the waste of storage and computational resource, the fixed-size or progressively scheduled "block partition" schemes are proposed. However, these methods are coarse-grained or static and are not effectively sparsity-aware. This work proposes the dynamic sparsity-aware mapping scheme generating method that models the problem with a sequential decision-making model, and optimizes it by reinforcement learning (RL) algorithm (REINFORCE). Our generating model [long short-term memory (LSTM), combined with the dynamic-fill scheme] generates remarkable mapping performance on the small-scale graph/matrix data (complete mapping costs 43% area of the original matrix) and two large-scale matrix data (costing 22.5% area on qh882 and 17.1% area on qh1484). Our method may be extended to sparse graph computing on other PIM architectures, not limited to the memristive device-based platforms.

Grapher: A Reconfigurable Graph Computing Accelerator with Optimized Processing Elements

In recent years, various graph computing architectures have been proposed to process graph data that represent complex dependencies between different objects in the world. The designs of the processing element (PE) in traditional graph computing accelerators are often optimized for specific graph algorithms or tasks, which limits their flexibility in processing different types of graph algorithms, or the parallel configuration that can be supported by their PE arrays is inefficient. To achieve both flexibility and efficiency, this paper proposes Grapher, a reconfigurable graph computing accelerator based on an optimized PE array, efficiently supporting multiple graph algorithms, enhancing parallel computation, and improving adaptability and system performance through dynamic hardware resource configuration. To verify the performance of Grapher, this paper selected six datasets from the Stanford Network Analysis Project (SNAP) database for testing. Compared with the existing typical graph frameworks Ligra, Gemini, and GraphBIG, the processing time for the six datasets using the BFS, CC, and PR algorithms was reduced by up to 39.31%, 35.43%, and 27.67%, respectively. The energy efficiency has also been improved by 1.8× compared to Hitgraph and 4.7× compared to ThunderGP.

The ultrametric backbone is the union of all minimum spanning forests

Minimum spanning trees and forests are powerful sparsification techniques that remove cycles from weighted graphs to minimize total edge weight while preserving node reachability, with applications in computer science, network science, and graph theory. Despite their utility and ubiquity, they have several limitations, including that they are only defined for undirected networks, they significantly alter dynamics on networks, and they do not generally preserve important network features such as shortest distances, shortest path distribution, and community structure. In contrast, distance backbones, which are subgraphs formed by all edges that obey a generalized triangle inequality, are well defined in directed and undirected graphs and preserve those and other important network features. The backbone of a graph is defined with respect to a specified path-length operator that aggregates weights along a path to define its length, thereby associating a cost to indirect connections. The backbone is the union of all shortest paths between each pair of nodes according to the specified operator. One such operator, the max function, computes the length of a path as the largest weight of the edges that compose it (a weakest link criterion). It is the only operator that yields an algebraic structure for computing shortest paths that is consistent with De Morgan's laws. Applying this operator yields the ultrametric backbone of a graph in that (semi-triangular) edges whose weights are larger than the length of an indirect path connecting the same nodes (i.e. those that break the generalized triangle inequality based on max as a path-length operator) are removed. We show that the ultrametric backbone is the union of minimum spanning forests in undirected graphs and provides a new generalization of minimum spanning trees to directed graphs that, unlike minimum equivalent graphs and minimum spanning arborescences, preserves all shortest paths and De Morgan's law consistency.

The Symplectic Adjoint Method: Memory-Efficient Backpropagation of Neural-Network-Based Differential Equations.

The combination of neural networks and numerical integration can provide highly accurate models of continuous-time dynamical systems and probabilistic distributions. However, if a neural network is used n times during numerical integration, the whole computation graph can be considered as a network n times deeper than the original. The backpropagation algorithm consumes memory in proportion to the number of uses times of the network size, causing practical difficulties. This is true even if a checkpointing scheme divides the computation graph into subgraphs. Alternatively, the adjoint method obtains a gradient by a numerical integration backward in time; although this method consumes memory only for single-network use, the computational cost of suppressing numerical errors is high. The symplectic adjoint method proposed in this study, an adjoint method solved by a symplectic integrator, obtains the exact gradient (up to rounding error) with memory proportional to the number of uses plus the network size. The theoretical analysis shows that it consumes much less memory than the naive backpropagation algorithm and checkpointing schemes. The experiments verify the theory, and they also demonstrate that the symplectic adjoint method is faster than the adjoint method and is more robust to rounding errors.

X-SLAM: Scalable Dense SLAM for Task-aware Optimization using CSFD

We present X-SLAM, a real-time dense differentiable SLAM system that leverages the complex-step finite difference (CSFD) method for efficient calculation of numerical derivatives, bypassing the need for a large-scale computational graph. The key to our approach is treating the SLAM process as a differentiable function, enabling the calculation of the derivatives of important SLAM parameters through Taylor series expansion within the complex domain. Our system allows for the real-time calculation of not just the gradient, but also higher-order differentiation. This facilitates the use of high-order optimizers to achieve better accuracy and faster convergence. Building on X-SLAM, we implemented end-to-end optimization frameworks for two important tasks: camera relocalization in wide outdoor scenes and active robotic scanning in complex indoor environments. Comprehensive evaluations on public benchmarks and intricate real scenes underscore the improvements in the accuracy of camera relocalization and the efficiency of robotic navigation achieved through our task-aware optimization. The code and data are available at https://gapszju.github.io/X-SLAM.

Modeling Multimodal Curbside Usage in Dynamic Networks

The proliferation of emerging mobility technology has led to a significant increase in demand for ride-hailing services, on-demand deliveries, and micromobility services, transforming curb spaces into valuable public infrastructure for which multimodal transportation competes. However, the increasing utilization of curbs by different traffic modes has substantial societal impacts, further altering travelers’ choices and polluting the urban environment. Integrating the spatiotemporal characteristics of various behaviors related to curb utilization into general dynamic networks and exploring mobility patterns with multisource data remain a challenge. To address this issue, this study proposes a comprehensive framework of modeling curbside usage by multimodal transportation in a general dynamic network. The framework encapsulates route choices, curb space competition, and interactive effects among different curb users, and it embeds the dynamics of curb usage into a mesoscopic dynamic network model. Furthermore, a curb-aware dynamic origin-destination demand estimation framework is proposed to reveal the network-level spatiotemporal mobility patterns associated with curb usage through a physics-informed data-driven approach. The framework integrates emerging real-world curb use data in conjunction with other mobility data represented on computational graphs, which can be solved efficiently using the forward-backward algorithm on large-scale networks. The framework is examined on a small network as well as a large-scale real-world network. The estimation results on both networks are satisfactory and compelling, demonstrating the capability of the framework to estimate the spatiotemporal curb usage by multimodal transportation. History: This paper has been accepted for the Transportation Science Special Issue on ISTTT25. Funding: This material is based upon work supported by the Office of Energy Efficiency and Renewable Energy, U.S. Department of Energy [Award DE-EE0009659]. Supplemental Material: The online appendix is available at https://doi.org/10.1287/trsc.2024.0522 .

Characterizing behavioural differentiation in gene regulatory networks with representation graphs.

We introduce the formal notion of representation graphs, encapsulating the state space structure of gene regulatory network models in a compact and concise form that highlights the most significant features of stable states and differentiation processes leading to distinct stability regions. The concept has been developed in the context of a hybrid system-based gene network modelling framework; however, we anticipate that it can also be adapted to other approaches of modelling gene networks in discrete terms. We describe a practical algorithm for representation graph computation as well as two case studies demonstrating their real-world application and utility. The first case study presents models for three phage viruses. It shows that the process of differentiation into lytic and lysogenic behavioural states for all these models is described by the same representation graph despite the distinctive underlying mechanisms for differentiation. The second case study shows the advantages of our approach for modelling the process of myeloid cell differentiation from a common progenitor into different cell types. Both case studies also demonstrate the potential of the representation graph approach for deriving and validating hypotheses about regulatory interactions that must be satisfied for biologically viable behaviours.

Efficient Algorithms for Pseudoarboricity Computation in Large Static and Dynamic Graphs

The arboricity a ( G ) of a graph G is defined as the minimum number of edge-disjoint forests that the edge set of G can be partitioned into. It is a fundamental metric and has been widely used in many graph analysis applications. However, computing a ( G ) is typically a challenging task. To address this, an easier-to-compute alternative called pseudoarboricity was proposed. Pseudoarboricity has been shown to be closely connected to many important measures in graphs, including the arboricity and the densest subgraph density ρ ( G ). Computing the exact pseudoarboricity can be achieved by employing a parametric max-flow algorithm, but it becomes computationally expensive for large graphs. Existing 2-approximation algorithms, while more efficient, often lack satisfactory approximation accuracy. To overcome these limitations, we propose two new approximation algorithms with theoretical guarantees to approximate the pseudoarboricity. We show that our approximation algorithms can significantly reduce the number of times the max-flow algorithm is invoked, greatly improving its efficiency for exact pseudoarboricity computation. In addition, we also study the pseudoarboricity maintenance problem in dynamic graphs. We propose two novel and efficient algorithms for maintaining the pseudoarboricity when the graph is updated by edge insertions or deletions. Furthermore, we develop two incremental pseudoarboricity maintenance algorithms specifically designed for insertion-only scenarios. We conduct extensive experiments on 195 real-world graphs, and the results demonstrate the high efficiency and scalability of the proposed algorithms in computing pseudoarboricity for both static and dynamic graphs.

A Systematic Review of Attention Models in Natural Language Processing

The role of attention models in neural networks has drawn much focus among scholars in recent years mainly because of its efficiency and versatility. Attention models are important for many Natural Language Processing applications like question answering, semantic analysis, sentiment analysis and machine translation performing significantly better than other usual approaches. These mechanisms improve neural networks capability in interpretation and tackle issues like performance decay, which affects the Recurrent Neural Networks by concentrating on significant data inputs and context. A stream of research has explored the integration of attention mechanisms to other fields, namely computer vision and graph analysis improving tasks like object detection, image captioning or node representation learning. However, for more detailed research of modern achievements and various domains and model architecture structures there is still the necessity to use the more profound survey-based investigation. This research article provides a review of previous work done on attention-based model in NLP. It also outlines the problems and provides relevant solutions using different methodologies. Future directions of the research are provided for the improvement of the attention models for NLP, discussing potential issues in terms of model performance and its capacity, as well as its interpretability. This review is intended to help further research to more universal attention-based solutions in the domain of NLP.

Memristor-based Bayesian spiking neural network for IBD diagnosis

Traditional deep learning methods, such as deep neural networks, have achieved remarkable success in image processing, natural language processing, graph computation, and other fields. However, in the diagnosis of Inflammatory Bowel Disease (IBD), conventional deep learning methods are highly susceptible to generating overconfident results, which can lead to great loss of life and property if it results in misdiagnosis of the patient. In this work, we propose a memristor-based IBD diagnostic system inspired by the idea of variational inference based on Bayes' law. The memristors in our system are used for both synaptic weight storage and entropy sources so that an efficient Bayesian inference system can be realized. Our memristor-based true random number generator can produce probability-tunable bitstreams required for Bayesian inference with a mean-square error that is reduced by 10.9 × than that of a conventional pseudo-random generator. The proposed system consists of a coarse-grained and a fined-grained spiking neural network, and an optimized strategy that trims the neuron's threshold is proposed for maintaining the appropriate model sparsity and reducing information loss. With Bayesian inference, our work achieves lower diagnostic error rates, which are reduced by 10.79 % and 0.49 % for the coarse-grained and fine-grained networks, respectively. The proposed system is easy to hardware implement and presents a novel solution for community-wide IBD diagnosis.

Conditional computation in neural networks: Principles and research trends

This article summarizes principles and ideas from the emerging area of applying conditional computation methods to the design of neural networks. In particular, we focus on neural networks that can dynamically activate or de-activate parts of their computational graph conditionally on their input. Examples include the dynamic selection of, e.g., input tokens, layers (or sets of layers), and sub-modules inside each layer (e.g., channels in a convolutional filter). We first provide a general formalism to describe these techniques in an uniform way. Then, we introduce three notable implementations of these principles: mixture-of-experts (MoEs) networks, token selection mechanisms, and early-exit neural networks. The paper aims to provide a tutorial-like introduction to this growing field. To this end, we analyze the benefits of these modular designs in terms of efficiency, explainability, and transfer learning, with a focus on emerging applicative areas ranging from automated scientific discovery to semantic communication.

AG-SpTRSV: An Automatic Framework to Optimize Sparse Triangular Solve on GPUs

Sparse Triangular Solve (SpTRSV) has long been an essential kernel in the field of scientific computing. Due to its low computational intensity and internal data dependencies, SpTRSV is hard to implement and optimize on GPUs. Based on our experimental observations, existing implementations on GPUs fail to achieve the optimal performance due to their sub-optimal parallelism setups and code implementations, and lack of consideration of the irregular data distribution. Moreover, their algorithm design lacks the adaptability to different input matrices, which may involve substantial manual efforts of algorithm redesigning and parameter tuning for performance consistency. In this work, we propose AG-SpTRSV, an automatic framework to optimize SpTRSV on GPUs, which provides high performance on various matrices while eliminating the costs of manual design. AG-SpTRSV abstracts the procedures of optimizing an SpTRSV kernel as a scheme and constructs a comprehensive optimization space based on it. By defining a unified code template and preparing code variants, AG-SpTRSV enables fine-grained dynamic parallelism and adaptive code optimizations to handle various tasks. Through computation graph transformation and multi-hierarchy heuristic scheduling, AG-SpTRSV generates schemes for task partitioning and mapping, which effectively address the issues of irregular data distribution and internal data dependencies. AG-SpTRSV searches for the best scheme to optimize the target kernel for the specific matrix. A learned lightweight performance model is also introduced to reduce search costs and provide an efficient end-to-end solution. Experimental results with SuiteSparse Matrix Collection on NVIDIA Tesla A100 and RTX 3080 Ti show that AG-SpTRSV outperforms state-of-the-art implementations with geometric average speedups of 2.12x ∼ 3.99x. With the performance model enabled, AG-SpTRSV can provide an efficient end-to-end solution, with preprocessing times ranging from 3.4 to 245 times of the execution time.