Model-Architecture Co-Design for High Performance Temporal GNN Inference on FPGA

Temporal graphs exhibit dynamic interactions between nodes over continuous\ntime, whose topologies evolve with time elapsing.\n The whole temporal neighborhood of nodes reveals the varying preferences of\nnodes.\n However, previous works usually generate dynamic representation with limited\nneighbors for simplicity, which results in both inferior performance and high\nlatency of online inference.\n Therefore, in this paper, we propose a novel method of temporal graph\nconvolution with the whole neighborhood, namely Temporal Aggregation and\nPropagation Graph Neural Networks (TAP-GNN).\n Specifically, we firstly analyze the computational complexity of the dynamic\nrepresentation problem by unfolding the temporal graph in a message-passing\nparadigm.\n The expensive complexity motivates us to design the AP (aggregation and\npropagation) block, which significantly reduces the repeated computation of\nhistorical neighbors.\n The final TAP-GNN supports online inference in the graph stream scenario,\nwhich incorporates the temporal information into node embeddings with a\ntemporal activation function and a projection layer besides several AP blocks.\n Experimental results on various real-life temporal networks show that our\nproposed TAP-GNN outperforms existing temporal graph methods by a large margin\nin terms of both predictive performance and online inference latency.\n Our code is available at \\url{https://github.com/doujiang-zheng/TAP-GNN}.\n

Representation learning over temporal networks has drawn considerable attention in recent years. Efforts are mainly focused on modeling structural dependencies and temporal evolving regularities in Euclidean space which, however, underestimates the inherent complex and hierarchical properties in many real-world temporal networks, leading to sub-optimal embeddings. To explore these properties of a complex temporal network, we propose a hyperbolic temporal graph network (HTGN) that fully takes advantage of the exponential capacity and hierarchical awareness of hyperbolic geometry. More specially, HTGN maps the temporal graph into hyperbolic space, and incorporates hyperbolic graph neural network and hyperbolic gated recurrent neural network, to capture the evolving behaviors and implicitly preserve hierarchical information simultaneously. Furthermore, in the hyperbolic space, we propose two important modules that enable HTGN to successfully model temporal networks: (1) hyperbolic temporal contextual self-attention (HTA) module to attend to historical states and (2) hyperbolic temporal consistency (HTC) module to ensure stability and generalization. Experimental results on multiple real-world datasets demonstrate the superiority of HTGN for temporal graph embedding, as it consistently outperforms competing methods by significant margins in various temporal link prediction tasks. Specifically, HTGN achieves AUC improvement up to 9.98% for link prediction and 11.4% for new link prediction. Moreover, the ablation study further validates the representational ability of hyperbolic geometry and the effectiveness of the proposed HTA and HTC modules.

Searching for local communities is an important research challenge that allows for personalized community discovery and supports advanced data analysis in various complex networks, such as the World Wide Web, social networks, and brain networks. The evolution of these networks over time has motivated several recent studies to identify local communities in temporal networks. Given any query nodes, Community Search aims to find a densely connected subgraph containing query nodes. However, existing community search approaches in temporal networks have two main limitations: (1) they adopt pre-defined subgraph patterns to model communities, which cannot find communities that do not conform to these patterns in real-world networks, and (2) they only use the aggregation of disjoint structural information to measure quality, missing the dynamic of connections and temporal properties. In this paper, we propose a query-driven Temporal Graph Convolutional Network (CS-TGN) that can capture flexible community structures by learning from the ground-truth communities in a data-driven manner. CS-TGN first combines the local query-dependent structure and the global graph embedding in each snapshot of the network and then uses a GRU cell with contextual attention to learn the dynamics of interactions and update node embeddings over time. We demonstrate how this model can be used for interactive community search in an online setting, allowing users to evaluate the found communities and provide feedback. Experiments on real-world temporal graphs with ground-truth communities validate the superior quality of the solutions obtained and the efficiency of our model in both temporal and interactive static settings.

Influence Maximization (IM) in temporal graphs focuses on identifying influential ``seeds'' that are pivotal for maximizing network expansion. We advocate defining these seeds through Influence Propagation Paths (IPPs), which is essential for scaling up the network. Our focus lies in efficiently labeling IPPs and accurately predicting these seeds, while addressing the often-overlooked cold-start issue prevalent in temporal networks. Our strategy introduces a motif-based labeling method and a tensorized Temporal Graph Network (TGN) tailored for multi-relational temporal graphs, bolstering prediction accuracy and computational efficiency. Moreover, we augment cold-start nodes with new neighbors from historical data sharing similar IPPs. The recommendation system within an online team-based gaming environment presents subtle impact on the social network, forming multi-relational (i.e., weak and strong) temporal graphs for our empirical IM study. We conduct offline experiments to assess prediction accuracy and model training efficiency, complemented by online A/B testing to validate practical network growth and the effectiveness in addressing the cold-start issue.

Microservices provide modularity and elastic scaling for cloud-native systems, yet highly dynamic workloads make autoscaling both critical and difficult. Bursty traffic surges risk SLA violations, while over-provisioning during low-load periods wastes up to $6.6B annually. An effective autoscaler must not only anticipate workload shifts but also coordinate scaling across interdependent services. Existing ML-, RL-, and GNN-based methods improve efficiency but fail to address the Degradation of Resource Oscillation (DRO)—unstable, repeated up/down scaling triggered by abrupt workload changes or prediction errors—which destabilizes performance and inflates costs. We present GraphPilot, the first Temporal Graph Actor–Critic (TGAC) autoscaler for kubernetes that directly links temporal dependency graph modeling to mitigating oscillations. By representing microservice topologies as evolving graphs and capturing their temporal dependencies through a Temporal Graph Network (TGN), GraphPilot can anticipate Long-Term Dependency Propagation (LTDP)—the underlying cause of DRO—where delayed cross-service dependencies amplify workload shifts and destabilize systems. These temporal graph embeddings are fed into TGAC, enabling coordinated and proactive scaling decisions that stabilize resource allocation under volatile workloads. Extensive experiments on realistic microservice benchmarks show that GraphPilot outperforms state-of-the-art autoscalers in 90.63% of metrics, delivering up to 95.02% higher request-per-pod time (RPT), 136.83% greater throughput, and 73.44% lower latency, while maintaining stable and cost-efficient scaling. The source code is available at https://github.com/ntu-ben/GRLScaler.git.

Periodicity is a frequently happening phenomenon for social interactions in temporal networks. Mining periodic communities are essential to understanding periodic group behaviors in temporal networks. Unfortunately, most previous studies for community mining in temporal networks ignore the periodic patterns of communities. In this paper, we study the problem of seeking periodic communities in a temporal network, where each edge is associated with a set of timestamps. We propose novel models, including <inline-formula><tex-math notation="LaTeX">$\sigma$</tex-math></inline-formula> -periodic <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -core and <inline-formula><tex-math notation="LaTeX">$\sigma$</tex-math></inline-formula> -periodic <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -clique, that represent periodic communities in temporal networks. Specifically, a <inline-formula><tex-math notation="LaTeX">$\sigma$</tex-math></inline-formula> -periodic <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -core (or <inline-formula><tex-math notation="LaTeX">$\sigma$</tex-math></inline-formula> -periodic <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -clique) is a <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -core (or clique with size larger than <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> ) that appears at least <inline-formula><tex-math notation="LaTeX">$\sigma$</tex-math></inline-formula> times periodically in the temporal graph. The problem of searching periodic core is efficient but the resulting communities may be not enough cohesive; the problem of enumerating all periodic cliques is not efficient (NP-hard) but the resulting communities are very cohesive. To compute all of them efficiently, we first develop two effective graph reduction techniques to significantly prune the temporal graph. Then, we transform the temporal graph into a static graph and prove that mining the periodic communities in the temporal graph equals mining communities in the transformed graph. Subsequently, we propose a decomposition algorithm to search maximal <inline-formula><tex-math notation="LaTeX">$\sigma$</tex-math></inline-formula> -periodic <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -core, a Bron-Kerbosch style algorithm to enumerate all maximal <inline-formula><tex-math notation="LaTeX">$\sigma$</tex-math></inline-formula> -periodic <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -cliques, and a branch-and-bound style algorithm to find the maximum <inline-formula><tex-math notation="LaTeX">$\sigma$</tex-math></inline-formula> -periodic clique. The results of extensive experiments on five real-life datasets demonstrate the efficiency, scalability, and effectiveness of our algorithms.

Sequential recommendation is a crucial task in understanding users' evolving interests and predicting their future behaviors. While existing approaches on sequence or graph modeling to learn interaction sequences of users have shown promising performance, how to effectively exploit temporal information and deal with the uncertainty noise in evolving user behaviors is still quite challenging. To this end, in this paper, we propose a Temporal Graph Contrastive Learning method for Sequential Recommendation (TGCL4SR) which leverages not only local interaction sequences but also global temporal graphs to comprehend item correlations and analyze user behaviors from a temporal perspective. Specifically, we first devise a Temporal Item Transition Graph (TITG) to fully leverage global interactions to understand item correlations, and augment this graph by dual transformations based on neighbor sampling and time disturbance. Accordingly, we design a Temporal item Transition graph Convolutional network (TiTConv) to capture temporal item transition patterns in TITG. Then, a novel Temporal Graph Contrastive Learning (TGCL) mechanism is designed to enhance the uniformity of representations between augmented graphs from identical sequences. For local interaction sequences, we design a temporal sequence encoder to incorporate time interval embeddings into the architecture of Transformer. At the training stage, we take maximum mean discrepancy and TGCL losses as auxiliary objectives. Extensive experiments on several real-world datasets show the effectiveness of TGCL4SR against state-of-the-art baselines of sequential recommendation.

Temporal network analysis and time evolution of network characteristics are powerful tools in describing the changing topology of dynamic networks. This paper uses such approaches to better visualize and provide analytical measures for the changes in performance that we observed in Voronoi-type spatial coverage, particularly for the example of time-evolving networks with a changing number of wireless sensors being deployed. Specifically, our analysis focuses on the role different combinations of impenetrable obstacles and environmental noise play in connectivity and overall network structure. It is shown how the use of (i) temporal network graphs, and (ii) network centrality and regularity measures illustrate the differences between various options developed for the balancing act of energy and time efficiency in network coverage. Last, we compare the outcome of these measures with the less abstract classification variables, such as percent area covered and cumulative distance traveled.

Many real-world complex systems can be modelled by temporal networks. Representation learning on these networks often captures their dynamic evolution and is a first step for performing further analysis, e.g. node classification. Node classification is a fundamental task for graph analysis in general and in the context of temporal graph, is often employed to categories nodes based on their activity patterns. Analysis of existing real world networks from different high-stake domains reveals that the rate of the malicious activities is on uptick, resulting in catastrophic social or economic consequences. This strongly motivates designing accurate node classification methods for temporal graphs. In this paper, we propose TGbase, for node classification on weighted temporal networks. TGbase efficiently extracts key features to consider the structural characteristics of each node and its neighborhood as well as the intensity and timestamp of the interactions among node pairs. These features accurately differentiate different classes of nodes, as shown on eight real-world benchmark datasets, outperforming multiple state-of-the-art (SOTA) deep/complex models. Our strong yet simple model is also generic, whereas the SOTA contenders are designed often for their specific (class of) datasets.

Spreading processes on graphs are a natural model for a wide variety of real-world phenomena, including information or behaviour spread over social networks, biological diseases spreading over contact or trade networks, and the potential flow of goods over logistical infrastructure. Often, the networks over which these processes spread are dynamic in nature, and can be modeled with graphs whose structure is subject to discrete changes over time, i.e. with temporal graphs. Here, we consider temporal graphs in which edges are available at specified timesteps, and study the problem of deleting edges from a given temporal graph in order to reduce the number of vertices (temporally) reachable from a given starting point. This could be used to control the spread of a disease, rumour, etc. in a temporal graph. In particular, our aim is to find a temporal subgraph in which a process starting at any single vertex can be transferred to only a limited number of other vertices using a temporally-feasible path (i.e. a path, along which the times of the edge availabilities increase). We introduce a natural deletion problem for temporal graphs and we provide positive and negative results on its computational complexity, both in the traditional and the parameterised sense (subject to various natural parameters), as well as addressing the approximability of this problem.

Predicting remaining useful life (RUL) plays a crucial role in the prognostics and health management of industrial systems that involve a variety of interrelated sensors. Given a constant stream of time-series sensory data from such systems, deep learning (DL) models have risen to prominence at identifying complex, nonlinear temporal dependencies in these data. In addition to the temporal dependencies of individual sensors, spatial dependencies emerge as important correlations among these sensors, which can be naturally modeled by a temporal graph that describes time-varying spatial relationships. However, the majority of existing studies have relied on capturing discrete snapshots of this temporal graph, a coarse-grained approach that leads to a loss of temporal information. Moreover, given the variety of heterogeneous sensors, it becomes vital that such inherent heterogeneity is leveraged for RUL prediction in temporal sensor graphs. To capture the nuances of the temporal and spatial relationships and heterogeneous characteristics in an interconnected graph of sensors, we introduce a novel model named temporal and heterogeneous graph neural networks (THGNNs). Specifically, THGNN aggregates historical data from neighboring nodes to accurately capture the temporal dynamics and spatial correlations within the stream of sensor data in a fine-grained manner. Moreover, the model leverages feature-wise linear modulation (FiLM) to address the diversity of sensor types, significantly improving the model's capacity to learn the heterogeneity in the data sources. Finally, we have validated the effectiveness of our approach through comprehensive experiments. Our empirical findings demonstrate significant advancements on the N-CMAPSS dataset, achieving improvements of up to 19.2% and 31.6% in terms of two different evaluation metrics over state-of-the-art methods.

This research aims to develop a novel framework that uncovers the causal influence of global events on public sentiment through temporal graph modeling and neural causal inference. Global events, such as pandemics, elections, and economic crises, profoundly affect public sentiments, shaping social behaviors and economic outcomes. Traditional models often fall short in capturing the complex, dynamic, and non-linear relationships between these events and sentiments. This paper presents the Neural Temporal Causal Graph Network (NTCGN), a unified framework that integrates temporal graph neural networks with a Causal Attention Network (CAN) to model and interpret these relationships. NTCGN constructs a temporal graph from event data and sentiment-labeled texts, learning dependencies and causal influences through advanced neural architectures. A thorough comparative analysis with state-of-the-art models such as Logistic Regression, SVM, LSTM, and transformer-based models demonstrates NTCGN’s superior performance. Experimental evaluation using the Sentiment140 and Global Database of Events, Language and Tone (GDELT) 2.0 datasets shows NTCGN achieving an accuracy of 0.798 and an F1 score of 0.795, outperforming these baseline models. The model’s causal inference capabilities are validated using the Causal Impact Score (CIS) and Causal Discovery Precision (CDP), highlighting its reliability in identifying true causal links. Visualizations of attention maps and causal pathways enhance interpretability, demonstrating how specific events influence public sentiments. This work provides a robust and interpretable tool for analyzing event-driven sentiment dynamics in real-world applications.

Temporal Graph Neural Networks (TGNN) have the ability to capture both the graph topology and dynamic dependencies of interactions within a graph over time. There has been a growing need to explain the predictions of TGNN models due to the difficulty in identifying how past events influence their predictions. Since the explanation model for a static graph cannot be readily applied to temporal graphs due to its inability to capture temporal dependencies, recent studies proposed explanation models for temporal graphs. However, existing explanation models for temporal graphs rely on post-hoc explanations, requiring separate models for prediction and explanation, which is limited in two aspects: efficiency and accuracy of explanation. In this work, we propose a novel built-in explanation framework for temporal graphs, called Self-Explainable Temporal Graph Networks based on Graph Information Bottleneck (TGIB). TGIB provides explanations for event occurrences by introducing stochasticity in each temporal event based on the Information Bottleneck theory. Experimental results demonstrate the superiority of TGIB in terms of both the link prediction performance and explainability compared to state-of-the-art methods. This is the first work that simultaneously performs prediction and explanation for temporal graphs in an end-to-end manner. The source code of TGIB is available at https://github.com/sang-woo-seo/TGIB.

As individuals are always changing, online social platforms can quickly create new communities and information cascades. Many apps depend on finding these occurrences early. Some of these are predicting trends, analyzing public opinion, targeted marketing, and stopping the spread of false information. Still, current approaches often assume graphs are static or employ coarse temporal aggregation, making it harder for them to capture community-aware interactions and fine-grained temporal diffusion dynamics. Because of these problems, the authors propose a mechanism for identifying new communities and viral cascades on social media platforms, called the Community Aware Temporal Pattern Diffusion Network (CTPDN). The proposed CTPDN uses both community-aware representation learning and temporal graph modeling. A community-aware attention mechanism tracks how relationships within and between communities change over time, while temporal diffusion layers operate like the spread of time-sensitive information. Using a hierarchical temporal encoder, we can observe both short-term diffusion patterns and long-term structural change. The model is trained from scratch to predict community formation and the onset of cascade virality in the early stages of diffusion. Compared with the best temporal graph and cascade prediction models, CTPDN performs far better in terms of lead-time gain, F1-score, and early-detection accuracy, as demonstrated by experimental tests on benchmark social network datasets. The results show that including explicit community awareness and temporal dispersion patterns strengthens and simplifies predictions of future social dynamics. CTPDN does substantially better than both old-school diffusion models and newer deep learning baselines, according to results from large-scale research on real-world social network datasets. The framework is effective for recognizing early viral trends, as shown by the results, which indicate an average 20–27% increase in the accuracy of early cascade detection, a 17–24% increase in the F1-score for identifying new communities, and a decrease in detection delay of up to 22%. Lastly, CTPDN increases overall performance on crucial assessment criteria by around 25% and gives a way to find early communities and cascades that can grow. CTPDN provides a scalable, efficient framework for analyzing and predicting viral cascades and the dynamic formation of communities. The proposed method not only facilitates future research in community-centric temporal graph learning but also has practical implications in real-time social analytics.

Learning on temporal heterogeneous graphs (THGs) has attracted substantial attention in applications of information retrieval. Such graphs are ubiquitous in real-world domains like recommender systems and social networks. However, the spatial heterogeneity, rich semantic information, and intricate evolution patterns of THGs make it still difficult to generate high-quality embeddings for graph nodes. In this paper, we focus on two valuable and understudied issues related to THG learning: (a) How to capture the specific evolutionary characteristics of diverse temporal heterogeneous graphs? (b) Due to the heterogeneous nature of the graph, how to capture the unique temporal patterns of different node types? We explore these questions and present our solution by proposing a new method named CasMLN (Cascaded Multi-level Learning Network) for THG learning. Through the multi-level learning structure and aggregation methods specifically designed for different levels, we obtain information of multiple levels and fuse them to improve embedding generation. Additionally, we pioneer the use of large language models (LLMs) in the THG field. By leveraging the universality and powerful capabilities of LLMs, our method introduces LLM-based external knowledge to effectively capture the implicit nature of graphs and node types, which helps to enhance type- and graph-level representations. We evaluate our method on several real-world THG datasets for different downstream tasks. Extensive experimental results show that CasMLN outperforms the state-of-the-art baselines in both accuracy and efficiency.