Temporal Point Process Research Articles

Retrieving Continuous Time Event Sequences using Neural Temporal Point Processes with Learnable Hashing

Temporal sequences have become pervasive in various real-world applications such as finance, spatial mobility, health records, etc. Consequently, the volume of data generated in the form of continuous time-event sequence(s) or CTES(s) has increased exponentially in the past few years. Thus, a significant fraction of the ongoing research on CTES datasets involves designing models to address downstream tasks such as next-event prediction, long-term forecasting, sequence classification etc. The recent developments in predictive modeling using marked temporal point processes (MTPP) have enabled an accurate characterization of several real-world applications involving the CTESs. However, due to the complex nature of these CTES datasets, the task of large-scale retrieval of temporal sequences has been overlooked by the past literature. In detail, by CTES retrieval we mean that for an input query sequence, a retrieval system must return a ranked list of relevant sequences from a large corpus. To tackle this, we propose NeuroSeqRet , a first-of-its-kind framework designed specifically for end-to-end CTES retrieval. Specifically, NeuroSeqRet introduces multiple enhancements over standard retrieval frameworks and first applies a trainable unwarping function on the query sequence which makes it comparable with corpus sequences, especially when a relevant query-corpus pair has individually different attributes. Next, it feeds the unwarped query sequence and the corpus sequence into MTPP-guided neural relevance models. We develop four variants of the relevance model for different kinds of applications based on the trade-off between accuracy and efficiency. We also propose an optimization framework to learn binary sequence embeddings from the relevance scores, suitable for the locality-sensitive hashing leading to a significant speedup in returning top-K results for a given query sequence. Our experiments with several datasets show the significant accuracy boost of NeuroSeqRet beyond several baselines, as well as the efficacy of our hashing mechanism.

Dynamic Graph Representation Learning via Coupling-Process Model.

Representation learning based on dynamic graphs has received a lot of attention in recent years due to its wide range of application scenarios. Although many discrete or continuous dynamic graph representation learning methods have been proposed, many of them ignore the role of edge types. Through the observation of dynamic graphs in the real world, it is found that the types of various edges are very different in nature. They are roughly divided into two categories according to the frequency of occurrence: evolutive edges that appear infrequently and interactive edges that appear frequently. For both types of edges, we propose a coupling-process model (DyCPM) to capture the dynamic mechanisms of them. The model not only generates low-dimensional embedding vectors of nodes, but also aggregates the structural information and temporal information of two kinds of edges. In particular, we design a neural network parameterized discrete process to depict the change law of the topology of evolutive edges and a neural network parameterized temporal point process (TPP) to characterize the temporal dynamic rule of interactive edges. More importantly, we propose a coupling mechanism to transfer the information of the two processes through a shared embedding matrix and finally generate an embedding matrix that aggregates the topology information and temporal information of the two kinds of edges for the dynamic link prediction task. We evaluate our model and several baselines on real datasets. The experimental results show that our model can better aggregate the topology information and temporal information of the two kinds of edges according to their properties and outperforms several state-of-the-art baselines in the performance of dynamic link prediction.

Predicting urban signal-controlled intersection congestion events using spatio-temporal neural point process

ABSTRACT The urban traffic signal-controlled intersections are of great significance for solving the problem of urban road congestion. Previous research on congestion prediction mainly aggregated data at the level of road segments or traffic flow at a coarse regulated time interval. Fine-grained prediction of congestion events at the lane-level and cycle-level enables detailed a understanding of spatio-temporal dependencies, leading to congestion reduction, improved efficiency. This paper presents a Spatio-Temporal Neural Point Process (STNPP) model that combines Graph Neural Networks and Neural Temporal Point Process to predict congestion events at urban intersections. The proposed model allows for complete prediction of congestion events, including their occurrence, development, dissipation. In the process of spatial correlation modeling, graph neural networks are used to model the spatial relationships between both region and intersections. The current intersection and its upstream/downstream areas are modeled separately. To model the temporal correlations at individual intersections, we focus on a specific lane and capture the evolution of congestion events using the Neural Point Process Gated Recurrent Unit (NPPGRU), which captures the temporal granularity changes of signal-controlled cycles in congestion events. Using actual traffic speed and signal-controlled data from Hangzhou city, we validate that the proposed method achieves stable predictive performance.

Distribution-free conformal joint prediction regions for neural marked temporal point processes

Distribution-free conformal joint prediction regions for neural marked temporal point processes

Unveiling Multi-Agent Strategies: A Data-Driven Approach for Extracting and Evaluating Team Tactics from Football Event and Freeze-Frame Data

Studying collective behavior in opposing multi-agent teams is crucial across game theory, robotics, and sports analytics. In sports, especially football, team tactics involve intricate strategic spatial and action behaviors displayed as event sequences during possession. Understanding and analyzing these tactics is essential for successful training, strategic planning, and on-field success. While traditional approaches, such as notational and statistical analyses, offer valuable insights into team tactics, they often lack a comprehensive consideration of contextual information, thereby limiting the holistic evaluation of teams’ performances. To bridge this gap and capture the nuanced intricacies of team tactics, we employed advanced methodologies. The sequential pattern mining algorithm PrefixSpan was utilized to extract tactical patterns from possession sequences, enabling a deeper understanding of how teams strategize and adapt during play. Additionally, the neural marked spatio temporal point process (NMSTPP) model was leveraged to model and predict team behaviors, facilitating a fair comparison among teams. The evaluation of team possessions was further enhanced through the innovative holistic possession utilization score metrics, providing a more nuanced assessment of performance. In our experimental exploration, we identified and classified five distinct team tactics, validated the efficacy of the NMSTPP model when integrating StatsBomb 360 data, and conducted a comprehensive analysis of English Premier League teams during the 2022/2023 season. The results were visualized using radar plots and scatter plots with mean shift clustering. Lastly, the potential applications to RoboCup were discussed.

MoMENt: Marked Point Processes with Memory-Enhanced Neural Networks for User Activity Modeling

Marked temporal point process models (MTPPs) aim to model event sequences and event markers (associated features) in continuous time. These models have been applied to various application domains where capturing event dynamics in continuous time is beneficial, such as education systems, social networks, and recommender systems. However, current MTPPs suffer from two major limitations, i.e., inefficient representation of event dynamic’s influence on marker distribution and losing fine-grained representation of historical marker distributions in the modeling. Motivated by these limitations, we propose a novel model called M arked P o int Processes with M emory- E nhanced N eural Ne t works (MoMENt) that can capture the bidirectional interrelations between markers and event dynamics while providing fine-grained marker representations. Specifically, MoMENt is constructed of two concurrent networks: Recurrent Activity Updater (RAU) to capture model event dynamics and Memory-Enhanced Marker Updater (MEMU) to represent markers. Both RAU and MEMU components are designed to update each other at every step to model the bidirectional influence of markers and event dynamics. To obtain a fine-grained representation of maker distributions, MEMU is devised with external memories that model detailed marker-level features with latent component vectors. Our extensive experiments on six real-world user interaction datasets demonstrate that MoMENt can accurately represent users’ activity dynamics, boosting time, type, and marker predictions, as well as recommendation performance up to 76.5%, 65.6%, 77.2%, and 57.7%, respectively, compared to baseline approaches. Furthermore, our case studies show the effectiveness of MoMENt in providing meaningful and fine-grained interpretations of user-system relations over time, e.g., how user choices influence their future preferences in the recommendation domain.

Tapestry of Time and Actions: Modeling Human Activity Sequences Using Temporal Point Process Flows

Human beings always engage in a vast range of activities and tasks that demonstrate their ability to adapt to different scenarios. These activities can range from the simplest daily routines, like walking and sitting, to multi-level complex endeavors such as cooking a four-course meal. Any human activity can be represented as a temporal sequence of actions performed to achieve a certain goal. Unlike the time series datasets extracted from electronics or machines, these action sequences are highly disparate in their nature—the time to finish a sequence of actions can vary between different persons. Therefore, understanding the dynamics of these sequences is essential for many downstream tasks such as activity length prediction, goal prediction, and next action recommendation. Existing neural network based approaches that learn a continuous-time activity sequence are limited to the presence of only visual data or are designed specifically for a particular task (i.e., limited to next action or goal prediction). In this article, we present ProActive , a neural marked temporal point process framework for modeling the continuous-time distribution of actions in an activity sequence while simultaneously addressing three high-impact problems: next action prediction, sequence goal prediction, and end-to-end sequence generation. Specifically, we utilize a self-attention module with temporal normalizing flows to model the influence and the inter-arrival times between actions in a sequence. Moreover, for time-sensitive prediction, we perform an early detection of sequence goal via a constrained margin-based optimization procedure. This in turn allows ProActive to predict the sequence goal using a limited number of actions. In addition, we propose a novel addition over the ProActive model, called ProActive++ , that can handle variations in the order of actions (i.e., different methods of achieving a given goal). We demonstrate that this variant can learn the order in which the person or actor prefers to do their actions. Extensive experiments on sequences derived from three activity recognition datasets show the significant accuracy boost of our ProActive and ProActive++ over the state of the art in terms of action and goal prediction, and the first-ever application of end-to-end action sequence generation.

Citation Forecasting with Multi-Context Attention-Aided Dependency Modeling

Forecasting citations of scientific patents and publications is a crucial task for understanding the evolution and development of technological domains and for foresight into emerging technologies. By construing citations as a time series, the task can be cast into the domain of temporal point processes. Most existing work on forecasting with temporal point processes, both conventional and neural network-based, only performs single-step forecasting. In citation forecasting, however, the more salient goal is n -step forecasting: predicting the arrival of the next n citations. In this article, we propose Dynamic Multi-Context Attention Networks (DMA-Nets), a novel deep learning sequence-to-sequence (Seq2Seq) model with a novel hierarchical dynamic attention mechanism for long-term citation forecasting. Extensive experiments on two real-world datasets demonstrate that the proposed model learns better representations of conditional dependencies over historical sequences compared to state-of-the-art counterparts and thus achieves significant performance for citation predictions.

A Variational Autoencoder for Neural Temporal Point Processes with Dynamic Latent Graphs

Continuously observed event occurrences, often exhibit self and mutually exciting effects, which can be well modeled using temporal point processes. Beyond that, these event dynamics may also change over time, with certain periodic trends. We propose a novel variational autoencoder to capture such a mixture of temporal dynamics. More specifically, the whole time interval of the input sequence is partitioned into a set of sub intervals. The event dynamics are assumed to be stationary within each subinterval, but could be changing across those subintervals. In particular, we use a sequential latent variable model to learn a dependency graph between the observed dimensions, for each subinterval. The model predicts the future event times, by using the learned dependency graph to remove the non contributing influences of past events. By doing so, the proposed model demonstrates its higher accuracy in predicting inter event times and event types for several real world event sequences, compared with existing state of the art neural point processes.

Ch-Oracle A Unified Statistical Framework for Churn Prediction

The User churn stands as a consequential challenge within the realm of online services, posing a substantial threat to the vitality and financial viability of such services. Traditionally, endeavors in churn prediction have transformed the issue into a binary classification task, wherein users are categorized as either churned or non-churned. More recently, a shift towards a more pragmatic approach has been witnessed in the domain of online services, wherein the focus has transitioned from predicting a binary churn label to anticipating the users' return times. This method, aligning more closely with the dynamics of real-world online services, involves the model predicting the specific time of user return at each temporal step, eschewing the simplistic churn label. Nevertheless, antecedent works within this paradigm have grappled with issues of limited generality and imposing computational complexities. This paper introduces ChOracle, an innovative oracle that prognosticates user churn by modeling user return times through the amalgamation of Temporal Point Processes and Recurrent Neural Networks. Furthermore, our approach incorporates latent variables into the proposed recurrent neural network, effectively capturing the latent user loyalty to the system. An efficient approximate variational algorithm, leveraging backpropagation through time, is developed for the purpose of learning parameters within the proposed RNN.

Statistical Depth in Spatial Point Process

Statistical depth is widely used as a powerful tool to measure the center-outward rank of multivariate and functional data. Recent studies have introduced the notion of depth to the temporal point process, which exhibits randomness in the cardinality as well as distribution in the observed events. The proposed methods can well capture the rank of a point process in a given time interval, where a critical step is to measure the rank by using inter-arrival events. In this paper, we propose to extend the depth concept to multivariate spatial point process. In this case, the observed process is in a multi-dimensional location and there are no conventional inter-arrival events in the temporal process. We adopt the newly developed depth in metric space by defining two different metrics, namely the penalized metric and the smoothing metric, to fully explore the depth in the spatial point process. The mathematical properties and the large sample theory, as well as depth-based hypothesis testings, are thoroughly discussed. We then use several simulations to illustrate the effectiveness of the proposed depth method. Finally, we apply the new method in a real-world dataset and obtain desirable ranking performance.

Who Should I Engage With at What Time? A Missing Event-Aware Temporal Graph Neural Network.

Temporal graph neural network (GNN) has recently received significant attention due to its wide application scenarios, such as bioinformatics, knowledge graphs, and social networks. There are some temporal GNNs that achieve remarkable results. However, these works focus on future event prediction and are performed under the assumption that all historical events are observable. In real-world applications, events are not always observable, and estimating event time is as important as predicting future events. In this article, we propose, a missing event-aware temporal GNN, which uniformly models evolving graph structure and timing of events to support predicting what will happen in the future and when it will happen. models the dynamic of both observed and missing events as two coupled temporal point processes (TPPs), thereby incorporating the effects of missing events into the network. Experimental results on several real-world temporal graphs demonstrate that significantly outperforms the existing methods with up to 89% and 112% more accurate time and link prediction. Code can be found on https://github.com/HIT-ICES/TNNLS-MTGN.

Point-process modeling of secondary crashes

Secondary crashes or crashes that occur in the wake of a preceding or primary crash are among the most critical incidents occurring on highways, due to the exceptional danger they present to the first responders and victims of the primary crash. In this work, we developed a self-exciting temporal point process to analyze crash events data and classify it into primary and secondary crashes. Our model uses a self-exciting function to describe secondary crashes while primary crashes are modeled using a background rate function. We fit the model to crash incidents data from the Florida Department of Transportation, on Interstate-4 (I-4) highway for the years 2015–2017, to determine the model parameters. These are used to estimate the probability that a given crash is secondary crash and to find queue times. To represent the periodically varying traffic levels and crash incidents, we model the background rate, as a stationary function, a sinusoidal non-stationary function, and a piecewise non-stationary function. We show that the sinusoidal non-stationary background rate fits the traffic data better and replicates the daily and weekly peaks in crash events due to traffic rush hours. Secondary crashes are found to account for up to 15.09% of traffic incidents, depending on the city on the I-4 Highway.

No Two Users Are Alike: Generating Audiences with Neural Clustering for Temporal Point Processes

No Two Users Are Alike: Generating Audiences with Neural Clustering for Temporal Point Processes

Entropy-Aware Time-Varying Graph Neural Networks with Generalized Temporal Hawkes Process: Dynamic Link Prediction in the Presence of Node Addition and Deletion

This paper addresses the problem of learning temporal graph representations, which capture the changing nature of complex evolving networks. Existing approaches mainly focus on adding new nodes and edges to capture dynamic graph structures. However, to achieve more accurate representation of graph evolution, we consider both the addition and deletion of nodes and edges as events. These events occur at irregular time scales and are modeled using temporal point processes. Our goal is to learn the conditional intensity function of the temporal point process to investigate the influence of deletion events on node representation learning for link-level prediction. We incorporate network entropy, a measure of node and edge significance, to capture the effect of node deletion and edge removal in our framework. Additionally, we leveraged the characteristics of a generalized temporal Hawkes process, which considers the inhibitory effects of events where past occurrences can reduce future intensity. This framework enables dynamic representation learning by effectively modeling both addition and deletion events in the temporal graph. To evaluate our approach, we utilize autonomous system graphs, a family of inhomogeneous sparse graphs with instances of node and edge additions and deletions, in a link prediction task. By integrating these enhancements into our framework, we improve the accuracy of dynamic link prediction and enable better understanding of the dynamic evolution of complex networks.

Proactive Resource Request for Disaster Response: A Deep Learning-Based Optimization Model

In the realm of disaster response operations, effective resource management is crucial. This research introduces an innovative approach that proactively determines the optimal quantities of resources that should be requested by local agencies. This determination is based on both current and anticipated demands, thereby ensuring a more efficient and effective response to disasters. The approach first utilizes a method that combines deep learning and temporal point process for predicting irregularly spaced future demands, and then, it formulates the resource allocation problem faced with randomly arrived demands as a stochastic optimization model. The superiority of this approach over existing resource allocation methods is demonstrated using both real-world data and simulated scenarios. The findings highlight the need for a shift from reactive to proactive strategies. Moreover, the research emphasizes the potential of advanced techniques, such as deep learning and stochastic optimization, in disaster management. These techniques can provide valuable tools for policy makers and practitioners in the field, enabling them to make more informed and effective decisions. Policies that encourage the adoption of such optimized resource allocation strategies could lead to more effective disaster response operations.

Using Deep Learning for Flexible and Scalable Earthquake Forecasting

AbstractSeismology is witnessing explosive growth in the diversity and scale of earthquake catalogs. A key motivation for this community effort is that more data should translate into better earthquake forecasts. Such improvements are yet to be seen. Here, we introduce the Recurrent Earthquake foreCAST (RECAST), a deep‐learning model based on recent developments in neural temporal point processes. The model enables access to a greater volume and diversity of earthquake observations, overcoming the theoretical and computational limitations of traditional approaches. We benchmark against a temporal Epidemic Type Aftershock Sequence model. Tests on synthetic data suggest that with a modest‐sized data set, RECAST accurately models earthquake‐like point processes directly from cataloged data. Tests on earthquake catalogs in Southern California indicate improved fit and forecast accuracy compared to our benchmark when the training set is sufficiently long (>104 events). The basic components in RECAST add flexibility and scalability for earthquake forecasting without sacrificing performance.

Modeling Terror Attacks with Self-Exciting Point Processes and Forecasting the Number of Terror Events.

Rampant terrorism poses a serious threat to the national security of many countries worldwide, particularly due to separatism and extreme nationalism. This paper focuses on the development and application of a temporal self-exciting point process model to the terror data of three countries: the US, Turkey, and the Philippines. To account for occurrences with the same time-stamp, this paper introduces the order mark and reward term in parameter selection. The reward term considers the triggering effect between events in the same time-stamp but different order. Additionally, this paper provides comparisons between the self-exciting models generated by day-based and month-based arrival times. Another highlight of this paper is the development of a model to predict the number of terror events using a combination of simulation and machine learning, specifically the random forest method, to achieve better predictions. This research offers an insightful approach to discover terror event patterns and forecast future occurrences of terror events, which may have practical application towards national security strategies.

Statistical depth for point process via the isometric log-ratio transformation

Statistical depth, a useful tool to measure the center-outward rank of multivariate and functional data, is still under-explored in temporal point processes. Recent studies on point process depth proposed a weighted product of two terms - one indicates the depth of the cardinality of the process, and the other characterizes the conditional depth of the temporal events given the cardinality. The second term is of great challenge because of the apparent nonlinear structure of event times, and so far only parametric representations such as Gaussian and Dirichlet densities have been adopted in the definitions. However, these parametric forms ignore the underlying distribution of the process events and are difficult to apply to complicated patterns. To deal with these problems, a novel distribution-based approach is proposed to the conditional depth via the well-known Isometric Log-Ratio (ILR) transformation on the inter-event times. Motivated by a uniform distribution on simplex, the new method, called the ILR depth, is formally defined for general point process via a Time Rescaling. The mathematical properties of the ILR depth are thoroughly examined and the new method is well illustrated by using Poisson and non-Poisson processes to demonstrate its superiority over previous methods. Finally, the ILR depth is applied in a real dataset and the result clearly shows its effectiveness in ranking.

Dynamic Representation Learning with Temporal Point Processes for Higher-Order Interaction Forecasting

The explosion of digital information and the growing involvement of people in social networks led to enormous research activity to develop methods that can extract meaningful information from interaction data. Commonly, interactions are represented by edges in a network or a graph, which implicitly assumes that the interactions are pairwise and static. However, real-world interactions deviate from these assumptions: (i) interactions can be multi-way, involving more than two nodes or individuals (e.g., family relationships, protein interactions), and (ii) interactions can change over a period of time (e.g., change of opinions and friendship status). While pairwise interactions have been studied in a dynamic network setting and multi-way interactions have been studied using hypergraphs in static networks, there exists no method, at present, that can predict multi-way interactions or hyperedges in dynamic settings. Existing related methods cannot answer temporal queries like what type of interaction will occur next and when it will occur. This paper proposes a temporal point process model for hyperedge prediction to address these problems. Our proposed model uses dynamic representation learning techniques for nodes in a neural point process framework to forecast hyperedges. We present several experimental results and set benchmark results. As far as our knowledge, this is the first work that uses the temporal point process to forecast hyperedges in dynamic networks.