Discrete-time Process Research Articles

Graph Burning in Community-based Networks

Graph burning is a deterministic, discrete-time process that can be used to model how influence or contagion spreads in a graph. In the graph burning process, each node starts as dormant, and becomes informed/burned over time; when a node is burned, it remains burned until the end of the process. In each round, one can burn a new node (source of fire) in the network. Once a node is burned in round $t$, in round $t+1$, each of its dormant neighbors becomes burned. The process ends when all nodes are burned; the goal is to minimize the number of rounds.We study a variation of graph burning in order to model spreading processes in community-based networks. With respect to a specific piece of information, a community is {\em satisfied} when this information reaches at least a prescribed number of its members. Specifically, we consider the problem of identifying a minimum length sequence of nodes that, according to a graph burning process, allows to satisfy all the communities of the network. We investigate this NP-hard problem from an approximation point of view, showing both a lower bound and a matching upper bound. We also investigate the case when the number of communities is constant and show how to solve the problem with a constant approximation factor.Moreover, we consider the problem of maximizing the number of satisfied groups, given a budget $k$ on the number of rounds.

Ruin Probabilities as Recurrence Sequences in a Discrete-Time Risk Process

The theory of linear recurrence sequences is applied to obtain an explicit formula for the ultimate ruin probability in a discrete-time risk process. It is assumed that the claims distribution is arbitrary but has finite support {0,1,…,m+1}\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\varvec{\\{0,1,\\ldots ,m+1\\}}$$\\end{document}, for some integer m≥1\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\varvec{m\\ge 1}$$\\end{document}. The method requires finding the zeroes of an m degree polynomial and solving a system of m linear equations. An approximation is derived and some numerical results and plots are provided as examples.

Reliability assessment of discrete-time [formula omitted](G) retrial system based on different failure types and the [formula omitted]-shock model

The k/n(G) system is a typical reliability system in engineering practice. However, research on this system under discrete-time and retrial conditions is still limited. This paper establishes a k/n(G) retrial system model under a discrete-time condition based on different failure types and the δ-shock model, whose objective is to investigate its various reliability measures and the relationship between measures and parameters. Two methodologies are utilized in this paper to evaluate the system’s various measures: the analytical method for the discrete-time Markov process and the Monte Carlo simulation method. The two methodologies are compared, and optimization is performed on certain measures concerning system parameters. A discrete PH distribution representation method for component lifetime under the δ-shock is presented, the relationship between various reliability measures of the system and various parameters is determined, and a novel approach for system reliability research based on the discrete PH distribution and the δ-shock is provided in this paper.

The Burning Number Conjecture is True for Trees without Degree-2 Vertices

Graph burning is a discrete time process which can be used to model the spread of social contagion. One is initially given a graph of unburned vertices. At each round (time step), one vertex is burned; unburned vertices with at least one burned neighbour from the previous round also becomes burned. The burning number of a graph is the fewest number of rounds required to burn the graph. It has been conjectured that for a graph on n vertices, the burning number is at most ⌈n⌉\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\lceil \\sqrt{n}\\rceil $$\\end{document}. We show that the graph burning conjecture is true for trees without degree-2 vertices.

Study of discrete-time Hawkes process and its compensator

The discrete-time Hawkes process (DTHP) is a sub-class of g-functions that serves as a discrete-time version of the continuous-time Hawkes process (CTHP). Like the CTHP, the DTHP also has the self-exciting property and its intensity depends on the entire history. In this paper, we study the asymptotic behavior of the DTHP and its compensator. We further analyze the moment generating function (MGF) of the DTHP and obtain some bounds and convergence results on the scaled logarithmic MGF of the DTHP.

Deep neural network expressivity for optimal stopping problems

This article studies deep neural network expression rates for optimal stopping problems of discrete-time Markov processes on high-dimensional state spaces. A general framework is established in which the value function and continuation value of an optimal stopping problem can be approximated with error at most ε\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$\\varepsilon $\\end{document} by a deep ReLU neural network of size at most κdqε−r\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$\\kappa d^{\\mathfrak{q}} \\varepsilon ^{-\\mathfrak{r}}$\\end{document}. The constants κ,q,r≥0\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$\\kappa ,\\mathfrak{q},\\mathfrak{r} \\geq 0$\\end{document} do not depend on the dimension d\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$d$\\end{document} of the state space or the approximation accuracy ε\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$\\varepsilon $\\end{document}. This proves that deep neural networks do not suffer from the curse of dimensionality when employed to approximate solutions of optimal stopping problems. The framework covers for example exponential Lévy models, discrete diffusion processes and their running minima and maxima. These results mathematically justify the use of deep neural networks for numerically solving optimal stopping problems and pricing American options in high dimensions.

Evaluating the uncertainty in mean residual times: Estimators based on residence times from discrete time processes

In this work, we propose estimators for the variance of mean residual times, which rely on individual residence times. The latter are assumed to be described by independent and identically distributed random variables, generated by a discrete-time stochastic process. We examine their performance through numerical experiments involving well-known probability distributions, and an application example using molecular dynamics simulation results, from an aqueous NaCl solution, is provided. These computationally inexpensive estimators, capable of achieving very accurate outcomes, serve as useful tools for assessing and reporting uncertainties in mean residual times across a wide range of simulations.

P-adic Brownian motion is a scaling limit

A p-adic Brownian motion is a continuous time stochastic process in a p-adic state space that has a Vladimirov operator as its infinitesimal generator. The current work shows that any such process is the scaling limit of a discrete time random walk on a discrete group. Earlier work required the exponent of the Vladimirov operator to be in (1,∞) , and the convergence was the weak convergence of probability measures on the Skorohod space of paths on a compact time interval. The current approach simplifies the earlier approach, allows for any positive exponent, eliminates the restriction to compact time intervals, and establishes some moment estimates for the discrete time processes that are of independent interest.

Gerber-Shiu theory for discrete risk processes in a regime switching environment

In this paper we develop the Gerber-Shiu theory for the classic and dual discrete risk processes in a Markovian (regime switching) environment. In particular, by expressing the Gerber-Shiu function in terms of potential measures of an upward (downward) skip-free discrete-time and discrete-space Markov Additive Process (MAP), we derive closed form expressions for the Gerber-Shiu function in terms of the so-called (discrete) Wv and Zv scale matrices, which were introduced in [27]. We show that the discrete scale matrices allow for a unified approach for identifying the Gerber-Shiu function as well as the value function of the associated constant dividend barrier problems.

Deep reinforcement learning based on balanced stratified prioritized experience replay for customer credit scoring in peer-to-peer lending

In recent years, deep reinforcement learning (DRL) models have been successfully utilised to solve various classification problems. However, these models have never been applied to customer credit scoring in peer-to-peer (P2P) lending. Moreover, the imbalanced class distribution in experience replay, which may affect the performance of DRL models, has rarely been considered. Therefore, this article proposes a novel DRL model, namely a deep Q-network based on a balanced stratified prioritized experience replay (DQN-BSPER) model, for customer credit scoring in P2P lending. Firstly, customer credit scoring is formulated as a discrete-time finite-Markov decision process. Subsequently, a balanced stratified prioritized experience replay technology is presented to optimize the loss function of the deep Q-network model. This technology can not only balance the numbers of minority and majority experience samples in the mini-batch by using stratified sampling technology but also select more important experience samples for replay based on the priority principle. To verify the model performance, four evaluation measures are introduced for the empirical analysis of two real-world customer credit scoring datasets in P2P lending. The experimental results show that the DQN-BSPER model can outperform four benchmark DRL models and seven traditional benchmark classification models. In addition, the DQN-BSPER model with a discount factor γ of 0.1 has excellent credit scoring performance.

Risk-sensitive discounted Markov decision processes with unbounded reward functions and Borel spaces

This paper attempts to study the risk-sensitive discounted discrete-time Markov decision processes in Borel spaces, in which the reward functions are allowed to be unbounded from above and from below. We find mild conditions imposed on the primitive data of the decision processes, which not only ensure the existence of a solution to the optimality equation (OE in short), but also are the generalization of the bounded reward case. Furthermore, using the OE and a novel technique, we prove the existence of an optimal policy out of the class of randomized history-dependent policies. Finally, we illustrate our results with an inventory system.

Reinforcement learning strategies using Monte-Carlo to solve the blackjack problem

Blackjack is a classic casino game in which the player attempts to outsmart the dealer by drawing a combination of cards with face values that add up to just under or equal to 21 but are more incredible than the hand of the dealer he manages to come up with. This study considers a simplified variation of blackjack, which has a dealer and plays no active role after the first two draws. A different game regime will be modeled for everyone to ten multiples of the conventional 52-card deck. Irrespective of the number of standard decks utilized, the game is played as a randomized discrete-time process. For determining the optimum course of action in terms of policy, we teach an agent-a decision maker-to optimize across the decision space of the game, considering the procedure as a finite Markov decision chain. To choose the most effective course of action, we mainly research Monte Carlo-based reinforcement learning approaches and compare them with q-learning, dynamic programming, and temporal difference. The performance of the distinct model-free policy iteration techniques is presented in this study, framing the game as a reinforcement learning problem.

Co-offending and the Persistence of Violence: A Dynamic Analysis

ObjectivesPrevious research underscores the influence of prior violent co-offending on subsequent violent behavior, linking it to a social contagion mechanism akin to the internalization of violence. However, these studies are limited by disregarding the entirety of a criminal career and overlooking diverse co-offending dynamics beyond co-offenders’ characteristics. This study examines the longitudinal impact of prior violent (solo and co-) offending on future individual-level violent behavior among Italian organized crime offenders.MethodsLeveraging criminal career data from 9819 Italian organized crime offenders, we model offending choices through a discrete-time Markov process. Subsequently, employing dynamic random-effects probit models, we quantify the influence of prior violent (solo and co-) offending on future violence, considering various confounders and unobserved individual-level effects.ResultsViolence is a persistent and long-lasting behavior among organized crime offenders. Prior violent co-offending has a greater impact than prior violent solo offending on the probability of future violence. Prior violent co-offending increases the probability of future violent co-offending but does not impact the probability of future violent solo offending.ConclusionsThe results show that co-offending promotes the transmission of violence but fail to support the internalization of violent behavior postulated by prior studies. We propose possible alternative mechanisms of violence transmission that operate through self-sustaining dynamics of violent co-offending within criminal groups. Although limited data on individual characteristics constrains interpretation, our results imply that violence transmission dynamics are independent from the individual characteristics of the co-offenders and more directly connected to group effects.

Fluctuations of an omega-type killed process in discrete time

Fluctuations of an omega-type killed process in discrete time

Detecting Abrupt Changes in High-Dimensional Self-Exciting Poisson Processes

High-dimensional self-exciting point processes have been widely used in many application areas to model discrete event data in which past and current events affect the likelihood of future events. In this paper, we are concerned with detecting abrupt changes of the coefficient matrices in discrete-time high-dimensional self-exciting Poisson processes, which have yet to be studied in the existing literature due to both theoretical and computational challenges rooted in the non-stationary and high-dimensional nature of the underlying process. We propose a penalized dynamic programming approach which is supported by a theoretical rate analysis and numerical evidence.

A Continuous Volatility Forecasting Model Based on Neural Differential Equations and Scale-Similarity.

Volatility forecasting is a problem in finance that attracts the attention of both academia and industry. While existing approaches typically utilize a discrete-time latent process that governs the volatility to forecast its future level, volatility is considered to evolve continuously, which makes discrete-time modeling inevitably lose some critical information about the evolution of volatility. In this article, a novel neural-network-based model, Continuous Volatility Forecasting Model, CVFM is proposed to tackle this problem. First, CVFM introduces a continuous-time latent process, whose evolution is modeled with neural differential equations (NDEs), to govern volatility, which effectively captures the continuous evolutionary behavior of volatility in a data-driven way. Second, a scale-similarity-based mechanism is designed to calibrate the evolution equation of the latent process with real-world observations in the absence of high-frequency data. CVFM is tested on six real-world stock index datasets. The main experimental results show that CVFM can significantly outperform existing models in terms of both forecasting accuracy and high-volatility recognition.

Moment stability of stochastic processes with applications to control systems

We establish new conditions for obtaining uniform bounds on the moments of discrete-time stochastic processes. Our results require a weak negative drift criterion along with a state-dependent restriction on the centered conditional moments of the process. They, in particular, generalize the main result of [22] which requires a constant bound on the averaged one-step jumps of the process. The state-dependent feature of our results make them suitable for a large class of multiplicative-noise processes. Under the additional assumption of the Markovian property, we prove new results on ergodicity that do not rely on a minorization condition typically needed in ergodic theorems. Several applications to iterative systems, control systems, and other dynamical systems with state-dependent multiplicative noise are included, and these illustrative examples demonstrate the wide applicability of our results.

A Review of Birth-Death and Other Markovian Discrete-Time Queues

In this review article, we consider discrete-time birth-death processes and their applications to discrete-time queues. To make the analysis simpler to follow, we focus on transform-free methods and consider instances of non-birth-death Markovian discrete-time systems. We present a number of results within one discrete-time framework that parallels the treatment of continuous time models. This approach has two advantages; first, it unifies the treatment of several discrete-time models in one framework, and second, it parallels to the extent possible the treatment of continuous time models. This allows us to draw parallels and contrasts between the discrete and continuous time queues. Specifically, we focus on birth-death applications to the single server discrete-time model with Bernoulli arrivals and geometric service times and provide the reader with a simple rigorous detailed analysis that covers all five scheduling rules considered in the literature, with attention to stationary distributions at slot edges, slot centers, and prearrival epochs. We also cover the waiting time distributions. Moreover, we cover three Markovian models that fit the global balance equations. Our approach provides interesting insights into the behavior of discrete-time queues. The article is intended for those who are familiar with queueing theory basics and would like a simple, yet rigorous introductory treatment to discrete-time queues.

A simulation-based approximate dynamic programming approach to dynamic and stochastic resource-constrained multi-project scheduling problem

We consider the dynamic and stochastic resource-constrained multi-project scheduling problem which allows for the random arrival of projects and stochastic task durations. Completing projects generates rewards, which are reduced by a tardiness cost in the case of late completion. Multiple types of resource are available, and projects consume different amounts of these resources when under processing. The problem is modelled as an infinite-horizon discrete-time Markov decision process and seeks to maximise the expected discounted long-run profit. We use an approximate dynamic programming algorithm (ADP) with a linear approximation model which can be used for online decision making. Our approximation model uses project elements that are easily accessible by a decision-maker, with the model coefficients obtained offline via a combination of Monte Carlo simulation and least squares estimation. Our numerical study shows that ADP often statistically significantly outperforms the optimal reactive baseline algorithm (ORBA). In experiments on smaller problems however, both typically perform suboptimally compared to the optimal scheduler obtained by stochastic dynamic programming. ADP has an advantage over ORBA and dynamic programming in that ADP can be applied to larger problems. We also show that ADP generally produces statistically significantly higher profits than common algorithms used in practice, such as a rule-based algorithm and a reactive genetic algorithm.

Structural properties of optimal fidelity selection policies for human-in-the-loop queues

We study optimal fidelity selection for a human operator servicing a queue of homogeneous tasks. The agent can service a task with a normal or high fidelity level, where fidelity refers to the degree of exactness and precision while servicing the task. Therefore, high-fidelity servicing results in higher-quality service but leads to larger service times and increased operator tiredness. We treat the human cognitive state as a lumped parameter that captures psychological factors such as workload and fatigue. The operator’s service time distribution depends on her cognitive dynamics and the fidelity level selected for servicing the task. Her cognitive dynamics evolve as a Markov chain in which the cognitive state increases with high probability whenever she is busy and decreases while resting. The tasks arrive according to a Poisson process and the operator is penalized at a fixed rate for each task waiting in the queue. We address the trade-off between high-quality service of the task and consequent penalty due to a subsequent increase in queue length using a discrete-time Semi-Markov Decision Process framework. We numerically determine an optimal policy and the corresponding optimal value function. Finally, we establish the structural properties of an optimal fidelity policy and provide conditions under which the optimal policy is a threshold-based policy.