Discrete-time Process Research Articles

On damping a control system with global aftereffect on quantum graphs: Stochastic interpretation

Quantum graphs model processes in complex systems represented as spatial networks in various fields of natural science and technology. An example is the oscillations of elastic string networks, the nodes of which, besides the continuity conditions, also obey the Kirchhoff conditions, expressing the balance of tensions. In this paper, we propose a new look at quantum graphs as temporal networks, which means that the variable parametrizing the edges of a graph is interpreted as time, while each internal vertex is a branching point giving several different scenarios for the further trajectory of a process. Then Kirchhoff‐type conditions may also arise. Namely, they will be satisfied by such a trajectory of the process that is optimal with account of all the scenarios simultaneously. By employing the recent concept of global delay, we extend the problem of damping a first‐order control system with aftereffect, considered earlier only on an interval, to an arbitrary tree graph. The first means that the delay, imposed starting from the initial moment of time, associated with the root of the tree, propagates through all internal vertices. Bringing the system into the equilibrium and minimizing the energy functional with account of the anticipated probability of each scenario, we come to a variational problem. Then, we establish its equivalence to a self‐adjoint boundary value problem on the tree for some second‐order equations involving both the global delay and the global advance. The unique solvability of both problems is proved. We also illustrate that the interval case when the coefficients of the equation are discrete stochastic processes in discrete time can be viewed as the extension to a tree.

On BASTA for Discrete-Time Queues

We address the issue of Bernoulli/Geometric arrivals see time-averages, BASTA, in discrete-time queues when using a variety of scheduling rules. It is well-known that BASTA/ASTA holds in the framework of a discrete-time process with an imbedded arrival process without regard to scheduling rules. However, when scheduling rules are invoked, it is not always clear whether arrivals see time averages as asserted by the general theory. We resolve this issue so that BASTA/ASTA holds regardless of the scheduling rule at play.

Graph Burning in Community-based Networks

Graph burning is a deterministic, discrete-time process that can be used to model how influence or contagion spreads in a graph. In the graph burning process, each node starts as dormant, and becomes informed/burned over time; when a node is burned, it remains burned until the end of the process. In each round, one can burn a new node (source of fire) in the network. Once a node is burned in round $t$, in round $t+1$, each of its dormant neighbors becomes burned. The process ends when all nodes are burned; the goal is to minimize the number of rounds.We study a variation of graph burning in order to model spreading processes in community-based networks. With respect to a specific piece of information, a community is {\em satisfied} when this information reaches at least a prescribed number of its members. Specifically, we consider the problem of identifying a minimum length sequence of nodes that, according to a graph burning process, allows to satisfy all the communities of the network. We investigate this NP-hard problem from an approximation point of view, showing both a lower bound and a matching upper bound. We also investigate the case when the number of communities is constant and show how to solve the problem with a constant approximation factor.Moreover, we consider the problem of maximizing the number of satisfied groups, given a budget $k$ on the number of rounds.

Ruin Probabilities as Recurrence Sequences in a Discrete-Time Risk Process

The theory of linear recurrence sequences is applied to obtain an explicit formula for the ultimate ruin probability in a discrete-time risk process. It is assumed that the claims distribution is arbitrary but has finite support {0,1,…,m+1}\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\varvec{\\{0,1,\\ldots ,m+1\\}}$$\\end{document}, for some integer m≥1\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\varvec{m\\ge 1}$$\\end{document}. The method requires finding the zeroes of an m degree polynomial and solving a system of m linear equations. An approximation is derived and some numerical results and plots are provided as examples.

Reliability assessment of discrete-time [formula omitted](G) retrial system based on different failure types and the [formula omitted]-shock model

The k/n(G) system is a typical reliability system in engineering practice. However, research on this system under discrete-time and retrial conditions is still limited. This paper establishes a k/n(G) retrial system model under a discrete-time condition based on different failure types and the δ-shock model, whose objective is to investigate its various reliability measures and the relationship between measures and parameters. Two methodologies are utilized in this paper to evaluate the system’s various measures: the analytical method for the discrete-time Markov process and the Monte Carlo simulation method. The two methodologies are compared, and optimization is performed on certain measures concerning system parameters. A discrete PH distribution representation method for component lifetime under the δ-shock is presented, the relationship between various reliability measures of the system and various parameters is determined, and a novel approach for system reliability research based on the discrete PH distribution and the δ-shock is provided in this paper.

The Burning Number Conjecture is True for Trees without Degree-2 Vertices

Graph burning is a discrete time process which can be used to model the spread of social contagion. One is initially given a graph of unburned vertices. At each round (time step), one vertex is burned; unburned vertices with at least one burned neighbour from the previous round also becomes burned. The burning number of a graph is the fewest number of rounds required to burn the graph. It has been conjectured that for a graph on n vertices, the burning number is at most ⌈n⌉\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\lceil \\sqrt{n}\\rceil $$\\end{document}. We show that the graph burning conjecture is true for trees without degree-2 vertices.

Study of discrete-time Hawkes process and its compensator

The discrete-time Hawkes process (DTHP) is a sub-class of g-functions that serves as a discrete-time version of the continuous-time Hawkes process (CTHP). Like the CTHP, the DTHP also has the self-exciting property and its intensity depends on the entire history. In this paper, we study the asymptotic behavior of the DTHP and its compensator. We further analyze the moment generating function (MGF) of the DTHP and obtain some bounds and convergence results on the scaled logarithmic MGF of the DTHP.

Deep neural network expressivity for optimal stopping problems

This article studies deep neural network expression rates for optimal stopping problems of discrete-time Markov processes on high-dimensional state spaces. A general framework is established in which the value function and continuation value of an optimal stopping problem can be approximated with error at most ε\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$\\varepsilon $\\end{document} by a deep ReLU neural network of size at most κdqε−r\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$\\kappa d^{\\mathfrak{q}} \\varepsilon ^{-\\mathfrak{r}}$\\end{document}. The constants κ,q,r≥0\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$\\kappa ,\\mathfrak{q},\\mathfrak{r} \\geq 0$\\end{document} do not depend on the dimension d\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$d$\\end{document} of the state space or the approximation accuracy ε\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$\\varepsilon $\\end{document}. This proves that deep neural networks do not suffer from the curse of dimensionality when employed to approximate solutions of optimal stopping problems. The framework covers for example exponential Lévy models, discrete diffusion processes and their running minima and maxima. These results mathematically justify the use of deep neural networks for numerically solving optimal stopping problems and pricing American options in high dimensions.

Evaluating the uncertainty in mean residual times: Estimators based on residence times from discrete time processes

In this work, we propose estimators for the variance of mean residual times, which rely on individual residence times. The latter are assumed to be described by independent and identically distributed random variables, generated by a discrete-time stochastic process. We examine their performance through numerical experiments involving well-known probability distributions, and an application example using molecular dynamics simulation results, from an aqueous NaCl solution, is provided. These computationally inexpensive estimators, capable of achieving very accurate outcomes, serve as useful tools for assessing and reporting uncertainties in mean residual times across a wide range of simulations.

P-adic Brownian motion is a scaling limit

A p-adic Brownian motion is a continuous time stochastic process in a p-adic state space that has a Vladimirov operator as its infinitesimal generator. The current work shows that any such process is the scaling limit of a discrete time random walk on a discrete group. Earlier work required the exponent of the Vladimirov operator to be in (1,∞) , and the convergence was the weak convergence of probability measures on the Skorohod space of paths on a compact time interval. The current approach simplifies the earlier approach, allows for any positive exponent, eliminates the restriction to compact time intervals, and establishes some moment estimates for the discrete time processes that are of independent interest.

Gerber-Shiu theory for discrete risk processes in a regime switching environment

In this paper we develop the Gerber-Shiu theory for the classic and dual discrete risk processes in a Markovian (regime switching) environment. In particular, by expressing the Gerber-Shiu function in terms of potential measures of an upward (downward) skip-free discrete-time and discrete-space Markov Additive Process (MAP), we derive closed form expressions for the Gerber-Shiu function in terms of the so-called (discrete) Wv and Zv scale matrices, which were introduced in [27]. We show that the discrete scale matrices allow for a unified approach for identifying the Gerber-Shiu function as well as the value function of the associated constant dividend barrier problems.

Deep reinforcement learning based on balanced stratified prioritized experience replay for customer credit scoring in peer-to-peer lending

In recent years, deep reinforcement learning (DRL) models have been successfully utilised to solve various classification problems. However, these models have never been applied to customer credit scoring in peer-to-peer (P2P) lending. Moreover, the imbalanced class distribution in experience replay, which may affect the performance of DRL models, has rarely been considered. Therefore, this article proposes a novel DRL model, namely a deep Q-network based on a balanced stratified prioritized experience replay (DQN-BSPER) model, for customer credit scoring in P2P lending. Firstly, customer credit scoring is formulated as a discrete-time finite-Markov decision process. Subsequently, a balanced stratified prioritized experience replay technology is presented to optimize the loss function of the deep Q-network model. This technology can not only balance the numbers of minority and majority experience samples in the mini-batch by using stratified sampling technology but also select more important experience samples for replay based on the priority principle. To verify the model performance, four evaluation measures are introduced for the empirical analysis of two real-world customer credit scoring datasets in P2P lending. The experimental results show that the DQN-BSPER model can outperform four benchmark DRL models and seven traditional benchmark classification models. In addition, the DQN-BSPER model with a discount factor γ of 0.1 has excellent credit scoring performance.

Risk-sensitive discounted Markov decision processes with unbounded reward functions and Borel spaces

This paper attempts to study the risk-sensitive discounted discrete-time Markov decision processes in Borel spaces, in which the reward functions are allowed to be unbounded from above and from below. We find mild conditions imposed on the primitive data of the decision processes, which not only ensure the existence of a solution to the optimality equation (OE in short), but also are the generalization of the bounded reward case. Furthermore, using the OE and a novel technique, we prove the existence of an optimal policy out of the class of randomized history-dependent policies. Finally, we illustrate our results with an inventory system.

Embedding memory-efficient stochastic simulators as quantum trajectories

By exploiting the complexity intrinsic to quantum dynamics, quantum technologies promise a host of computational advantages. One such advantage lies in the field of stochastic modeling, where it has been shown that quantum stochastic simulators can operate with a lower memory overhead than their best classical counterparts. This advantage is particularly pronounced for continuous-time stochastic processes; however, the corresponding quantum stochastic simulators heretofore prescribed operate only on a quasi-continuous-time basis and suffer an ever-increasing circuit complexity with increasing temporal resolution. Here, by establishing a correspondence with quantum trajectories (a method for modeling open quantum systems), we show how truly continuous-time quantum stochastic simulators can be embedded in such open quantum systems, bridging this gap and obviating previous constraints. We further show how such an embedding can be made for discrete-time stochastic processes, which manifest as jump-only trajectories, and discuss how viewing the correspondence in the reverse direction provides a means of studying structural complexity in quantum systems themselves. Published by the American Physical Society 2024

Reinforcement learning strategies using Monte-Carlo to solve the blackjack problem

Blackjack is a classic casino game in which the player attempts to outsmart the dealer by drawing a combination of cards with face values that add up to just under or equal to 21 but are more incredible than the hand of the dealer he manages to come up with. This study considers a simplified variation of blackjack, which has a dealer and plays no active role after the first two draws. A different game regime will be modeled for everyone to ten multiples of the conventional 52-card deck. Irrespective of the number of standard decks utilized, the game is played as a randomized discrete-time process. For determining the optimum course of action in terms of policy, we teach an agent-a decision maker-to optimize across the decision space of the game, considering the procedure as a finite Markov decision chain. To choose the most effective course of action, we mainly research Monte Carlo-based reinforcement learning approaches and compare them with q-learning, dynamic programming, and temporal difference. The performance of the distinct model-free policy iteration techniques is presented in this study, framing the game as a reinforcement learning problem.

Co-offending and the Persistence of Violence: A Dynamic Analysis

ObjectivesPrevious research underscores the influence of prior violent co-offending on subsequent violent behavior, linking it to a social contagion mechanism akin to the internalization of violence. However, these studies are limited by disregarding the entirety of a criminal career and overlooking diverse co-offending dynamics beyond co-offenders’ characteristics. This study examines the longitudinal impact of prior violent (solo and co-) offending on future individual-level violent behavior among Italian organized crime offenders.MethodsLeveraging criminal career data from 9819 Italian organized crime offenders, we model offending choices through a discrete-time Markov process. Subsequently, employing dynamic random-effects probit models, we quantify the influence of prior violent (solo and co-) offending on future violence, considering various confounders and unobserved individual-level effects.ResultsViolence is a persistent and long-lasting behavior among organized crime offenders. Prior violent co-offending has a greater impact than prior violent solo offending on the probability of future violence. Prior violent co-offending increases the probability of future violent co-offending but does not impact the probability of future violent solo offending.ConclusionsThe results show that co-offending promotes the transmission of violence but fail to support the internalization of violent behavior postulated by prior studies. We propose possible alternative mechanisms of violence transmission that operate through self-sustaining dynamics of violent co-offending within criminal groups. Although limited data on individual characteristics constrains interpretation, our results imply that violence transmission dynamics are independent from the individual characteristics of the co-offenders and more directly connected to group effects.

Fluctuations of an omega-type killed process in discrete time

Fluctuations of an omega-type killed process in discrete time

Detecting Abrupt Changes in High-Dimensional Self-Exciting Poisson Processes

High-dimensional self-exciting point processes have been widely used in many application areas to model discrete event data in which past and current events affect the likelihood of future events. In this paper, we are concerned with detecting abrupt changes of the coefficient matrices in discrete-time high-dimensional self-exciting Poisson processes, which have yet to be studied in the existing literature due to both theoretical and computational challenges rooted in the non-stationary and high-dimensional nature of the underlying process. We propose a penalized dynamic programming approach which is supported by a theoretical rate analysis and numerical evidence.

A Continuous Volatility Forecasting Model Based on Neural Differential Equations and Scale-Similarity.

Volatility forecasting is a problem in finance that attracts the attention of both academia and industry. While existing approaches typically utilize a discrete-time latent process that governs the volatility to forecast its future level, volatility is considered to evolve continuously, which makes discrete-time modeling inevitably lose some critical information about the evolution of volatility. In this article, a novel neural-network-based model, Continuous Volatility Forecasting Model, CVFM is proposed to tackle this problem. First, CVFM introduces a continuous-time latent process, whose evolution is modeled with neural differential equations (NDEs), to govern volatility, which effectively captures the continuous evolutionary behavior of volatility in a data-driven way. Second, a scale-similarity-based mechanism is designed to calibrate the evolution equation of the latent process with real-world observations in the absence of high-frequency data. CVFM is tested on six real-world stock index datasets. The main experimental results show that CVFM can significantly outperform existing models in terms of both forecasting accuracy and high-volatility recognition.

Moment stability of stochastic processes with applications to control systems

We establish new conditions for obtaining uniform bounds on the moments of discrete-time stochastic processes. Our results require a weak negative drift criterion along with a state-dependent restriction on the centered conditional moments of the process. They, in particular, generalize the main result of [22] which requires a constant bound on the averaged one-step jumps of the process. The state-dependent feature of our results make them suitable for a large class of multiplicative-noise processes. Under the additional assumption of the Markovian property, we prove new results on ergodicity that do not rely on a minorization condition typically needed in ergodic theorems. Several applications to iterative systems, control systems, and other dynamical systems with state-dependent multiplicative noise are included, and these illustrative examples demonstrate the wide applicability of our results.