Discrete-time Setting Research Articles

Siegmund duality for physicists: a bridge between spatial and first-passage properties of continuous- and discrete-time stochastic processes

Abstract We consider a generic one-dimensional stochastic process x(t), or a random walk Xn , which describes the position of a particle evolving inside an interval [ a , b ] , with absorbing walls located at a and b. In continuous time, x(t) is driven by some equilibrium process θ ( t ) , while in discrete time, the jumps of Xn follow a stationary process that obeys a time-reversal property. An important observable to characterize its behavior is the exit probability E b ( x , t ) , which is the probability for the particle to be absorbed first at the wall b, before or at time t, given its initial position x. In this paper we show that the derivation of this quantity can be tackled by studying a dual process y(t) very similar to x(t) but with hard walls at a and b. More precisely, we show that the quantity E b ( x , t ) for the process x(t) is equal to the probability Φ ~ ( x , t | b ) of finding the dual process inside the interval [ a , x ] at time t, with y ( 0 ) = b . This is known as Siegmund duality in mathematics. Here we show that this duality applies to various processes that are of interest in physics, including models of active particles, diffusing diffusivity models, a large class of discrete- and continuous-time random walks, and even processes subjected to stochastic resetting. For all these cases, we provide an explicit construction of the dual process. We also give simple derivations of this identity both in the continuous and in the discrete time setting, as well as numerical tests for a large number of models of interest. Finally, we use simulations to show that the duality is also likely to hold for more complex processes, such as fractional Brownian motion.

Persistency and stability of a class of nonlinear forced positive discrete-time systems with delays

Persistence, excitability and stability properties are considered for a class of nonlinear, forced, positive discrete-time systems with delays. As will be illustrated, these equations arise in a number of biological and ecological contexts. Novel sufficient conditions for persistence, excitability and stability are presented. Further, similarities and differences between the delayed equations considered presently and their corresponding undelayed versions are explored, and some striking differences are noted. It is shown that recent results for a corresponding class of positive, nonlinear delay-differential (continuous-time) systems do not carry over to the discrete-time setting. Detailed discussion of three examples from population dynamics is provided.

Sparse dynamic discretization discovery via arc-dependent time discretizations

While many time-dependent network design problems can be formulated as time-indexed formulations with strong relaxations, the size of these formulations depends on the discretization of the time horizon and can become prohibitively large. The recently-developed dynamic discretization discovery (DDD) method allows many time-dependent problems to become more tractable by iteratively solving instances of the problem on smaller networks where each node has its own discrete set of departure times. However, in the current implementation of DDD, all arcs departing a common node share the same set of departure times. This causes DDD to be ineffective for solving problems where all near-optimal solutions require many distinct departure times at the majority of the high-degree nodes in the network. Region-based networks are one such structure that often leads to many high-degree nodes, and their increasing popularity underscores the importance of tailoring solution methods for these networks.To improve methods for solving problems that require many departure times at nodes, we develop a DDD framework where the set of departure times is determined on the arc level rather than the node level. We apply this arc-based DDD method to a temporal variant of the service network design problem (SND). We show that an arc-based approach is particularly advantageous when instances arise from region-based networks, and when candidate paths are fixed in the base graph for each commodity. Moreover, our algorithm builds upon the existing DDD framework and achieves these improvements with only benign modifications to the original implementation.

Is Kyle’s equilibrium model stable?

AbstractIn the dynamic discrete-time trading setting of Kyle (Econometrica 53:1315–1336, 1985), we prove that Kyle’s equilibrium model is stable when there are one or two trading times. For three or more trading times, we prove that Kyle’s equilibrium is not stable. These theoretical results are proven to hold irrespectively of all Kyle’s input parameters.

Time-Variant Reliability for Systems with Non-monotonic Limit-State Functions

Most engineering time-variant reliability problems are the result of component degradation and stochastic loading. The resultant failure modes, and their resultant limit-state functions, produce limit-state surfaces with unpredictable temporal trajectories that may exhibit a combination of increasing and decreasing failure probabilities. In many cases the trajectories are monotonic so that failure increases predictably: in other cases, this is not so. In this paper we present the discrete-time set theory derivation for non-monotonic situations wherein the limit-state surface may recede to provide, what only appears to be, ever decreasing failure probability. The presence of both monotonic and non-monotonic limit-state functions can be easily detected by a parametric polar plot of the most-likely failure points in standard normal space. The polar plot reveals the temporal limit-state surfaces that need to be retained to represent the system limit-state surfaces at any time instant. The minimum set herein is called the extreme limit-state surface. The impact of the work is that the cumulative distribution function (cdf) can be provided with a minimum of failure and safe events. This in turn gives rise to several solution options such as the multi-normal integral method or a special Monte Carlo simulation that obviates the tedious marching-out routine. A series system and a parallel system show the efficacy of the theory.

Adaptive impulsive consensus of nonlinear multi-agent systems via a distributed self-triggered strategy

This article investigates the consensus problem of nonlinear leader-following multi-agent systems (MASs) via adaptive impulsive control. A distributed impulsive controller is employed, in which an adaptive updating law in the discrete-time setting is designed for the impulsive gain. Moreover, in order to reduce the communication cost, a distributed self-triggered strategy is proposed to determine when the impulsive instant occurs. Some criteria are derived to guarantee consensus of leader-following MASs based on Lyapunov stability theory and algebraic Riccati equation. It is proved that the self-triggered impulsive sequence does not exhibit Zeno behaviour. Finally, an illustrative example is presented to empirically validate the effectiveness of the theoretical results.

Equilibrium multi-period investment strategy for a DC pension plan with incomplete information: Hidden Markov model

In the financial market with both observable and unobservable market states, this article explores the equilibrium investment strategy for a DC pension plan with the mean-variance criterion in a discrete-time setting. The dynamics of the partially observed market state are described by a discrete-time finite-state hidden Markov chain. There is a riskless asset and a risky asset in the financial market, where the return rate of the risky asset depends both on the observable and unobservable market states. Meanwhile, the stochastic salary process is also modulated by the observable and unobservable market states. Under the framework of non cooperative game, we first define the equilibrium investment strategy for the multi-period mean-variance DC pension plan. By adopting the sufficient statistics method, the investment problem for the mean-variance DC pension plan with incomplete information is transformed into the one with complete information. The closed-form equilibrium investment strategy is derived by solving the extended Bellman equation. Finally, numerical results show that the incomplete information has significant impacts on the equilibrium investment strategy and the equilibrium efficient frontier. Neglecting the reality of incomplete information in the financial market will reduce the investment benefit of the DC pension plan. The longer the investment horizon, the greater the investment benefit loss for the DC pension plan.

The chemostat reactor: A stability analysis and model predictive control

This contribution develops the model predictive control for an unstable chemostat reactor. Initially, we analyze the system’s model — a nonlinear first-order hyperbolic partial integro-differential equation (PIDE) — and carry the model linearization around an unstable operating condition. Employing the structure-preserving Cayley–Tustin transformation, we obtain a discrete-time model representation of the continuous model. Subsequently, we solve the operator Ricatti equations in the discrete-time setting to derive a full state feedback controller that stabilizes the closed-loop and design a Luenberger observer for state reconstruction given the system output measures. Finally, we formulate a dual-mode MPC ensuring constraint satisfaction and optimality, integrating the gain-based unconstrained full-state feedback optimal control obtained from the Ricatti equation. This dual-mode strategy describes an optimization problem where the predictive controller acts only if constraints become active within the control horizon. Simulation studies validate the controller performance, where the MPC only takes action if the constraints are predicted to be active within the control horizon while also guaranteeing closed-loop stabilization under only output feedback. This type of controller can be easily implemented with other control strategies and significantly decreases the computational costs of solving the optimal control problems when compared to other MPC approaches.

A Framework for Leveraging Machine Learning Tools to Estimate Personalized Survival Curves

The conditional survival function of a time-to-event outcome subject to censoring and truncation is a common target of estimation in survival analysis. This parameter may be of scientific interest and also often appears as a nuisance in nonparametric and semiparametric problems. In addition to classical parametric and semiparametric methods (e.g., based on the Cox proportional hazards model), flexible machine learning approaches have been developed to estimate the conditional survival function. However, many of these methods are either implicitly or explicitly targeted toward risk stratification rather than overall survival function estimation. Others apply only to discrete-time settings or require inverse probability of censoring weights, which can be as difficult to estimate as the outcome survival function itself. Here, we employ a decomposition of the conditional survival function in terms of observable regression models in which censoring and truncation play no role. This allows application of an array of flexible regression and classification methods rather than only approaches that explicitly handle the complexities inherent to survival data. We outline estimation procedures based on this decomposition, empirically assess their performance, and demonstrate their use on data from an HIV vaccine trial. Supplementary materials for this article are available online.

Almost sure stability criteria for linear Markovian switching systems

We present some stability criteria mainly for continuous-time Markovian switching systems, which can provide us with an insight into the mechanism that drives the subsystems to interact with each other so as to achieve stability. On the one hand, directly computing the Lyapunov exponent through the state-transition matrix of the system under consideration allows us to capture the interaction of subsystems by using the method developed in discrete-time setting. On the other hand, we can make sense of how the subsystems are orchestrated by the statistical property of switching signal through a family of coupled Lyapunov inequalities, which naturally covers the stability in the mean-square sense as a special case. Three examples are included to illustrate the theoretical results.

Discrete-time Control by Interconnection using energy-preserving collocation methods

The energy-preserving collocation methods combine a high order of accuracy with an exact discrete-time energy balance equation when discretizing port-Hamiltonian systems. We show how they can be used to translate Control by Interconnection to the discrete-time setting. We set up the discrete-time plant and controller model and compute the discrete-time inputs for which the desired equilibrium of the continuous-time closed-loop system becomes a stable equilibrium of the discrete-time models. To obtain a continuous-time control signal for the plant system in the sampling interval, these discrete inputs are then interpolated by a polynomial. The advantages of the proposed approach at large sampling times are illustrated in simulations using the example of the controlled pendulum.

Who benefits from postponement in multi-period supply channel optimization?

Duopolistic price-setting supply channels competing in a bilevel framework have been extensively studied in single-period (static) settings. However, such supply channels typically face uncertain and time-varying demand; and thus require a dynamic analysis. Dynamic channel optimization while addressing uncertain demand has received limited attention due to the highly nested structure of the ensuing equilibrium problems. The level of complexity rises when demand is dependent on current and previous prices. We consider a decentralized (non-cooperative) supply channel whose members, a manufacturer and a retailer, competing in a Stackelberg framework, must address the demand for a perishable commodity within a multi-period discrete-time setting. In the first part of the paper, we propose a constructive theorem providing an explicit solution algorithm to obtain equilibrium states at each period. Next, we prove that the resulting equilibria are subgame perfect. In the second part, we allow the retailer (follower) to postpone the supply and pricing decisions until demand uncertainty is resolved in each period. Using subgame perfection of the equilibria, we propose solution algorithms that use the delayed information obtained by the postponement. Our comparative theorems and simulated scenarios indicate that postponement strategies are always beneficial for the follower, and, for a centralized (cooperative) channel. Whereas in a decentralized channel, due to vertical competition, there may be scenarios wherein postponement strategies, i.e., access to extra information, turn out to be detrimental to the manufacturer (leader).

The general maximum principle for discrete-time stochastic control problems

In this paper, we discuss the problem of discrete-time stochastic control with multiplicative noise, where the control domains may not be convex. We propose a novel approach inspired by the classical spike variation in continuous-time cases, but adapted to the discrete-time setting by perturbing in a random scale instead of time scale. Our approach allows us to obtain the maximum principle recursively without the need for variation equations or adjoint equations, which are typically required in previous methods. Moreover, we provide two maximum principles, one for the general case and the other when feedback forms exist. Our approach covers the previous results when the control domains are convex. Finally, we demonstrate the effectiveness of our new results through an example.

Discrete Bismut formula: Conditional integration by parts and a representation for delta hedging process

The paper gives discrete conditional integration by parts formula using a Malliavin calculus approach in discrete-time setting. Then the discrete Bismut formula is introduced for asymmetric random walk model and asymmetric exponential process. In particular, a new formula for delta hedging process is obtained as an extension of the Malliavin derivative representation of the delta where the conditional integration by parts formula plays a role in the proof.

On-the-Fly Control of Unknown Systems: From Side Information to Performance Guarantees Through Reachability

We develop data-driven algorithms for reachability analysis and control of systems with a priori unknown nonlinear dynamics. The resulting algorithms not only are suitable for settings with real-time requirements but also provide provable performance guarantees. To this end, they merge noisy data from only a single finite-horizon trajectory and, if available, various forms of side information. Such side information may include knowledge of the regularity of the dynamics, algebraic constraints on the states, monotonicity, or decoupling in the dynamics between the states. Specifically, we develop two algorithms, <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">DaTaReach</monospace> and <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">DaTaControl</monospace> , to over-approximate the reachable set and design control signals for the system on the fly. <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">DaTaReach</monospace> constructs a differential inclusion that contains the unknown dynamics. Then, in a discrete-time setting, it over-approximates the reachable set through interval Taylor-based methods applied to systems with dynamics described as differential inclusions. We provide a bound on the time step size that ensures the correctness and termination of <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">DaTaReach</monospace> . <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">DaTaControl</monospace> enables convex-optimization-based control using the computed over-approximation and the receding-horizon control framework. Besides, <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">DaTaControl</monospace> achieves near-optimal control and is suitable for real-time control of such systems. We establish a bound on its suboptimality and the number of primitive operations it requires to compute control values. Then, we theoretically show that <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">DaTaControl</monospace> achieves tighter suboptimality bounds with an increasing amount of data and richer side information. Finally, experiments on a unicycle, quadrotor, and aircraft systems demonstrate the efficacy of both algorithms over existing approaches.

Linear fixed-effects estimation with nonrepeated outcomes

We demonstrate that popular linear fixed-effects panel-data estimators are biased and inconsistent when applied in a discrete-time hazard setting, even if the data-generating process is consistent with the linear model. The bias is not just survival bias, but originates from the impossibility to transform the model such that the remaining disturbance term becomes conditional mean independent of the explanatory variables. The bias is hence present even in the absence of unobserved heterogeneity. We discuss instrumental variables estimation, using first-differences of the explanatory variables as instruments, as alternative estimation strategy. Monte Carlo simulations and an empirical application substantiate our theoretical results.

Stability properties of a Cournot-type oligopoly model with time delay

Abstract The paper studies a Cournot game involving n private firms, where the firms’ interactions are analyzed in a discrete-time setting with time delay. The past production levels of other private firms have an impact on the production levels of each private firm. We performed a local stability analysis on the nonlinear system associated with the model and found that it has only one equilibrium point, which is asymptotically stable if and only if some conditions involving the number of firms, the degree of product di erentiation and the time delay are fulfilled. In addition, numerical examples are presented in this paper, which illustrate the theoretical results obtained.

Formal Verification of Unknown Discrete- and Continuous-Time Systems: A Data-Driven Approach

This paper is concerned with a formal verification scheme for both discrete- and continuous-time deterministic systems with unknown mathematical models. The main target is to verify the safety of unknown systems based on the construction of barrier certificates via a set of data collected from trajectories of systems while providing an a-priori guaranteed confidence on the safety. In our proposed framework, we first cast the original safety problem as a robust convex program (RCP). Solving the proposed RCP is not tractable in general since the unknown model appears in one of the constraints. Instead, we collect finite numbers of data from trajectories of the system and provide a scenario convex program (SCP) corresponding to the original RCP. We then establish a probabilistic closeness between the optimal value of SCP and that of RCP, and as a result, we formally quantify the safety guarantee of unknown systems based on the number of data points and a required level of confidence. We propose our framework in both discrete-time and continuous-time settings. We illustrate the effectiveness of our proposed results by first applying them to an unknown continuous-time room temperature system. We verify that the temperature of the room maintains in a comfort zone with some desirable confidence by collecting data from trajectories of the system. To show the applicability of our techniques to <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">higher dimensional</i> systems with <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">nonlinear</i> dynamics, we then apply our results to a continuous-time <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">nonlinear</i> jet engine compressor and a discrete-time DC motor.

Limit theorems for discrete multitype branching processes counted with a characteristic

For a discrete time multitype supercritical Galton–Watson process (Zn)n∈N and corresponding genealogical tree T, we associate a new discrete time process (ZnΦ)n∈N such that, for each n∈N, the contribution of each individual u∈T to ZnΦ is determined by a (random) characteristic Φ evaluated at the age of u at time n. In other words, ZnΦ is obtained by summing over all u∈T the corresponding contributions Φu, where (Φu)u∈T are i.i.d. copies of Φ. Such processes are known in the literature under the name of Crump–Mode–Jagers (CMJ) processes counted with characteristicΦ. We derive a LLN and a CLT for the process (ZnΦ)n∈N in the discrete time setting, and in particular, we show a dichotomy in its limit behavior. By applying our main result, we also obtain a generalization of the results in Kesten and Stigum (1966).

Behavioural equivalences for continuous-time Markov processes

AbstractBisimulation is a concept that captures behavioural equivalence of states in a variety of types of transition systems. It has been widely studied in a discrete-time setting. The core of this work is to generalise the discrete-time picture to continuous time by providing a notion of behavioural equivalence for continuous-time Markov processes. In Chen et al. [(2019). Electronic Notes in Theoretical Computer Science347 45–63.], we proposed two equivalent definitions of bisimulation for continuous-time stochastic processes where the evolution is a flow through time: the first one as an equivalence relation and the second one as a cospan of morphisms. In Chen et al. [(2020). Electronic Notes in Theoretical Computer Science.], we developed the theory further: we introduced different concepts that correspond to different behavioural equivalences and compared them to bisimulation. In particular, we studied the relation between bisimulation and symmetry groups of the dynamics. We also provided a game interpretation for two of the behavioural equivalences. The present work unifies the cited conference presentations and gives detailed proofs.