Discrete-time Setting Research Articles

Discrete and continuous strategies for timed-arc Petri net games

Automatic strategy synthesis for a given control objective can be used to generate correct-by-construction controllers of real-time reactive systems. The existing symbolic approach for continuous timed game is a computationally hard task and current tools like UPPAAL TiGa often scale poorly with the model complexity. We suggest an explicit approach for strategy synthesis in the discrete-time setting and show that even for systems with closed guards, the existence of a safety discrete-time strategy does not imply the existence of a safety continuous-time strategy and vice versa. Nevertheless, we prove that the answers to the existence of discrete-time and continuous-time safety strategies coincide on a practically motivated subclass of urgent controllers that either react immediately after receiving an environmental input or wait with the decision until a next event is triggered by the environment. We then develop an on-the-fly synthesis algorithm for discrete timed-arc Petri net games. The algorithm is implemented in our tool TAPAAL, and based on the experimental evidence, we discuss the advantages of our approach compared to the symbolic continuous-time techniques.

Discrete-Time Mean-CVaR Portfolio Selection and Time-Consistency Induced Term Structure of the CVaR

We investigate a discrete-time mean-risk portfolio selection problem, where risk is measured by the conditional value-at-risk (CVaR). By embedding this time-inconsistent problem into a family of expected utility maximization problems with a piecewise linear utility function, we solve the problem analytically. In contrast to the case of a complete, continuous-time market, the mean-CVaR efficient frontier in this generally incomplete, discrete-time setting is a straight line in the mean-CVaR plane and there is in particular a constant trade-off between risk and return. The cumulated amount invested in the risky assets under the optimal strategy is of a V -shaped pattern as a function of the current wealth. We further solve an inverse investment problem, where we investigate how mean-CVaR preferences need to adapt such that the pre-commited optimal strategy remains optimal at any point in time. Our result shows that, although conceptually distinct, a pre-commited mean-CVaR investor behaves like a naive mean-CVaR investor with a time-increasing confidence level for the CVaR, who revises her investment decision at every point in time. Finally, an empirical application of our results suggests that risk measured by the CVaR might help to understand the long-standing equity premium puzzle.

Rate of convergence to equilibrium for discrete-time stochastic dynamics with memory

The main objective of the paper is to study the long-time behavior of general discrete dynamics driven by an ergodic stationary Gaussian noise. In our main result, we prove existence and uniqueness of the invariant distribution and exhibit some upper-bounds on the rate of convergence to equilibrium in terms of the asymptotic behavior of the covariance function of the Gaussian noise (or equivalently to its moving average representation). Then, we apply our general results to fractional dynamics (including the Euler Scheme associated to fractional driven Stochastic Differential Equations). When the Hurst parameter $H$ belongs to $(0,1/2)$ we retrieve, with a slightly more explicit approach due to the discrete-time setting, the rate exhibited by Hairer in a continuous time setting (Ann. Probab. 33 (2005) 703–758). In this fractional setting, we also emphasize the significant dependence of the rate of convergence to equilibrium on the local behaviour of the covariance function of the Gaussian noise.

Dynamic treatment assignment and evaluation of active labor market policies

This paper considers treatment evaluation in a discrete time setting in which treatment can start at any point in time. We consider evaluation under unconfoundedness and propose a dynamic inverse probability weighting estimator. A typical application is an active labor market program that can start after any elapsed unemployment duration. The identification and estimation results concern both cases with one single treatment as well as sequences of programs. The new estimator is applied to Swedish data on participants in a training program and a work practice program. The work practice program increases re-employment rates. Most sequences of the two programs are inefficient when compared to one single program episode.

Evaluating the accuracy of biodiversity changes through geologic times: from simulation to solution

AbstractEstimating biodiversity and its variations through geologic time is a notoriously difficult task, due to several taphonomic and methodological effects that make the reconstructed signal potentially distinct from the unknown, original one. Through a simulation approach, we examine the effect of a major, surprisingly still understudied, source of potential disturbance: the effect of time discretization through biochronological construction, which generates spurious coexistences of taxa within discrete time intervals (i.e., biozones), and thus potentially makes continuous- and discrete-time biodiversity curves very different. Focusing on the taxonomic-richness dimension of biodiversity (including estimates of origination and extinction rates), our approach relies on generation of random continuous-time richness curves, which are then time-discretized to estimate the noise generated by this manipulation. A broad spectrum of data-set parameters (including average taxon longevity and biozone duration, total number of taxa, and simulated time interval) is evaluated through sensitivity analysis. We show that the deteriorating effect of time discretization on the richness signal depends highly on such parameters, most particularly on average biozone duration and taxonomic longevity because of their direct relationship with the number of false coexistences generated by time discretization. With several worst-case but realistic parameter combinations (e.g., when relatively short-lived taxa are analyzed in a long-ranging biozone framework), the original and time-discretized richness curves can ultimately show a very weak to zero correlation, making these two time series independent. Based on these simulation results, we propose a simple algorithm allowing the back-transformation of a discrete-time taxonomic-richness data set, as customarily constructed by paleontologists, into a continuous-time data set. We show that the reconstructed richness curve obtained this way fits the original signal much more closely, even when the parameter combination of the original data set is particularly adverse to an effective time-discretized reconstruction.

Superreplication of Financial Derivatives via Convex Programming

We give a method based on convex programming to calculate the optimal superreplicating and subreplicating prices and corresponding hedging portfolios of a financial derivative in terms of other financial derivatives in a discrete-time setting. Our method produces a model that matches the superreplicating (or subreplicating) price within an arbitrary precision and is consistent with the other financial derivatives prices. Applications include robust replication in terms of call prices with various strikes and maturities of forward start options, volatility and variance swaps and derivatives, cliquet calls, barrier options, and lookback and Asian options. Numerical examples show that, in some cases, the best superreplicating and/or subreplicating prices are within 10% of the price obtained by a standard model but considerably differ from it in other cases. Our method can incorporate bid-ask spreads, interest rates and dividends, and various limitations to the diffusion model. The supplementary material is available at https://doi.org/10.1287/mnsc.2017.2786 . This paper was accepted by Gustavo Manso, finance.

Capacity of Systems with Queue-Length Dependent Service Quality

We study the information-theoretic limit of reliable information processing by a server with queue-length dependent quality of service. We define the capacity for such a system as the number of bits reliably processed per unit time, and characterize it in terms of queuing system parameters. We also characterize the distributions of the arrival and service processes that maximize and minimize the capacity of such systems in a discrete-time setting. For arrival processes with at most one arrival per time slot, we observed a minimum around the memoryless distribution. We also studied the case of multiple arrivals per time slot, and observed that burstiness in arrival has adverse effects on the system. The problem is theoretically motivated by an effort to incorporate the notion of reliability in queuing systems, and is applicable in the contexts of crowdsourcing, multimedia communication, and stream computing.

LQG Online Learning.

Optimal control theory and machine learning techniques are combined to formulate and solve in closed form an optimal control formulation of online learning from supervised examples with regularization of the updates. The connections with the classical linear quadratic gaussian (LQG) optimal control problem, of which the proposed learning paradigm is a nontrivial variation as it involves random matrices, are investigated. The obtained optimal solutions are compared with the Kalman filter estimate of the parameter vector to be learned. It is shown that the proposed algorithm is less sensitive to outliers with respect to the Kalman estimate (thanks to the presence of the regularization term), thus providing smoother estimates with respect to time. The basic formulation of the proposed online learning framework refers to a discrete-time setting with a finite learning horizon and a linear model. Various extensions are investigated, including the infinite learning horizon and, via the so-called kernel trick, the case of nonlinear models.

A time-fractional generalised advection equation from a stochastic process

A generalised advection equation with a time fractional derivative is derived from a continuous time random walk on a one-dimensional lattice, with power law distributed waiting times. We consider walks governed by a two-sided jump density and walks governed by a one-sided jump density. With the two-sided density, the particle can jump in both directions on the lattice, whereas with the one-sided density the particle cannot jump in one of these directions. The master equations describing the evolution of the probability density for the position of the particle are different for each of the jump densities. However in an advective limit both master equations limit to a common generalized advection equation with time fractional derivatives.We have also considered the stochastic processes in a discrete time setting, again arriving at different discrete time master equations for each of the jump densities. The discrete time master equations can be used to provide different numerical approximations to the solutions of the fractional generalized advection equation. The approximations allow us to compare the efficacy of the one-sided and two-sided densities.

Sensorimotor self-learning model based on operant conditioning for two-wheeled robot

Traditional control methods of two-wheeled robot are usually model-based and require the robot’s precise mathematic model which is hard to get. A sensorimotor self-learning model named SMM TWR is presented in this paper to handle these problems. The model consists of seven elements: the discrete learning time set, the sensory state set, the motion set, the sensorimotor mapping, the state orientation unit, the learning mechanism and the model’s entropy. The learning mechanism for SMM TWR is designed based on the theory of operant conditioning (OC), and it adjusts the sensorimotor mapping at every learning step. This helps the robot to choose motions. The leaning direction of the mechanism is decided by the state orientation unit. Simulation results show that with the sensorimotor model designed, the robot is endowed the abilities of self-learning and self-organizing, and it can learn the skills to keep itself balance through interacting with the environment.

First gap statistics of long random walks with bounded jumps

We obtain exact analytical results for the joint statistics of the gap and time interval between the first two maxima of long, one-dimensional, random walks (RWs) with bounded jumps. Both discrete and continuous time settings are considered. For discrete time RWs, we find that the joint distribution exhibits a concentration effect in the sense that a gap close to its maximum possible value is much more likely to be achieved by a single jump (i.e. by realizations with adjacent first two maxima) rather than by a long walk between the first two maxima. We show that a similar, albeit slightly different, concentration phenomenon also occurs for continuous time random walks (CTRWs). Our numerical simulations confirm this concentration effect.

Sub-Riemannian Curvature of Carnot Groups with Rank-Two Distributions

The notion of curvature discussed in this paper is a far-going generalization of the Riemannian sectional curvature. It was first introduced by Agrachev et al. ([2015]), and it is defined for a wide class of optimal control problems: a unified framework including geometric structures such as Riemannian, sub-Riemannian, Finsler, and sub-Finsler structures. In this work, we study the generalized sectional curvature of Carnot groups with rank-two distributions. In particular, we consider the Cartan group and Carnot groups with horizontal distribution of Goursat-type. In these Carnot groups, we characterize ample and equiregular geodesics. For Carnot groups with horizontal Goursat distribution, we show that their generalized sectional curvatures depend only on the Engel part of the distribution. This family of Carnot groups contains naturally the three-dimensional Heisenberg group, as well as the Engel group. Moreover, we also show that in the Engel and Cartan groups, there exist initial covectors for which there is an infinite discrete set of times at which the corresponding ample geodesics are not equiregular.

Correlation and relaxation times for a stochastic process with a fat-tailed steady-state distribution

We study a stochastic process defined by the interaction strength for the return to the mean and a stochastic term proportional to the magnitude of the variable. Its steady-state distribution is the Inverse Gamma distribution, whose power-law tail exponent is determined by the ratio of the interaction strength to stochasticity. Its time-dependence is characterized by a set of discrete times describing relaxation of respective cumulants to their steady-state values. We show that as the progressively lower cumulants diverge with the increase of stochasticity, so do their relaxation times. We analytically evaluate the correlation function and show that it is determined by the longest of these times, namely the inverse interaction strength, which is also the relaxation time of the mean. We also investigate relaxation of the entire distribution to the steady state and the distribution of relaxation times, which we argue to be Inverse Gaussian.

Branching processes in continuous time as models of mutations: Computational approaches and algorithms

The appearance of mutations in cancer development plays a crucial role in the disease control and its medical treatment. Motivated by the practical significance, it is of interest to model the event of occurrence of a mutant cell that will possibly lead to a path of indefinite survival. A two-type branching process model in continuous time is proposed for describing the relationship between the waiting time till the first escaping extinction mutant cell is born and the lifespan distribution of cells, which due to the applied treatment have small reproductive ratio. A numerical method and related algorithm for solving the integral equations are developed, in order to estimate the distribution of the waiting time to the escaping extinction mutant cell is born. Numerical studies demonstrate that the proposed approximation algorithm reveals the substantial difference of the results in discrete-time setting. In addition, to study the time needed for the mutant cell population to reach high levels a simulation algorithm for continuous two-type decomposable branching process is proposed. Two different computational approaches together with the theoretical studies might be applied to different kinds of cancer and their proper treatment.

Optimal Control Problems with Symmetry Breaking Cost Functions

We investigate symmetry reduction of optimal control problems for left-invariant control affine systems on Lie groups, with partial symmetry breaking cost functions. Our approach emphasizes the role of variational principles and considers a discrete-time setting as well as the standard continuous-time formulation. Specifically, we recast the optimal control problem as a constrained variational problem with a partial symmetry breaking Lagrangian and obtain the Euler--Poincaré equations from a variational principle. By using a Legendre transformation, we recover the Lie--Poisson equations obtained by Borum and Bretl [IEEE Trans. Automat. Control, 62 (2017), pp. 3209--3224] in the same context. We also discretize the variational principle in time and obtain the discrete-time Lie--Poisson equations. We illustrate the theory with some practical examples including a motion planning problem in the presence of an obstacle.

Forecasting online user activeness for behavioural targeting: the effect of data sampling

Probability models have been built to model online ad click and conversion, but few studies have examined user activeness, which is the start of any further online behaviour. Using a discrete-time setting, this study builds a three-parameter Bayesian model to forecast user activeness. Users with the same arrival count in the training period are grouped into a segment and their activeness in the test period is forecasted accordingly. The forecasting results are affected by data sparsity and history, while the first factor impacts how to sample the users and the second decides how much historic data should be used in forecasting. Using data from a major ecommerce website, we find that the model performs well when the training period is short while the users are active.

Forecasting online user activeness for behavioural targeting: the effect of data sampling

Probability models have been built to model online ad click and conversion, but few studies have examined user activeness, which is the start of any further online behaviour. Using a discrete-time setting, this study builds a three-parameter Bayesian model to forecast user activeness. Users with the same arrival count in the training period are grouped into a segment and their activeness in the test period is forecasted accordingly. The forecasting results are affected by data sparsity and history, while the first factor impacts how to sample the users and the second decides how much historic data should be used in forecasting. Using data from a major ecommerce website, we find that the model performs well when the training period is short while the users are active.

Gaussian approximations of small noise diffusions in Kullback–Leibler divergence

We study Gaussian approximations to the distribution of a diffusion. The approximations are easy to compute: they are defined by two simple ordinary differential equations for the mean and the covariance. Time correlations can also be computed via solution of a linear stochastic differential equation. We show, using the Kullback-Leibler divergence, that the approximations are accurate in the small noise regime. An analogous discrete time setting is also studied. The results provide both theoretical support for the use of Gaussian processes in the approximation of diffusions, and methodological guidance in the construction of Gaussian approximations in applications.

Small Deviations of Sums of Correlated Stationary Gaussian Sequences

We consider the small deviation probabilities (SDP) in the uniform norm for sums of stationary Gaussian sequences. For the cases of constant boundaries and boundaries tending to zero, we obtain quite general results. For the case of the boundaries tending to infinity, we focus our attention on the discrete analogues of the fractional Brownian motion (FBM). It turns out that the lower bounds for the SDP can be transferred from the well-studied FBM case to the discrete time setting under the usual assumptions that imply weak convergence, while the transfer of the corresponding upper bounds necessarily requires a deeper knowledge of the spectral structure of the underlying stationary sequence.

Non-negativity constraints in the one-dimensional discrete-time phase retrieval problem

Phase retrieval problems occur in a width range of applications in physics and engineering such as crystallography, astronomy, and laser optics. Common to all of them is the recovery of an unknown signal from the intensity of its Fourier transform. Because of the well-known ambiguousness of these problems, the determination of the original signal is generally challenging. Although there are many approaches in the literature to incorporate the assumption of non-negativity of the solution into numerical algorithms, theoretical considerations about the solvability with this constraint occur rarely. In this paper, we consider the one-dimensional discrete-time setting and investigate whether the usually applied a priori non-negativity can overcame the ambiguousness of the phase retrieval problem or not. We show that the assumed non-negativity of the solution is usually not a sufficient a priori condition to ensure uniqueness in one-dimensional phase retrieval. More precisely, using an appropriate characterization of the occurring ambiguities, we show that neither the uniqueness nor the ambiguousness are rare exceptions.