Discrete Stochastic Processes Research Articles

Tree approximation for discrete time stochastic processes: a process distance approach

Approximating stochastic processes by scenario trees is important in decision analysis. In this paper we focus on improving the approximation quality of trees by smaller, tractable trees. In particular we propose and analyze an iterative algorithm to construct improved approximations: given a stochastic process in discrete time and starting with an arbitrary, approximating tree, the algorithm improves both, the probabilities on the tree and the related path-values of the smaller tree, leading to significantly improved approximations of the initial stochastic process. The quality of the approximation is measured by the process distance (nested distance), which was introduced recently. For the important case of quadratic process distances the algorithm finds locally best approximating trees in finitely many iterations by generalizing multistage k-means clustering.

A New Class of Petri Nets for Modeling and Property Verification of Switched Stochastic Systems

Switched stochastic systems (SSS) can be used to describe hybrid systems with randomness. However, the languages to describe their discrete switching logic and stochastic dynamic processes are different, and this difference makes their design and analysis hard. This paper proposes a new Petri net model, namely stochastic-differential Petri net (S-DPN), to describe both discrete switching logic, represented by a Markov chain, and stochastic dynamic processes, represented by a set of stochastic differential equations. We then apply a model checking technique to S-DPN to check the correctness of the requirements of SSS. A temperature control system is used to demonstrate the effectiveness of our method.

Moderate Deviations for Recursive Stochastic Algorithms

We prove a moderate deviation principle for the continuous time interpolation of discrete time recursive stochastic processes. The methods of proof are somewhat different from the corresponding large deviation result, and in particular the proof of the upper bound is more complicated.

HASL: A new approach for performance evaluation and model checking from concepts to experimentation

We introduce the Hybrid Automata Stochastic Language (HASL), a new temporal logic formalism for the verification of Discrete Event Stochastic Processes (DESP). HASL employs a Linear Hybrid Automaton (LHA) to select prefixes of relevant execution paths of a DESP. LHA allows rather elaborate information to be collected on-the-fly during path selection, providing the user with powerful means to express sophisticated measures. A formula of HASL consists of an LHA and an expression Z referring to moments of path random variables. A simulation-based statistical engine is employed to obtain a confidence interval estimate of the expected value of Z. In essence, HASL provides a unifying verification framework where temporal reasoning is naturally blended with elaborate reward-based analysis. Moreover, we have implemented a tool, named Cosmos, for performing analysis of HASL formula for DESP modelled by Petri nets. Using this tool we have developed two detailed case studies: a flexible manufacturing system and a genetic oscillator.

A continuous-time diffusion limit theorem for dynamical decoupling and intrinsic decoherence

We discuss a few mathematical aspects of random dynamical decoupling, a key tool procedure in quantum information theory. In particular, we place it in the context of discrete stochastic processes, limit theorems and completely positive trace-preserving semigroups on matrix algebras. We obtain precise analytical expressions for expectation and variance of the density matrix and fidelity over time in the continuum-time limit depending on the system Lindbladian, which then lead to rough short-time estimates depending only on certain coupling strengths. We prove that dynamical decoupling does not work in the case of intrinsic (i.e., not environment-induced) decoherence, and together with the above-mentioned estimates this yields a novel method of partially identifying intrinsic decoherence.

Modelling the movement of interacting cell populations: a moment dynamics approach.

Mathematical models describing the movement of multiple interacting subpopulations are relevant to many biological and ecological processes. Standard mean-field partial differential equation descriptions of these processes suffer from the limitation that they implicitly neglect to incorporate the impact of spatial correlations and clustering. To overcome this, we derive a moment dynamics description of a discrete stochastic process which describes the spreading of distinct interacting subpopulations. In particular, we motivate our model by mimicking the geometry of two typical cell biology experiments. Comparing the performance of the moment dynamics model with a traditional mean-field model confirms that the moment dynamics approach always outperforms the traditional mean-field approach. To provide more general insight we summarise the performance of the moment dynamics model and the traditional mean-field model over a wide range of parameter regimes. These results help distinguish between those situations where spatial correlation effects are sufficiently strong, such that a moment dynamics model is required, from other situations where spatial correlation effects are sufficiently weak, such that a traditional mean-field model is adequate.

Moderate deviations for recursive stochastic algorithms

Abstract : We prove a moderate deviation principle for the continuous time interpolation of discrete time recursive stochastic processes. The methods of proof are somewhat different from the corresponding large deviation result, and in particular the proof of the upper bound is more complicated.

Prediction and Predictability in Population Biology: Noise and Chaos

To determine best predictors and quantify prediction insecurities we investigate an analytically solvable stochastic system from epidemiology in which the time dependent solution of the model, the likelihood function and Bayesian posterior among other quantities can be calculated explicitly as function of given data. We give analytical expressions for the prediction probability conditioned on best estimators of parameters versus conditioned on data only, and marginalized over the parameters, and observe that the prediction insecurity is wider in the second case of conditioning on the data only, as it should be done in empirical studies. Though the concept becomes clear in the analytical study the differences between prediction based on data directly and prediction based on best estimates of parameters is small due to the simplicity of the model which allowed the analytic treatment. In an only slightly more complex model which however already cannot be treated analytically we clearly observe the expected large differences between the two predictions. Finally, we discuss additional aspects in more extended stochastic population biological systems which do not only exhibit fixed point dynamics but also bifurcations into deterministic chaos with short term predictability and long term unpredictability as quantified by the characteristic exponent, the largest Lyapunov exponent. All is developed in a unified framework of time dependent and state discrete stochastic processes typical for many population biological systems, in ecology and in epidemiology, and can also incorporate non-exponential waiting times by simply including more intermediate classes between transitions.

Waiting time distribution for continuous stochastic systems.

The waiting time distribution (WTD) is a common tool for analyzing discrete stochastic processes in classical and quantum systems. However, there are many physical examples where the dynamics is continuous and only approximately discrete, or where it is favourable to discuss the dynamics on a discretized and a continuous level in parallel. An example is the hindered motion of particles through potential landscapes with barriers. In the present paper we propose a consistent generalization of the WTD from the discrete case to situations where the particles perform continuous barrier crossing characterized by a finite duration. To this end, we introduce a recipe to calculate the WTD from the Fokker-Planck (Smoluchowski) equation. In contrast to the closely related first passage time distribution (FPTD), which is frequently used to describe continuous processes, the WTD contains information about the direction of motion. As an application, we consider the paradigmatic example of an overdamped particle diffusing through a washboard potential. To verify the approach and to elucidate its numerical implications, we compare the WTD defined via the Smoluchowski equation with data from direct simulation of the underlying Langevin equation and find full consistency provided that the jumps in the Langevin approach are defined properly. Moreover, for sufficiently large energy barriers, the WTD defined via the Smoluchowski equation becomes consistent with that resulting from the analytical solution of a (two-state) master equation model for the short-time dynamics developed previously by us [Phys. Rev. E 86, 061135 (2012)]. Thus, our approach "interpolates" between these two types of stochastic motion. We illustrate our approach for both symmetric systems and systems under constant force.

Simulating non-Markovian stochastic processes.

We present a simple and general framework to simulate statistically correct realizations of a system of non-Markovian discrete stochastic processes. We give the exact analytical solution and a practical and efficient algorithm like the Gillespie algorithm for Markovian processes, with the difference being that now the occurrence rates of the events depend on the time elapsed since the event last took place. We use our non-Markovian generalized Gillespie stochastic simulation methodology to investigate the effects of nonexponential interevent time distributions in the susceptible-infected-susceptible model of epidemic spreading. Strikingly, our results unveil the drastic effects that very subtle differences in the modeling of non-Markovian processes have on the global behavior of complex systems, with important implications for their understanding and prediction. We also assess our generalized Gillespie algorithm on a system of biochemical reactions with time delays. As compared to other existing methods, we find that the generalized Gillespie algorithm is the most general because it can be implemented very easily in cases (such as for delays coupled to the evolution of the system) in which other algorithms do not work or need adapted versions that are less efficient in computational terms.

Probability of Error Derivatives for Binary and Chaos-Based CDMA Systems in Wide-Band Channels

This paper presents detailed mathematical models of DS-CDMA systems based on binary and chaotic spreading sequences. These systems have been well investigated in the case of flat fading and noise presence in the channel. However, a comprehensive analysis of the systems operating in a wide-band channel does not exist. Therefore, in this paper, the systems are analyzed in detail, and expressions for the probability of error are derived in closed form assuming a wide-band transmission channel. Using both the signal representation in pure discrete time domain and the theory of discrete time stochastic processes, which have not been practically used before, the probability of error expressions are derived in closed form for single- and multi-user binary and chaos based DS-CDMA systems. The improvements in the probability of bit error due to multipath channel nature are quantified in dependence of the random delay and the number of users in the system. The wide-band channel is based on the Saleh-Valenzuela model proposed for the modern wireless networks.

On limit periodicity of discrete time stochastic processes

We consider a discrete time dynamic system described by a difference equation with periodic coefficients and with additive stochastic noise. We investigate the possibility of the periodicity of the solution. In particular, we established sufficient conditions for convergence of the solution in mean square or almost surely to some stochastic periodic process.

A fast derivation of Karhunen-Loeve transform kernel for first-order autoregressive discrete process

Karhunen-Loève Transform (KLT), also called principal component analysis (PCA) or factor analysis, based signal processing methods have been successfully used in applications spanning from eigenfiltering to recommending systems. KLT is a signal dependent transform and comprised of three major steps where each has its own computational requirement. Namely, statistical measurement of random data is performed to populate its covariance matrix. Then, eigenvectors (eigenmatrix) and eigenvalues are calculated for the given covariance matrix. Last, incoming random data vector is mapped onto the eigenspace (subspace) by using the calculated eigenmatrix. The recently developed method by Torun and Akansu offers an efficient derivation of the explicit eigenmatrix for the covariance matrix of first-order autoregressive, AR(1), discrete stochastic process. It is the second step of the eigenanalysis implementation as summarized in the paper. Its computational complexity is investigated and compared with the currently used techniques. It is shown that the new method significantly outperforms the others, in particular, for very large matrix sizes that are common in big data applications.

Entropy rate of message sources driven by quantum walks

The amount of information generated by a discrete time stochastic processes in a single step can be quantified by the entropy rate. We investigate the differences between two discrete time walk models, the discrete time quantum walk and the classical random walk in terms of entropy rate. We develop analytical methods to calculate and approximate it. This allows us to draw conclusions about the differences between classical stochastic and quantum processes in terms of the classical information theory.

Theoretical analysis and simulation of a code division multiple access system (CDMA) for secure signal transmission in wideband channels

Theoretical analysis and simulation of a code division multiple access system (CDMA) for secure signal transmission in wideband channels

Statistical analysis technique on Ad Hoc network topology dynamic characteristics: Markov stochastic process

On the basis of analysis on the scene files of mobility models in Ad Hoc network, the paper presents a network topology snapshots capturing method to obtain the Ad Hoc network topology architecture at any moment. Through analyzing on the Ad Hoc network topology snapshots, some dynamic characteristic parameters of Ad Hoc network, such as the number of network topology in steady state or unsteady state appearing during a certain time, as well as the durative time of network topology in steady state or unsteady state, could be obtained statistically. Furthermore, the probability of the network topology invariability and variability event could be predicated by adopting the discrete time and continuous time Markov stochastic process theory. The simulation result shows that the statistical analysis technique on Ad Hoc network topology dynamic characteristic not only is effective, but also has the general attribute, which could be used in the statistical analysis technique on Ad Hoc network topology dynamic characteristic under any mobility models.

A note on values for Markovian coalition processes

The Shapley value is defined as the average marginal contribution of a player, taken over all possible ways to form the grand coalition \(N\) when one starts from the empty coalition and adds players one by one. The authors have proposed in a previous paper an allocation scheme for a general model of coalition formation where the evolution of the coalition of active players is ruled by a Markov chain, and need not finish at the grand coalition. The aim of this note is to develop some explanations in the general context of time discrete stochastic processes, exhibit new properties of the model, correct some inaccuracies in the original paper, and give a new version of the axiomatization.

Assessment of emergency response policy based on Markov process

Major accidents not only endanger the health and safety of the population, but also bring bad influence to the environment around. In order to mitigate the adverse effects, emergency response policy (ERP) and corresponding protection action should be established. To maximise its effectiveness, ERP should be evaluated and optimised, while the most important criteria is minimising the health consequence of the accident. A discrete state stochastic Markov process was used to simulate the movement of the evacuees in this paper. Solution of the Markov process provided the expected distribution of the evacuees in the area as a function of time. Then, according to the way how extreme phenomena impact individual and the dose-response relationship, the people’s health effects were calculated, so that the accident’s health consequence was determined. Finally, different emergency response policies were evaluated with corresponding health consequence, so that the emergency policy can be optimised.

Nonparametric multivariate breakpoint detection for the means, variances, and covariances of a discrete time stochastic process

We introduce a nonparametric breakpoint detection method for the means and covariances of a multivariate discrete time stochastic process. Breakpoints are defined as left or right endpoints of maximal intervals of local time homogeneity for the means and covariances. The breakpoint detection method is an adaptive algorithm that estimates the last maximal interval of homogeneity. Applied recursively, it allows us to find an arbitrary number of breakpoints. We then study a second breakpoint detection algorithm that makes use of a sliding window. The quality of both methods is analysed. For the adaptive algorithm, we provide the quality of the estimation of the one-step-ahead means and covariance matrix as well as upper bounds on the type I and type II errors when applying the procedure to a change-point model. Regarding the second method, the probability of correctly detecting the breakpoint of a change-point model is bounded from below. Numerical simulations assess the performance of both methods using simulated data.

Distributed delays in a hybrid model of tumor-Immune system interplay

A tumor is kinetically characterized by the presence of multiple spatio-temporal scales in which its cells interplay with, for instance, endothelial cells or Immune system effectors, exchanging various chemical signals. By its nature, tumor growth is an ideal object of hybrid modeling where discrete stochastic processes model low-numbers entities, and mean-field equations model abundant chemical signals. Thus, we follow this approach to model tumor cells, effector cells and Interleukin-2, in order to capture the Immune surveillance effect. We here present a hybrid model with a generic delay kernel accounting that, due to many complex phenomena such as chemical transportation and cellular differentiation, the tumor-induced recruitment of effectors exhibits a lag period. This model is a Stochastic Hybrid Automata and its semantics is a Piecewise Deterministic Markov process where a two-dimensional stochastic process is interlinked to a multi-dimensional mean-field system. We instantiate the model with two well-known weak and strong delay kernels and perform simulations by using an algorithm to generate trajectories of this process. Via simulations and parametric sensitivity analysis techniques we (i) relate tumor mass growth with the two kernels, we (ii) measure the strength of the Immune surveillance in terms of probability distribution of the eradication times, and (iii) we prove, in the oscillatory regime, the existence of a stochastic bifurcation resulting in delay-induced tumor eradication.