Stationary State Distribution Research Articles

Stochastic equilibria control and chaos suppression for 3D systems via stochastic sensitivity synthesis

In this paper, we consider a nonlinear controlled system forced by stochastic disturbances. The problem addressed is to design a feedback regulator that can stabilize an equilibrium of the closed-loop system and, around this equilibrium, to synthesize a required dispersion of random states of the corresponding system. We use a stochastic sensitivity function technique to approximate the stationary probabilistic distribution of these random states. We also develop a new method for stabilization based on the stochastic sensitivity synthesis. A constructive description of the attainability set of the stochastic sensitivity matrices for a 3D system is given. The effectiveness of the new approach is demonstrated by the 3D stochastic Chen system. It is shown that the new regulator provides a low level of sensitivity and can suppress both regular and chaotic oscillations.

A Tandem BMAP/G/1 → •/M/N/0 Queue with Group Occupation of Servers at the Second Station

We consider a two‐stage tandem queue with single‐server first station and multiserver second station. Customers arrive to Station 1 according to a batch Markovian arrival process (BMAP). A batch may consist of heterogeneous customers. The type of a customer is determined upon completion of a service at Station 1. The customer′s type is classified based on the number of servers required to process the request of the customer at Station 2. If the required number of servers is not available, the customer may leave the system forever or block Station 1 by waiting for the required number of servers. We determine the stationary distribution of the system states at embedded epochs and derive the Laplace‐Stieltjes transform of the sojourn time distribution. Some key performance measures are calculated, and illustrative numerical results are presented.

Call center operation model as a MAP/PH/N/R−N system with impatient customers

We analyze a multiserver queueing system with a finite buffer and impatient customers. The arrival customer flow is assumed to be Markovian. Service times of each server are phase-type distributed. If all servers are busy and a new arrival occurs, it enters the buffer with a probability depending on the total number of customers in the system and waits for service, or leaves the system with the complementary probability. A waiting customer may become impatient and abandon the system. We give an algorithm for finding the stationary distribution of system states and derive formulas for basic performance characteristics. We find Laplace-Stieltjes transforms for sojourn and waiting times. Numeric examples are given.

On conditions for a product-form stationary probability distribution of states of a multimode service network

We consider open and closed queueing networks with Markovian routing and symmetric service policies. Single-server nodes may operate in several modes. In each node, the amount of work required for servicing an arrival or for switching the mode is distributed arbitrarily. The performance rate for each of these operations depends on the residual amount of work. For open networks, the arrival flow is Poissonian. We establish conditions for a productform stationary distribution of states of the piecewise continuous process that describes the network behavior.

Invariance of the stationary state distribution for mass service networks with multi-regime strategies, different demands, and a “generalized processor sharing” discipline

Mass service networks with multiregime strategies and several types of demands are studied. Single-line nodes can operate in several regimes corresponding to different degrees of performance. The time of existence of each regime has an exponential distribution. The discipline of the demand servicing by the device is the “generalized processor sharing” (GPS) with random channel choice in the node. The amount of work on servicing a demand is a random quantity with an arbitrary distribution function. The invariance of the stationary probability distribution of network states with respect to the functional form of the distribution of the amount of work required for the demand servicing is established.

Optimal Limit-Cycle Control recast as Bayesian Inference

AbstractWe introduce an algorithm that generates an optimal controller for stochastic nonlinear problems with a periodic solution, e.g. locomotion. Uniquely, the quantity we approximate is neither the Value nor Policy functions, but rather the stationary state-distribution of the optimally-controlled process. We recast the control problem as Bayesian inference over a graphical model with a ring topology. The posterior approximates the controlled stationary distribution with local gaussians along the optimal limit-cycle. Linear-feedback gains and open-loop controls are extracted from the covariances and the means, respectively. Complexity scales linearly or quadratically with the state dimension, depending on the dynamics approximation. We demonstrate our algorithm on a toy 2-dimensional problem and then on a challenging 23-dimensional simulated walking robot.

Characterization of the stationary distribution of the batch-transfer networks as a product of biased geometric distributions

Consideration was given to the class of exponential queuing networks with batch arrivals and batch servicing. The messages arriving to the network nodes form random-size batches of customers associated with these nodes. Upon completion of servicing a current batch, it discharges the network and sends a message, if any, to the rest of the nodes according to some routing matrix. The necessary and sufficient conditions under which the stationary distribution of the network states is multiplicative were established in terms of the isolated nodes placed in a dummy random medium. These conditions underlie characterization of the stationary distribution in the form of biased geometric distributions.

Nonuniform networks with restrictions on the residence time in maintenance mode

An open nonuniform network of queuing with the simplest incoming flow, exponential service at the nodes, and Markov routing was studied. In each of the nodes there is only one device, which can operate in several modes. For isolated nodes in a fictitious medium, conditions of reversibility, under which the stationary probability distribution of states of the network has a multiplicative form, were obtained. It is proved that customers flows coming out of the network are independent and Poisson.

Derivatives of Logarithmic Stationary Distributions for Policy Gradient Reinforcement Learning

Most conventional policy gradient reinforcement learning (PGRL) algorithms neglect (or do not explicitly make use of) a term in the average reward gradient with respect to the policy parameter. That term involves the derivative of the stationary state distribution that corresponds to the sensitivity of its distribution to changes in the policy parameter. Although the bias introduced by this omission can be reduced by setting the forgetting rate gamma for the value functions close to 1, these algorithms do not permit gamma to be set exactly at gamma = 1. In this article, we propose a method for estimating the log stationary state distribution derivative (LSD) as a useful form of the derivative of the stationary state distribution through backward Markov chain formulation and a temporal difference learning framework. A new policy gradient (PG) framework with an LSD is also proposed, in which the average reward gradient can be estimated by setting gamma = 0, so it becomes unnecessary to learn the value functions. We also test the performance of the proposed algorithms using simple benchmark tasks and show that these can improve the performances of existing PG methods.

Life table response experiment analysis of the stochastic growth rate

Summary 1. Life table response experiment (LTRE) analyses decompose treatment effects on a dependent variable (usually, but not necessarily, population growth rate) into contributions from differences in the parameters that determine that variable. 2. Fixed, random and regression LTRE designs have been applied to plant populations in many contexts. These designs all make use of the derivative of the dependent variable with respect to the parameters, and describe differences as sums of linear approximations. 3. Here, I extend LTRE methods to analyse treatment effects on the stochastic growth rate log λs. The problem is challenging because a stochastic model contains two layers of dynamics: the stochastic dynamics of the environment and the response of the vital rates to the state of the environment. I consider the widely used case where the environment is described by a Markov chain. 4. As the parameters describing the environmental Markov chain do not appear explicitly in the calculation of log λs, derivatives cannot be calculated. The solution presented here combines derivatives for the vital rates with an alternative (and older) approach, due to Kitagawa and Keyfitz, that calculates contributions in a way analogous to the calculation of main effects in statistical models. 5. The resulting LTRE analysis decomposes log λs into contributions from differences in: (i) the stationary distribution of environmental states, (ii) the autocorrelation pattern of the environment, and (iii) the stage‐specific vital rate responses within each environmental state. 6. As an example, the methods are applied to a stage‐classified model of the prairie plant Lomatium bradshawii in a stochastic fire environment. 7. Synthesis. The stochastic growth rate is an important parameter describing the effects of environmental fluctuations on population viability. Like any growth rate, it responds to differences in environmental factors. Without a decomposition analysis there is no way to attribute differences in the stochastic growth rate to particular parts of the life cycle or particular aspects of the stochastic environment. The methods presented here provide such an analysis, extending the LTRE analyses already available for deterministic environments.

SENSITIVITY TO SERVICE TIMES IN INFINITE-SERVER SYSTEMS

The stationary distribution of the number of customers in the infinite-server system with non- Poissonian arrivals is dependent on the form of the service time distribution. In particular, when the interarrival time is a hyperexponential random variable that is more variable than an exponential random variable, the stationary state distribution becomes stochastically less variable, as the service time becomes more variable. On the other hand, when the interarrival time is an Erlang random variable that is less variable than an exponential random variable, the more variable service time yields the more variable stationary state distribution.

The BMAP/ PH/ N retrial queueing system operating in Markovian random environment

We consider the BMAP/ PH/ N retrial queueing system operating in a finite state space Markovian random environment. The stationary distribution of the system states is computed. The main performance measures of the system are derived. Presented numerical examples illustrate a poor quality of the approximation of the main performance measures of the system by means of the simpler queueing models. An effect of smoothing the traffic and an impact of intensity of retrials are shown.

Evolutionary dynamics on graphs: Efficient method for weak selection

Investigating the evolutionary dynamics of game theoretical interactions in populations where individuals are arranged on a graph can be challenging in terms of computation time. Here, we propose an efficient method to study any type of game on arbitrary graph structures for weak selection. In this limit, evolutionary game dynamics represents a first-order correction to neutral evolution. Spatial correlations can be empirically determined under neutral evolution and provide the basis for formulating the game dynamics as a discrete Markov process by incorporating a detailed description of the microscopic dynamics based on the neutral correlations. This framework is then applied to one of the most intriguing questions in evolutionary biology: the evolution of cooperation. We demonstrate that the degree heterogeneity of a graph impedes cooperation and that the success of tit for tat depends not only on the number of rounds but also on the degree of the graph. Moreover, considering the mutation-selection equilibrium shows that the symmetry of the stationary distribution of states under weak selection is skewed in favor of defectors for larger selection strengths. In particular, degree heterogeneity--a prominent feature of scale-free networks--generally results in a more pronounced increase in the critical benefit-to-cost ratio required for evolution to favor cooperation as compared to regular graphs. This conclusion is corroborated by an analysis of the effects of population structures on the fixation probabilities of strategies in general 2 x 2 games for different types of graphs. Computer simulations confirm the predictive power of our method and illustrate the improved accuracy as compared to previous studies.

On control of stochastic sensitivity

Consideration was given to generation of the desired probabilistic characteristics of the attractors of the nonlinear stochastic control systems. In the case of small random perturbations, the scatter of trajectories in the neighborhood of stable equilibrium is describable in terms of the stochastic sensitivity matrix. Problem was posed of generating the desired stochastic sensitivity matrix. The corresponding notions of reachability and full controllability were examined. A criterion for full controllability was obtained, and a constructive description of the reachability set was given. Efficiency of the proposed constructions was demonstrated by way of example of control of the stationary distribution of the random states of the perturbed stochastic Van der Pol's equation.

Time-series analysis of vibrational nuclear wave-packet dynamics inD2+

We discuss the extent to which measured time-dependent fragment kinetic energy release (KER) spectra and calculated nuclear probability densities can reveal (1) the transition frequencies between stationary vibrational states, (2) the nodal structure of stationary vibrational states, (3) the ground-state adiabatic electronic potential curve of the molecular ion, and (4) the progression of decoherence induced by random interactions with the environment. We illustrate our discussion with numerical simulations for the time-dependent nuclear motion of vibrational wave packets in the $\text{D}_{2}{}^{+}$ molecular ion caused by the ionization of its neutral ${\text{D}}_{2}$ parent molecule with an intense pump laser pulse. Based on a harmonic time-series analysis, we suggest a general scheme for the full reconstruction, up to an overall phase factor, of the initial wave packets based on measured KER spectra. We apply this scheme in a numerical simulation for vibrational wave packets in $\text{D}_{2}{}^{+}$ molecular ions and show how this reconstruction allows the clear distinction between commonly assumed stationary vibrational state distributions of the molecular ion following the ionization of ${\text{D}}_{2}$.

Piecewise-linear networks with multimode service strategies

Consideration is given to open and closed queueing networks with Markov routing and symmetric service procedures. Single-line nodes can operate in several modes; the switching time from one mode to another is of exponential distribution. The switching occurs only to nearby modes. The quantity of work on service of the arriving customer is a random quantity with an arbitrary distribution function. For open networks, the input flow is the simplest. The invariancy of the stationary distribution of network states with respect to the functional form of distributions of quantities of works at fixed first moments is established.

Adaptive topography of fluctuating selection in a Mendelian population

An adaptive topography is derived for a large randomly mating diploid population under weak density-independent selection in a fluctuating environment. Assuming a stationary distribution of environmental states with no temporal autocorrelation, a diffusion approximation for population size and allele frequency, p, reveals that the expected change in p involves the gradient with respect to p of the stochastic intrinsic rate of increase (the density-independent long-run growth rate), r = r - sigma 2 e/2, where r is the mean Malthusian fitness in the average environment and is the environmental variance in population growth rate. The expected relative fitness of a genotype is its Malthusian fitness in the average environment minus the covariance of its fitness with population growth rate. The influence of fitness correlation between genotypes is illustrated by an analysis of the Haldane-Jayakar model of fluctuating selection on a single diallelic locus, and on two loci with additive effects on a quantitative character.

Erlang loss queueing system with batch arrivals operating in a random environment

We consider the BMAP / PH / N / 0 queueing system operating in a finite state space Markovian random environment. Disciplines of partial admission, complete rejection and complete admission are analyzed. The stationary distribution of the system states is calculated. The loss probability and other main performance measures of the system are derived. The Laplace–Stieltjes transform of the sojourn time distribution of accepted customers is obtained. Illustrative numerical examples are presented. They show effect of an admission strategy, a correlation in an arrival process, a variation of a service process. Poor quality of the loss probability approximation by means of more simple models utilization is illustrated.

Mathematical properties of a kinetic transport model for carriers and phonons in semiconductors

We present studies on the mathematical properties of a multigroup formulation of the Bloch–Boltzmann–Peierls equations. The considered model equations are based on a general carrier dispersion law and contain the full quantum statistics of both the carriers and the phonons. Moreover, the transport model allows the investigation of particle distributions with arbitrary anisotropy with respect to the main direction. We prove the boundedness of the solution according to the Pauli principle and study the conservational properties of the multigroup equations. In addition, the existence of a Lyapounov functional to the proposed model equations is proved and expressions for the equilibrium solution are given. Numerical results are presented for the stationary state distributions of a coupled system of electrons and longitudinal optical phonons in GaAs.

Modeling the sequential electric field assisted diffusion into glass of two ion species

The sequential electric field assisted diffusion of two ion species into glass is modeled in this work. Using standard assumptions, two coupled nonlinear drift-diffusion equations are derived for the ion concentration distributions. These equations are solved numerically for the case where the two ion species each have a lower mobility than the indigenous sodium ions in the glass. It is shown that stationary state distributions form if the final species injected into the glass is the less mobile of the two. Analytical expressions are derived for the stationary state distributions. A possible application in integrated optics involving Ag+ and Cu+ ion diffusion is considered.