Stationary State Distribution Research Articles

Characterizing the nonmonotonic behavior of mutual information along biochemical reaction cascades.

Cells sense environmental signals and transmit information intracellularly through changes in the abundance of molecular components. Such molecular abundances can be measured in single cells and exhibit significant heterogeneity in clonal populations even in identical environments. Experimentally observed joint probability distributions can then be used to quantify the covariability and mutual information between molecular abundances along signaling cascades. However, because stationary state abundances along stochastic biochemical reaction cascades are not conditionally independent, their mutual information is not constrained by the data-processing inequality. Here, we report the conditions under which the mutual information between stationary state abundances increases along a cascade of biochemical reactions. This nonmonotonic behavior can be intuitively understood in terms of noise propagation and time-averaging stochastic fluctuations that are short-lived compared to an extrinsic signal. Our results reemphasize that mutual information measurements of stationary state distributions of cellular components may be of limited utility for characterizing cellular signaling processes because they do not measure information transfer.

Markov chain models of emitter activations in single molecule localization microscopy.

A well-reasoned model of data movie in single molecule localization microscopy (SMLM) is desired. A model of data movie can be decomposed into a model of emitter activation process and a model of data frame. In this paper, we focus on Markov chain modeling and analyzing of emitter activation process for both cycled and continuous illuminations. First, a two-phase Markov chain is proposed to model the activation process for a pair of conjugated activator and emitter with cycled illumination. By converting the frame-based Markov chain into several cycle-based Markov chains, the stationary state distribution in the photoactivatable period is derived. Further obtained are several formulas that capture the characterization of the two-phase Markov chain. Second, the Markov chain and analytical result are extended to the continuous illumination where an emitter is excited continuously in all frames. Finally, incorporating the model of emitter activation process with our previous model of data frame, the model of data movie for both cycled and continuous illuminations in 3D and 2D imaging are simulated by custom codes. It is shown that the model can synthesize data movies well and the analytical formulas predict the simulation results accurately. The models provide a means to be broadly utilized in generating well-reasoned data movies for training of neural networks and evaluation of localization algorithms.

Dual-channel call center model with exchange of requests and repeat calls

Computer simulation of stationary distribution of states in a call center for a two-channel RQ system with exchange of requests is presented. Such systems are becoming increasingly relevant due to the parallel use of both human dispatchers and voice intelligent bots for servicing calls. The simulation is based on solving the system of stationary Chapman-Kolmogorov equations for the Markov process describing the RQ-system. The system of equations proposed in the work differs from previous models by the presence of an exchange of requests between service channels in accordance with customer preferences. It takes into account the possibility of random repeated calls within a given average time. Service time in channels and delay time of requests in orbits have exponential distribution laws. Requests in orbit have the property of impatience, i.e. leave the system after some random time. To find the stationary distribution of states in orbits, the iterative numerical Gauss-Seidel method is used, which ensures fast convergence of calculations. Each channel has its own orbit of requests. The accuracy of the solution is controlled by increasing the maximum number of requests in orbit until the result stabilizes. The model demonstrates the sensitivity of system performance to the asymmetry of customer preferences when changing channels. Numerical simulation was carried out for the call center of the housing management company "StroyTekhnika" in the city of Voronezh. Application flows and repeat call parameters were calculated based on data from the company website and social network analysis. Accounting for repeat calls reduces system throughput compared to the option of completely impatient customers who do not use repeat calls. The case of completely impatient customers describes the limit state of a service system. At the same time, redistributing calls in favor of a more productive channel improves the overall performance of the system. The results obtained show the feasibility of using high-performance multi-channel voice bots while simultaneously stimulating a shift in customer preferences in favor of intelligent automata.

Optimal Hysteresis Control via a Queuing System with Two Heterogeneous Energy-Consuming Servers

A queuing system having two different servers is under study. Demands enter the system according to a Markov arrival process. Service times have phase-type distribution. Service of demands is possible only if the fixed number of energy units, probably different for two servers, is available in the system at the potential service beginning moment. Energy units arrive in the system also according to a Markov arrival process and are stored in a stock (battery) of a finite capacity. Leakage of energy units from the stock can occur. Demands waiting in the infinite buffer are impatient and can leave the buffer after an exponentially distributed waiting time. One server is the main one and permanently provides service when the buffer is not empty and the required number of energy units is available. The second server is the assistant server and is switched on or off depending on the availability of energy units and queue length according to the hysteresis strategy defined by two thresholds. The assistant server is switched on when the queue length is not less than the greater threshold and is switched off when the queue length becomes smaller than the smaller threshold. The use of the assistant server has to be paid. Thus, the problem of the optimal selection of the thresholds defining the control strategy naturally arises. To solve this problem, the study of the behavior of the system under any fixed values of the parameters of the control strategy is necessary. Such a study is given in this paper. Numerical results are presented. They illustrate the feasibility of computer realization of the developed algorithms for computation of the stationary distribution of the system states and the main key performance indicators as well as the result of solving one of the possible optimization tasks.

Optimal pricing strategy in an unreliable M/G/1 retrial queue with Bernoulli preventive maintenance

This paper studies an unreliable M/G/1 retrial queue with Bernoulli preventive maintenance, in which the server may break down while providing service to customers. Before the start of each service, the server is subjected to preventive maintenance with a certain probability, which extends the lifetime of the server. If arriving customers find the server unavailable, they decide to balk or enter the orbit based on the linear reward-cost structure. First, the stationary distribution of system states and some important performance measures are derived. Then, we investigate the equilibrium joining strategies of customers in both cooperative and non-cooperative cases, respectively. Next, the optimal pricing strategy is considered from the perspective of the social planner. Finally, numerical examples are provided to illustrate the influence of system parameters on customer equilibrium joining probabilities and the optimal pricing strategy.

SIS Epidemic Propagation Under Strategic Non-Myopic Protection: A Dynamic Population Game Approach

We consider a dynamic game setting in which a large population of strategic individuals decides whether to adopt protective measures to protect themselves against an infectious disease, specifically the susceptible-infected-susceptible (SIS) epidemic. Protection is costly and partially effective, and adopting protection reduces the probability of becoming infected for susceptible individuals and the probability of transmitting the infection for infected individuals. In a departure from most prior works that assume the decision-makers to be myopic, we model individuals who choose their actions to maximize the infinite horizon discounted expected reward. We define the notion of best response and stationary Nash equilibrium in this class of games, and completely characterize the equilibrium policy and stationary state distribution for different parameter regimes. Numerical results illustrate the evolution and convergence of the infected proportion and the policy of protection adoption to the equilibrium.

Optimal Steady-State Disturbance Compensation for Constrained Linear Systems: The Gaussian Noise Case

We consider the problem of designing a disturbance compensator for a discrete time linear system, so as to optimize a performance index while satisfying probabilistic state and input constraints in steady-state conditions. The problem is formulated as a chance-constrained program that depends on the compensator parameters through the state and input stationary distributions. In this article, we focus on the Gaussian noise case and provide an analytic expression of the stationary state distribution as a function of the compensator parameters. This expression can be used in the chance-constrained program, which can then be tackled via the scenario approach. Some useful extensions of the setup are also discussed to further broaden the applicability of the approach. Performance of the proposed design methodology is shown on a building energy management problem where cyclostationary disturbances are compensated, thus providing a stochastic periodic control solution.

Queuing Models for Analyzing the Steady-State Distribution of Stochastic Inventory Systems with Random Lead Time and Impatient Customers

In material management, the inventory systems may have good management aspects in terms of materials; however, this negatively affects the relationship between the facility and customers. In classical inventory models, arriving demands are satisfied immediately if there is enough on-hand inventory. Traditional inventory models consider optimization problems and find the optimal policy of decision variables without computing the stationary distribution of the inventory states for random demand. Hence, a detailed analysis of inventory management systems requires a joint distribution of system stock levels and the number of requests to be investigated thoroughly. This research provides a new stochastic mathematical model for inventory systems with lead times and impatient customers under deterministic and uniform order sizes. The proposed model identifies the performance measures in a stochastic environment, analyzing the properties of the inventory system with stochastic and probabilistic parameters, and finally, validating the model’s accuracy. To analyze the system, balance equations were derived from a mathematical characterization of the underlying queuing model dependent on the Markov chain formalism. The precise performance was achieved by examining the graphical representation of the service process in a steady-state as a function of both arrival distribution and the customer patience coefficient, while it was challenging to derive an optimal curve fit in a three-dimensional space that features two input variables and a single output variable.

Path integral description of semiflexible active Brownian polymers.

Semiflexible polymers comprised of active Brownian particles (ABPOs) exhibit intriguing activity-driven conformational and dynamical features. Analytically, the generic properties of ABPOs can be obtained in a mean-field description applying the Gaussian semiflexible polymer model. In this article, we derive a path integral representation of the stationary-state distribution function of such ABPOs, based on the stationary-state distribution function of the normal mode amplitudes following from the Langevin equation of motion. The path integral includes characteristic semiflexible polymer contributions from entropy and bending energy, with activity dependent coefficients, and, in addition, activity-induced torsional and higher order correlations along the polymer contour. Focusing on a semiflexible polymer approximation, we determine various properties such as the tangent-vector correlation function, effective persistence length, and the mean-square end-to-end distance. The latter reflects the characteristic features of ABPOs, and good quantitative agreement is obtained with the full solution for larger activities, specifically for flexible polymers. Moreover, the approximation indicates the relevance of torsional and higher order contour correlations for the ABPO conformations. In general, the ABPO path integral illustrates how colored noise (active fluctuations) affects semiflexible polymer conformations in comparison to white noise thermal fluctuations.

Multi-queue system with finite shared buffer under batch Poisson and/or interrupted batch Poisson arrivals: a computational analysis

This paper studies a model of a multi-queue system with a finite common buffer shared by queues with batch arrivals. The study is performed for a few buffer-sharing policies. For batch Poisson arrivals with an arbitrary batch size distribution, the analysis benefits from the fact that the stationary state distribution has a product form. For the special case of geometrically distributed batches, closed form solutions for normalisation constant are obtained. They lead to efficient computational procedures. For bursty interrupted batch Poisson arrivals, it is shown that an exact numerical analytic analysis is only possible for very small models. For real-life sized systems, an approximation method has been developed.

Optimization of road design via the use of a queueing model with transit and local users and processor sharing discipline

A problem of the optimal choice of the number of lanes and admission control in the road that is shared by the local and transit vehicles is considered. The operation of a road is modelled by the type queue with two types of customers. The service of type-2 (local) customers is mandatory. Type-1 (transit) customers can use alternative routes and, therefore, abandon or interrupt service in the considered system. All admitted customers are serviced simultaneously. A customer receives a constant part of a server bandwidth if the number of customers does not exceed a predefined in advance threshold and the reduced part, otherwise. The stability condition is derive and the stationary distribution of the system states is computed. A potential application for optimizing the number of lanes in the road is numerically illustrated.

Quasipotentials in the nonequilibrium stationary states or a method to get explicit solutions of Hamilton–Jacobi equations

We assume that a system at a mesoscopic scale is described by a field ϕ(x, t) that evolves by a Langevin equation with a white noise whose intensity is controlled by a parameter . The system stationary state distribution in the small noise limit (Ω → ∞) is of the form P st [ϕ] ≃ exp(−ΩV 0[ϕ]), where V 0[ϕ] is called the quasipotential. V 0 is the unknown of a Hamilton–Jacobi equation. Therefore, V 0 can be written as an action computed along a path that is the solution from Hamilton’s equation that typically cannot be solved explicitly. This paper presents a theoretical scheme that builds a suitable canonical transformation that permits us to do such integration by deforming the original path into a straight line and including some weights along with it. We get the functional form of such weights through conditions on the existence and structure of the canonical transformation. We apply the scheme to get the quasipotential algebraically for several one-dimensional nonequilibrium models as the diffusive and reaction–diffusion systems.

Brownian motion under intermittent harmonic potentials

We study the effects of an intermittent harmonic potential of strength μ = μ 0 ν—that switches on and off stochastically at a constant rate γ, on an overdamped Brownian particle with damping coefficient ν. This can be thought of as a realistic model for realisation of stochastic resetting. We show that this dynamics admits a stationary solution in all parameter regimes and compute the full time dependent variance for the position distribution and find the characteristic relaxation time. We find the exact non-equilibrium stationary state distributions in the limits—(i) γ ≪ μ 0 which shows a non-trivial distribution, in addition as μ 0 → ∞, we get back the result for resetting with refractory period; (ii) γ ≫ μ 0 where the particle relaxes to a Boltzmann distribution of an Ornstein–Uhlenbeck process with half the strength of the original potential and (iii) intermediate γ = 2nμ 0 for n = 1, 2. The mean first passage time (MFPT) to find a target exhibits an optimisation with the switching rate, however unlike instantaneous resetting the MFPT does not diverge but reaches a stationary value at large rates. MFPT also shows similar behavior with respect to the potential strength. Our results can be verified in experiments on colloids using optical tweezers.

The Evolution of Networks and Local Public Good Provision: A Potential Approach

In this paper, we propose a game in which each player decides with whom to establish a costly connection and how much local public good is provided when benefits are shared among neighbors. We show that, when agents are homogeneous, Nash equilibrium networks are nested split graphs. Additionally, we show that the game is a potential game, even when we introduce heterogeneity along several dimensions. Using this result, we introduce stochastic best reply dynamics and show that this admits a unique and stationary steady state distribution expressed in terms of the potential function of the game. Hence, even if the set of Nash equilibria is potentially very large, the long run predictions are sharp.

Direct and indirect reinforcement learning

Reinforcement learning (RL) algorithms have been successfully applied to a range of challenging sequential decision making and control tasks. In this paper, we classify RL into direct and indirect RL according to how they seek the optimal policy of the Markov decision process problem. The former solves the optimal policy by directly maximizing an objective function using gradient descent methods, in which the objective function is usually the expectation of accumulative future rewards. The latter indirectly finds the optimal policy by solving the Bellman equation, which is the sufficient and necessary condition from Bellman's principle of optimality. We study policy gradient forms of direct and indirect RL and show that both of them can derive the actor-critic architecture and can be unified into a policy gradient with the approximate value function and the stationary state distribution, revealing the equivalence of direct and indirect RL. We employ a Gridworld task to verify the influence of different forms of policy gradient, suggesting their differences and relationships experimentally. Finally, we classify current mainstream RL algorithms using the direct and indirect taxonomy, together with other ones including value-based and policy-based, model-based and model-free.

Analysis of Single-Server Multi-Class Queue with Unreliable Service, Batch Correlated Arrivals, Customers Impatience, and Dynamical Change of Priorities

A single-server non-pre-emptive priority queueing system of a finite capacity with many types of customers is analyzed. Inter-arrival times can be correlated and batch arrivals are allowed. Possible unreliability of the server, implying the loss of a customer or the necessity of its service from the early beginning or some phase of the service, is taken into account. Initial priorities provided to various types of customers at the arrival moment can be varied (increased or decreased) after the random amount of time during the customer stay in the buffer. Such a type of queues arises in the modeling operation of various emergency care systems, information, and perishable goods delivering systems, etc. The stationary behavior of the system is described by the finite state multi-dimensional continuous-time Markov chain with the upper-Hessenberg block structure of the generator. The stationary distribution of the system states and some important characteristics of the system are calculated. The presented numerical examples illustrate opportunities to quantitatively evaluate the impact of the buffer capacity and customers’ mean arrival rate on the most important characteristics of the system. The possibility of solving optimization problems is briefly shown.

Improvement on Age of Information for Information Update Systems With HARQ Chase Combining and Sensor Harvesting-Transmitting Diversities

The age of information (AoI) has attracted increasing attentions as a new metric and tool to capture the timeliness of reception and freshness of data, which is different from the conventional metric such as outage, delay, and throughput. While most existing works focused on AoI analysis and optimizing method, the information update system design by utilizing diversity to reduce AoI is not fully exploited so far. Inspired by this fact, a novel information status update system is designed by jointing wireless energy harvesting diversity, status update transmitting diversity, and hybrid automatic repeat request (HARQ). Specially, the update system consists of two sensors that can be seen as an unified entity from the destination's perspective, and one destination. The two sensors, which have finite-size batteries and synchronously operate in half-duplex mode, harvest energy from a Bernoulli process source, and cooperatively deliver the status update to destination by sharing a common information source. To exploit the information of the unsuccessfully decoding update packet, the HARQ and chase combining (HARQ-CC) are jointly used at destination so that the HARQ-CC receiving diversity is also exploited. By using Bernoulli process energy arrival probability and sensor transmission probability, we first propose one novel sensor energy harvesting and update delivering model that can be represented by Markov Chain (MC) and develop one novel construction method of the transition matrix of sensor battery energy states. Considering the developed energy state transmission matrix column stochastic, irreducible, and aperiodic, we obtain the stationary distribution of sensor battery energy states so that the steady transmitting power is derived. Secondly, the average AoI of the dual-sensor status update system is derived in terms of a closed-form expression by accurately calculating the statistical descriptions of the maximum retransmission number (time) in HARQ-CC and the service time of one packet. The analytical results are validated by using the comparison from simulations. Moreover, compared with traditional single sensor system, the dual-sensor system not only harvests more energy, but also has smaller average AoI because of the joint exploitation of multi-diversity. Besides, the impact of the system parameters on harvested energy and average AoI is provided, which provides insights in terms of the optimal design of Internet of things system.

Run-and-tumble particles in two dimensions under stochastic resetting conditions

We study the effect of stochastic resetting on a run-and-tumble particle (RTP) in two spatial dimensions. We consider a resetting protocol which affects both the position and orientation of the RTP: the particle undergoes constant-rate positional resetting to a fixed point in space and a random orientation. We compute the radial and x-marginal stationary-state distributions and show that while the former approaches a constant value as r → 0, the latter diverges logarithmically as x → 0. On the other hand, both the marginal distributions decay exponentially with the same exponent when they are far from the origin. We also study the temporal relaxation of the RTP and show that the positional distribution undergoes a dynamic transition to a stationary state. We also study the first-passage properties of the RTP in the presence of resetting and show that the optimization of the resetting rate can minimize the mean first-passage time. We also provide a brief discussion of the stationary states for resetting a particle to an initial position with a fixed orientation.

Random acceleration process under stochastic resetting

We consider the motion of a randomly accelerated particle in one dimension under stochastic resetting mechanism. Denoting the position and velocity by x and v respectively, we consider two different resetting protocols—(i) complete resetting: here both x and v reset to their initial values x 0 and v 0 at a constant rate r, (ii) partial resetting: here only x resets to x 0 while v evolves without interruption. For complete resetting, we find that the particle attains stationary state in both x and v. We compute the non-equilibrium joint stationary state of x and v from which we obtain the stationary state distribution for position by integrating over v. We also study the late time relaxation of the position distribution function. On the other hand, for partial resetting, the joint distribution is always in the transient state. At large t, the position distribution possesses a scaling behaviour which we rigorously derive. Next, we study the first passage time properties with an absorbing wall at the origin. For complete resetting, we find that the mean first passage time (MFPT) is rendered finite by the resetting mechanism. We explicitly derive the expressions for the MFPT and the survival probability at large t. However, in stark contrast, for partial resetting, we find that resetting does not render finite MFPT. This is because even though x is brought to x 0, the large fluctuation in v (typically of the order ) can take the particle substantially far from the origin. All our analytic results are corroborated by the numerical simulations.

Strategic behavior and social optimization in a clearing queueing system with N-policy and stochastic restarting scheme

Considering the power consumption and delay performance, we propose a clearing queueing system with N-policy and stochastic restarting scheme. At the service completion epoch, the server switches to the dormant state to save energy if there are no customers waiting for services, otherwise initiates a new clearing service. After a dormant period, the server will be activated if there are at least N waiting customers, and be activated stochastically with less than N waiting customers. We derive the stationary distribution of system states with the closed form and an effective algorithm is designed to obtain the distribution of the sojourn time. In addition, a reward-cost structure is introduced in the queueing system. We analyze customers’ equilibrium strategic behaviours based on the optimistic value and provide a comparison between the criteria of using average and optimistic value by numerical examples. Finally, we discuss the social optimization problems and present some numerical experiences to illustrate impacts of system parameters on the socially optimal strategies and social optimal welfare.