Discrete-event Stochastic Systems Research Articles

Discrete-event stochastic systems with correlated inputs: Modeling and performance evaluation

Discrete-event stochastic systems with correlated inputs: Modeling and performance evaluation

Discrete-event stochastic systems with copula correlated input processes

In this article, we develop a new method based on copulas to model correlated inputs in discrete-event stochastic systems. We first define a type of correlated stochastic process, called Copula Correlated Processes (CCPs), which we then use to model correlated inputs for discrete-event stochastic systems. In general, it is very difficult to analyze discrete-event stochastic systems with correlated inputs. However, we show that discrete-event stochastic systems with CCPs can be discretized and approximated by discrete-event stochastic systems with discrete copula correlated processes, which are equivalent to discrete-event stochastic systems driven by Markov-modulated processes and are much easier to analyze. An illustrative queueing example is provided to demonstrate how our method works.

Complexity reduction techniques for quantified diagnosability of stochastic systems

In a discrete event stochastic system, the natural notion of diagnosability, called A-diagnosability, requires that each fault event is eventually detected with probability one. Several definitions of diagnosability degree have been derived from this notion. They examine the detection probability after a fault occurs. To check diagnosability and compute diagnosability degrees, one usually attaches to the original stochastic system the information of a so-called diagnoser, which is in general exponentially larger than the original system. In this paper, we show that the full complexity of such diagnosers is not necessary, and that one can rely on simpler systems, with up to an exponential gain in complexity.

On Transience and Recurrence in Irreducible Finite-State Stochastic Systems

Long-run stochastic stability is a precondition for applying steady-state simulation output analysis methods to a discrete-event stochastic system, and is of interest in its own right. We focus on systems whose underlying stochastic process can be represented as a Generalized Semi-Markov Process (GSMP); a wide variety of stochastic systems fall within this framework. A fundamental stability requirement for an irreducible GSMP is that the states be “recurrent” in that the GSMP visits each state infinitely often with probability 1. We study recurrence properties of irreducible GSMPs with finite state space. Our focus is on the “clocks” that govern the occurrence of events, and we consider GSMPs in which zero, one, or at least two simultaneously active events can have clock-setting distributions that are “heavy tailed” in the sense that they have infinite mean. We establish positive recurrence, null recurrence, and, perhaps surprisingly, possible transience of states for these respective regimes. The transience result stands in strong contrast to Markovian or semi-Markovian GSMPs, where irreducibility and finiteness of the state space guarantee positive recurrence.

Controlled exploration of state space in off-line ADP and its application to stochastic shortest path problems

This paper addresses the problem of finding a control policy that drives a generic discrete event stochastic system from an initial state to a set of goal states with a specified probability. The control policy is iteratively constructed via an approximate dynamic programming (ADP) technique over a small subset of the state space that is evolved via Monte Carlo simulations. The effect of certain user-chosen parameters on the performance of the algorithm is investigated. The method is evaluated on several stochastic shortest path (SSP) examples and on a manufacturing job shop problem. We solve SSP problems that contain up to one million states to illustrate the scaling of computational and memory benefits with respect to the problem size. In the case of the manufacturing job shop example, the proposed ADP approach outperforms a traditional rolling horizon math programming approach.

Optimization Models of Discrete-Event System Dynamics

A methodology is given for modeling the dynamics of discrete-event stochastic systems as optimization problems. The intent is to provide a means to utilize the rich mathematical theory and algorithms of optimization in the study of this important class of systems. A procedure for mapping a simulation event relationship graph into a mixed-integer program is presented, along with examples of queueing networks and manufacturing systems that illustrate the approach. Several potential applications are examined, including automatic constraint generation for optimal resource scheduling, representations of max-plus algebra models for queueing system dynamics, response gradient estimation, and an unconventional technique for simulating queueing systems using virtual resources that are identified from the optimization models for these systems.

Laws of Large Numbers and Functional Central Limit Theorems for Generalized Semi-Markov Processes

Because of the fundamental role played by generalized semi-Markov processes (GSMPs) in the modeling and analysis of complex discrete-event stochastic systems, it is important to understand the conditions under which a GSMP exhibits stable long-run behavior. To this end, we review existing work on strong laws of large numbers (SLLNs) and functional central limit theorems (FCLTs) for GSMPs; our discussion highlights the role played by the theory of both martingales and regenerative processes. We also sharpen previous limit theorems for finite-state irreducible GSMPs by establishing a SLLN and FCLT under the “natural” requirements of finite first (resp., second) moments on the clock-setting distribution functions. These moment conditions are comparable to the minimal conditions required in the setting of ordinary semi-Markov processes (SMPs). Corresponding discrete-time results for the underlying Markov chain of a GSMP are also provided. In contrast to the SMP setting, limit theorems for finite-state GSMPs require additional structural assumptions beyond irreducibility, due to the presence of multiple clocks. In our new limit theorems, the structural assumption takes the form of a “positive density” condition for specified clock-setting distributions. As part of our analysis, we show that finite moments for new clock readings imply finite moments for the od-regenerative cycles of both the GSMP and its underlying chain.

On Functional Central Limit Theorems for Semi-Markov and Related Processes

The semi-Markov process (SMP) has long been used as a model for the underlying process of a discrete-event stochastic system. Important refinements of this model include the continuous-time Markov chain (CTMC) and important extensions include the generalized semi-Markov process (GSMP). Functional central limit theorems (FCLTS) give basic conditions under which these various processes exhibit stable long-run behavior, as well as providing approximations for cumulative-reward distributions and confidence intervals for statistical estimators. We give FCLTS for finite-state CTMCS, SMPS, and GSMPS under minimal conditions that involve irreducibility and finite second moments on the “holding time” distributions. We consider both continuous and lump-sum rewards; our emphasis is on the use of martingale theory and on the explicit computation, when possible, of the variance constant in the FCLT.

Estimation of delays in non-regenerative discrete-event stochastic systems

Estimation of delays in non-regenerative discrete-event stochastic systems

Transient analysis of deterministic and stochastic Petri nets with concurrent deterministic transitions

This paper introduces an efficient numerical algorithm for transient analysis of deterministic and stochastic Petri nets (DSPNs) and other discrete-event stochastic systems with exponential and deterministic events. The proposed approach is based on the analysis of a general state space Markov chain (GSSMC) whose state equations constitute a system of multidimensional Fredholm integral equations. Key contributions of this paper constitute the observations that the transition kernel of this system of Fredholm equations is piece-wise continuous and separable. Due to the exploitation of these properties, the GSSMC approach shows great promise for being effectively applicable for the transient analysis of large DSPNs with concurrent deterministic transitions. Moreover, for DSPNs without concurrent deterministic transitions the proposed GSSMC approach requires three orders of magnitude less computational effort than the previously known approach based on the method of supplementary variables.

On simulation output analysis for generalized semi-markov processes

The usual model for the underlying process of a discrete-event stochastic system is the generalized semi-Markov process (GSMP). A GSMP is defined in terms of a general state space Markov chain that describes the process at successive state-transition times. We provide conditions on the clock-setting distributions and state-transition probabilities of a finite state GSMP under which this underlying chain is φ-irreducible and satisfies a drift criterion for stability due to Meyn and Tweedie. If the GSMP also has a “single state” in which exactly one event is scheduled to occur, then this state is hit infinitely often with probability I and the time between successive hits has finite second moment. It follows that the standard regenerative method for analysis of simulation output can be used to obtain strongly consistent point estimates and asymptotic confidence intervals for time-average limits of the process. We also show that, under our conditions, point estimates and confidence intervals for time-average limits can be obtained using methods based on standardized time series. In particular, the method of batch means is applicable. Our results rest on a new functional central limit theorem for GSMP's together with results of Glynn and Iglehart. The standardized-time-series methods apply even when the GSMP does not have a single state or indeed any type of classical regenerative structure

Information-efficient control of discrete event stochastic systems

A state of a discrete event stochastic system (DESS) can be represented by a tuple of time-varying discrete parameters. The authors have extended the theory of controlled discrete event systems developed by Ramadge, Wonham and other researchers to stochastic modeling and performance measurement. This work presents some important characteristics of the DESS model. The problem of making the most efficient use of information-processing resources (sensors, computer capacity, etc.) in a special class of controlled systems is stated in a new, two-stage format. The format is used to develop a heuristic solution procedure of the problem. Finally, we perform a brief discussion of the model applicability to real-life design of automatic supervisors.

Numerical analysis of deterministic and stochastic Petri nets with concurrent deterministic transitions

This paper introduces an efficient numerical algorithm for the steady-state analysis of deterministic and stochastic Petri nets (DSPNs) without structural restrictions on the enabling of deterministic transitions. The method rests on observation, at equidistant time points, of the continuous-time Markov process that records tangible markings of the DSPN and remaining firing times associated with deterministic transitions. This approach results in the analysis of a general state space Markov chain whose system of stationary equations can be transformed into a system of Volterra equations. The techniques of this paper are also applicable to queueing networks, stochastic process algebras, and other discrete-event stochastic systems with an underlying stochastic process which can be represented as a generalized semi-Markov process with exponential and deterministic events.

Numerical analysis of deterministic and stochastic Petri nets with concurrent deterministic transitions

This paper introduces an efficient numerical algorithm for transient analysis of deterministic and stochastic Petri nets (DSPNs) and other discrete-event stochastic systems with exponential and deterministic events. The proposed approach is based on the analysis of a general state space Markov chain (GSSMC) whose state equations constitute a system of multidimensional Fredholm integral equations. Key contributions of this paper constitute the observations that the transition kernel of this system of Fredholm equations is piece-wise continuous and separable. Due to the exploitation of these properties, the GSSMC approach shows great promise for being effectively applicable for the transient analysis of large DSPNs with concurrent deterministic transitions. Moreover, for DSPNs without concurrent deterministic transitions the proposed GSSMC approach requires three orders of magnitude less computational effort than the previously known approach based on the method of supplementary variables.

Maximum likelihood estimation for generalized semi-markov processes

Parametric statistical inference for generalized semi-Markov processes is addressed. This class of processes encompasses a large number of “real-world” discrete-event stochastic systems. Because of its properties (e.g., consistency, asymptotic normality, etc.), maximum likelihood estimation is considered here. Under reasonable conditions on the process, we show that a maximum likelihood estimator exists, and that it converges to the true parameter at ratet−1/2, wheret is the length of the observation period. A related estimator, which is typically easier to compute, is also introduced. We show that the use of this estimator results in no loss of statistical efficiency. It is also shown that the estimation problem does decouple into separate subproblems when the process' transition probabilities and event distributions depend on different parameters.

Estimation methods for passage times using one-dependent cycles

The lengths of certain passage-time intervals (random time intervals) in discrete-event stochastic systems correspond to delays in computer, communication, manufacturing, and transportation systems. Simulation is often the only available means for analyzing a sequence of such lengths. It is sometimes possible to obtain meaningful estimates for the limiting average delay indirectly, that is, without measuring lengths of individual passage-time intervals. For general time-average limits of a sequence of delays, however, it is necessary to measure individual lengths and combine them to form point and interval estimates. We consider sequences of delays determined by state transitions of a generalized semi-Markov process and introduce a recursively-generated sequence of realvalued random vectors, called start vectors, to provide the link between the starts and terminations of passage-time intervals. This method of start vectors for measuring delays avoids the need to “tag” entities in the system. We show that if the generalized semi-Markov process has a recurrent single-state, then the sample paths of any sequence of delays can be decomposed into one-dependent, identically distributed cycles. We then show that an extension of the regenerative method for analysis of simulation output can be used to obtain meaningful point estimates and confidence intervals for time-average limits. This estimation procedure is valid not only when there are no ongoing passage times at any regeneration point but, unlike previous methods, also when the sequence of delays does not inherit regenerative structure. Application of these methods to a manufacturing cell with robots is discussed.

Information-efficient robotic control

SUMMARYThe work demonstrates a new approach to design of a level of intelligent control of robotic systems. The analytical model results in operational decisions. The structure of these decisions make them readily available to be implemented as an expert system. The approach is applied to a case study of mobile supervisory robots. The model presented here was motivated by manufacturing robotic systems and a type of autonomous robots that collect information at different sites for safety and other control purposes. The robot actions are directly affected by the obta~ned data. At each site the amount of available information (and thus the correctness of the robot decision) can be increased if the robot keeps collecting data at that site for a longer period of t~me. This means a delay in reacting and in reaching next site and accordingly, a decrease in the general amount of robot's information on the whole system.The method of finding an economic amount of information collected by a robot at each site is based on the theory of controlled discrete event stochastic systems developed in our earlier works. This theory combines he basic concepts of discrete event control extended to stochastic systems with some aspects of information economics.

Passage times in colored stochastic petri nets

Passage times in colored stochastic Petri nets correspond to delays in discrete-event stochastic systems. Formal definition of a sequence of passage times in a colored stochastic Petri net is in terms of the underlying general state space Markov chain of the marking process. Using symmetry of the net with respect to color, we provide conditions under which a sequence of passage times is a regenerative process in discrete time with finite cycle-length moments. The regenerative property implies time-average convergence, convergence in distribution, and a central limit theorem for sequences of passage times. It follows that strongly consistent point estimates and asymptotic confidence intervals for general characteristics of passage times can be obtained by simulating a finite portion of a single sample path of the underlying general state space Markov chain. Using regenerative structure of the marking process and a version of Little's Law given by Glynn and Whitt, we also obtain conditions under which a seq...

Optimization and Analysis of Discrete Event Stochastic Systems with Application to Manufacturing Automation

This paper outlines a nonprobabilistic (time domain) approach to the study of discrete event dynamical systems typically represented by queueing networks and production systems in manufacturing. It attempts to unify the development of "operational analysis" in queueing theory with some recently developed techniques in production systems based on control-theoretic considerations.