Generalized semi-Markov Process Research Articles

The ORIS tool

ORIS is a tool for quantitative modeling and evaluation of concurrent systems with non-Markovian durations. It provides a Graphical User Interface (GUI) for model specification as Stochastic Time Petri Nets (STPNs), validation by interactive simulation, and evaluation by several techniques, computing instantaneous and cumulative rewards. It also provides an open-source Java Application Programming Interface (API) to automate the workflow, and it can be used as a toolkit for derivation and evaluation of STPNs in model driven engineering. As distinguishing features, ORIS implements transient and steady-state analysis of STPNs with underlying Markov Regenerative Process (MRP), and transient analysis of STPNs with underlying Generalized Semi- Markov Process (GSMP). It also implements nondeterministic analysis of Time Petri Nets (TPNs), simulation of STPNs, and solution methods for Continuous-Time Markov Chains (CTMCs) and MRPs with at most one non-exponential timer in each state. The well-engineered software architecture of ORIS supports agile implementation of new STPN features, new modeling formalisms, and new analysis methods.

A Language for Agent-based Discrete-event Modeling and Simulation of Linked Lives

In agent-based modeling and simulation, discrete-time methods prevail. While there is a need to cover the agents’ dynamics in continuous time, commonly used agent-based modeling frameworks offer little support for discrete-event simulation. Here, we present a formal syntax and semantics of the language ML3 (Modeling Language for Linked Lives) for modeling and simulating multi-agent systems as discrete-event systems. The language focuses on applications in demography, such as migration processes, and considers this discipline’s specific requirements. These include the importance of life courses being linked and the age-dependency of activities and events. The developed abstract syntax of the language combines the metaphor of agents with guarded commands. Its semantics is defined in terms of Generalized Semi-Markov Processes. The concrete language has been realized as an external domain-specific language. We discuss implications for efficient simulation algorithms and elucidate benefits of formally defining domain-specific languages for modeling and simulation.

BIRNBAUM CRITICALITY AND IMPORTANCE MEASURES FOR MULTISTATE SYSTEMS WITH REPAIRABLE COMPONENTS

We suggest four new measures of importance for repairable multistate systems based on the classical Birnbaum measure. Periodic component life cycles and general semi-Markov processes are considered. Similar to the Birnbaum measure, the proposed measures are generic in the sense that they only depend on the probabilistic properties of the components and the system structure. The multistate system model encodes physical properties of the components and the system directly into the structure function. As a result, calculating importance is easy, especially in the asymptotic case. Moreover, the proposed measures are composite measures, combining importance for all component states into a unified quantity. This simplifies ranking of the components with respect to importance. The proposed measures can be characterized with respect to two features: forward-looking versus backward-looking and with respect to how criticality is measured. Forward-looking importance measures focus on the next component states, while backward-looking importance measures focus on the previous component states. Two approaches to measuring criticality are considered: probability of criticality versus expected impact. Examples show that the different importance measures may result in unequal rankings.

When are stochastic transition systems tameable?

A decade ago, Abdulla, Ben Henda and Mayr introduced the elegant concept of decisiveness for denumerable Markov chains [1]. Roughly speaking, decisiveness allows one to lift most good properties from finite Markov chains to denumerable ones, and therefore to adapt existing verification algorithms to infinite-state models. Decisive Markov chains however do not encompass stochastic real-time systems, and general stochastic transition systems (STSs for short) are needed. In this article, we provide a framework to perform both the qualitative and the quantitative analysis of STSs. First, we define various notions of decisiveness (inherited from [1]), notions of fairness and of attractors for STSs, and make explicit the relationships between them. Then, we define a notion of abstraction, together with natural concepts of soundness and completeness, and we give general transfer properties, which will be central to several verification algorithms on STSs. We further design a generic construction which will be useful for the analysis of ω-regular properties, when a finite attractor exists, either in the system (if it is denumerable), or in a sound denumerable abstraction of the system. We next provide algorithms for qualitative model-checking, and generic approximation procedures for quantitative model-checking. Finally, we instantiate our framework with stochastic timed automata (STA), generalized semi-Markov processes (GSMPs) and stochastic time Petri nets (STPNs), three models combining dense-time and probabilities. This allows us to derive decidability and approximability results for the verification of these models. Some of these results were known from the literature, but our generic approach permits to view them in a unified framework, and to obtain them with less effort. We also derive interesting new approximability results for STA, GSMPs and STPNs.

On Transience and Recurrence in Irreducible Finite-State Stochastic Systems

Long-run stochastic stability is a precondition for applying steady-state simulation output analysis methods to a discrete-event stochastic system, and is of interest in its own right. We focus on systems whose underlying stochastic process can be represented as a Generalized Semi-Markov Process (GSMP); a wide variety of stochastic systems fall within this framework. A fundamental stability requirement for an irreducible GSMP is that the states be “recurrent” in that the GSMP visits each state infinitely often with probability 1. We study recurrence properties of irreducible GSMPs with finite state space. Our focus is on the “clocks” that govern the occurrence of events, and we consider GSMPs in which zero, one, or at least two simultaneously active events can have clock-setting distributions that are “heavy tailed” in the sense that they have infinite mean. We establish positive recurrence, null recurrence, and, perhaps surprisingly, possible transience of states for these respective regimes. The transience result stands in strong contrast to Markovian or semi-Markovian GSMPs, where irreducibility and finiteness of the state space guarantee positive recurrence.

Transient analysis of non-Markovian models using stochastic state classes

The method of stochastic state classes approaches the analysis of Generalised Semi Markov Processes (GSMPs) through the symbolic derivation of probability density functions over supports described by Difference Bounds Matrix (DBM) zones. This makes steady state analysis viable, provided that at least one regeneration point is visited by every cyclic behaviour of the model.We extend the approach providing a way to derive transient probabilities. To this end, stochastic state classes are extended with a supplementary timer that enables the symbolic derivation of the distribution of time at which a class can be entered. The approach is amenable to efficient implementation when model timings are given by expolynomial distributions, and it can be applied to perform transient analysis of GSMPs within any given time bound. In the special case of models underlying a Markov Regenerative Process (MRGP), the method can also be applied to the symbolic derivation of local and global kernels, which in turn provide transient probabilities through numerical integration of generalised renewal equations. Since much of the complexity of this analysis is due to the local kernel, we propose a selective derivation of its entries depending on the specific transient measure targeted by the analysis.

Bio-PEPAd: A non-Markovian extension of Bio-PEPA

Delays in biological systems may be used to model events for which the underlying dynamics cannot be precisely observed, or to provide abstraction of some behavior of the system resulting in more compact models. In this paper, we enrich the stochastic process algebra Bio-PEPA, with the possibility of assigning delays to actions, yielding a new non-Markovian stochastic process algebra: Bio-PEPAd. This is a conservative extension meaning that the original syntax of Bio-PEPA is retained and the delay specification which can now be associated with actions may be added to existing Bio-PEPA models. The semantics of the firing of the actions with delays is the delay-as-duration approach, earlier presented in papers on the stochastic simulation of biological systems with delays. This semantics of the algebra is given in the Starting–Terminating style, meaning that the state and the completion of an action are observed as two separate events, as required by delays. We formally define the encoding of Bio-PEPAd systems in Generalized Semi-Markov Processes (GSMPs), as input for a Delay Stochastic Simulation Algorithm (DSSA) and as sets of Delay Differential Equations (DDEs), the deterministic framework for modeling of biological systems with delays. Finally, we prove theorems stating the relation between Bio-PEPA and Bio-PEPAd models. We end the paper with an example model of biological systems with delays to illustrate the approach.

Occupation measure functionals in merging phase space

This paper deals with the averaging and diffusion approximation for occupation time of general phase space semi-Markov processes under split with simple and double merging in a series scheme. These results seem to be particularly interesting since the variance coefficient is obtained in a simple form. The method of proof of weak convergence results here is based on the construction of the compensating operator of the extended Markov renewal processes and on the solution of the singular perturbation problem of this operator acting on test functions.

Simulation of Generalised Semi-Markov Processes based on Graph Transformation Systems

Stochastic Graph Transformation combines graphical modelling of various software artefacts with stochastic analysis techniques. Existing approaches are restricted to processes with exponential time distribution. Such processes are sufficient for modelling a significant class of stochastic systems, however there are interesting systems which cannot be specified appropriately in such a framework. In several cases one needs to consider non-exponential time distributions. This paper proposes a stochastic model based on graph transformation with general probability distributions. This model is well suited to represent concurrency and performance aspects of architecture reconfiguration. It is also possible to apply Monte Carlo simulation techniques in order to analyse behaviour of complex stochastic systems. The new model is implemented and used to simulate simple networks.

Optimal Threshold Levels in Stochastic Fluid Models via Simulation-based Optimization

A number of important problems in production and inventory control involve optimization of multiple threshold levels or hedging points. We address the problem of finding such levels in a stochastic system whose dynamics can be modelled using generalized semi-Markov processes (GSMP). The GSMP framework enables us to compute several performance measures and their sensitivities from a single simulation run for a general system with several states and fairly general state transitions. We then use a simulation-based optimization method, sample-path optimization, for finding optimal hedging points. We report numerical results for systems with more than twenty hedging points and service-level type probabilistic constraints. In these numerical studies, our method performed quite well on problems which are considered very difficult by current standards. Some applications falling into this framework include designing manufacturing flow controllers, using capacity options and subcontracting strategies, and coordinating production and marketing activities under demand uncertainty.

Laws of Large Numbers and Functional Central Limit Theorems for Generalized Semi-Markov Processes

Because of the fundamental role played by generalized semi-Markov processes (GSMPs) in the modeling and analysis of complex discrete-event stochastic systems, it is important to understand the conditions under which a GSMP exhibits stable long-run behavior. To this end, we review existing work on strong laws of large numbers (SLLNs) and functional central limit theorems (FCLTs) for GSMPs; our discussion highlights the role played by the theory of both martingales and regenerative processes. We also sharpen previous limit theorems for finite-state irreducible GSMPs by establishing a SLLN and FCLT under the “natural” requirements of finite first (resp., second) moments on the clock-setting distribution functions. These moment conditions are comparable to the minimal conditions required in the setting of ordinary semi-Markov processes (SMPs). Corresponding discrete-time results for the underlying Markov chain of a GSMP are also provided. In contrast to the SMP setting, limit theorems for finite-state GSMPs require additional structural assumptions beyond irreducibility, due to the presence of multiple clocks. In our new limit theorems, the structural assumption takes the form of a “positive density” condition for specified clock-setting distributions. As part of our analysis, we show that finite moments for new clock readings imply finite moments for the od-regenerative cycles of both the GSMP and its underlying chain.

Existence of Optimal Policies for Semi-Markov Decision Processes Using Duality for Infinite Linear Programming

Semi-Markov decision processes on Borel spaces with deterministic kernels have many practical applications, particularly in inventory theory. Most of the results from general semi-Markov decision processes do not carry over to a deterministic kernel since such a kernel does not provide "smoothness." We develop infinite dimensional linear programming theory for a general stochastic semi-Markov decision process. We give conditions, general enough to allow deterministic kernels, for solvability and strong duality of the resulting linear programs. By using the developed linear programming theory we give conditions for the existence of a stationary deterministic policy for deterministic kernels, which is optimal among all possible policies.

Specification and analysis of power-managed systems

Dynamic power management encompasses several techniques for reducing energy dissipation in electronic systems by selective slowdown or shutdown of components. We present a theoretical framework for explaining and classifying different approaches to power management. Within this framework, we model power-manageable components, workloads, and controllers as discrete-event systems (DESs). The structure of these DESs is specified in terms of physical states (representing operation modes) and events (triggering state transitions), while system behavior is specified in terms of next-event and next-state functions. In particular, nondeterministic next-event and next-state functions are modeled by conditional probability distributions, according to generalized semi-Markov processes (GSMPs). The modeling framework provides a general denotational model for system specification and a rigorous execution semantics that enables event-driven simulation. We introduce a modeling framework, built on top of MathWork's Simulink, supporting the specification and execution of our model. In particular, we present templates for the Simulink simulator to execute GSMP models, and we describe how to use such templates for specifying, analyzing, and optimizing dynamic power-managed systems. Finally, we demonstrate the expressive power and versatility of the proposed approach by using the modeling framework and the simulator for the analysis of representative real-life case studies, including the Intel Xscale processor architecture, a multitasking real-time system, and a sensor network.

On Functional Central Limit Theorems for Semi-Markov and Related Processes

The semi-Markov process (SMP) has long been used as a model for the underlying process of a discrete-event stochastic system. Important refinements of this model include the continuous-time Markov chain (CTMC) and important extensions include the generalized semi-Markov process (GSMP). Functional central limit theorems (FCLTS) give basic conditions under which these various processes exhibit stable long-run behavior, as well as providing approximations for cumulative-reward distributions and confidence intervals for statistical estimators. We give FCLTS for finite-state CTMCS, SMPS, and GSMPS under minimal conditions that involve irreducibility and finite second moments on the “holding time” distributions. We consider both continuous and lump-sum rewards; our emphasis is on the use of martingale theory and on the explicit computation, when possible, of the variance constant in the FCLT.

An Integrated Approach for the Specification and Analysis of Stochastic Real-Time Systems

A formal approach for the specification and analysis of concurrent systems is proposed which integrates two different orthogonal aspects of time: (i) real-time, concerning the expression of time constraints and the verification of exact time properties, and (ii) probabilistic-time, concerning the probabilistic quantification of durations of system activities via exponential probability distributions and the evaluation of system performance. We show that these two aspects, that led to different specification paradigms called timed automata and Markovian process algebras, respectively, can be expressed in an integrated way by a single language: a process algebra capable of expressing activities with generally distributed durations. In particular, we consider the calculus of Interactive Generalized Semi-Markov Processes (IGSMPs) and we present formal techniques for compositionally deriving, from an IGSMP specification, (i) a pure real-time model (called Interactive Timed Automaton), by considering the support of general distributions, and (ii) a pure probabilistic-time model (called Interactive Weighted Markov Chain), by approximating general distributions with phase-type distributions.

Estimation Methods for Delays in Non-regenerative Discrete-Event Systems

Many long-run delay characteristics of computer, communication, manufacturing, and transportation systems can be specified as time-average limits of sequences of delays in generalized semi-Markov processes (GSMPs). We consider sequences of delays that are determined from the state transitions of the GSMP using the method of start vectors. In this setting, time-average limits typically must be estimated using simulation. Previous work on estimation methods for delays has focused on GSMPs for which there exists a sequence of regeneration points. For such systems, it is often possible to find an explicit sequence of regeneration points or od-regeneration points for the sequence of delays, so that point estimates and confidence intervals for time-average limits can be obtained using the regenerative method for simulation output analysis or one of its variants. This paper is concerned with GSMPS for which this approach is not feasible, either because regeneration points for the GSMP cannot be identified or because regenerations occur too infrequently. We provide conditions on the building blocks of a GSMP and start-vector mechanism under which the sequence of delays is an od-regenerative process and cycle sums have finite moments—these conditions do not require that there exist regeneration points for the GSMP. Although in our setting the od-regeneration points for the sequence of delays usually cannot be determined explicitly, the mere existence of these points implies that time-average limits are well defined and the sequence of delays obeys a multivariate functional central limit theorem. It then follows from results of Glynn, Iglehart, and Muñoz that methods based on standardized time series can be used to obtain strongly consistent point estimates and asymptotic confidence intervals for time-average limits and functions of time-average limits. In particular, the method of batch means is applicable.

The theory of interactive generalized semi-Markov processes

In this paper we introduce the calculus of interactive generalized semi-Markov processes (IGSMPs), a stochastic process algebra which can express probabilistic timed delays with general distributions and synchronizable actions with zero duration, and where choices may be probabilistic, non-deterministic and prioritized. IGSMP is equipped with a structural operational semantics which generates semantic models in the form of generalized semi-Markov processes (GSMPs), i.e. probabilistic systems with generally distributed time, extended with action transitions representing interaction among system components. This is obtained by expressing the concurrent execution of delays through a variant of ST semantics which is based on dynamic names. The fact that names for delays are generated dynamically by the semantics makes it possible to define a notion of observational congruence for IGSMP (that abstracts from internal actions with zero duration) simply as a combination of standard observational congruence and probabilistic bisimulation. We also present a complete axiomatization for observational congruence over IGSMP. Finally, we show how to derive a GSMP from a given IGSMP specification in order to evaluate the system performance and we present a case study.

Non-determinism in Probabilistic Timed Systems with General Distributions

In this paper we address the problem of adequately handling non-deterministic choices in Generalised Semi-Markov Processes (GSMPs), i.e. probabilistic timed systems where durations of delays are expressed by means of random variables with a general probability distribution. In particular we want the probabilistic duration of a delay not to be decided all at once when the delay starts, but step by step in each system state (in the theory of GSMPs this corresponds to recording spent lifetimes instead of residual lifetimes of delays). In this way an adversary cannot take decisions a priori, based on the knowledge he may get about the future behavior of the system. In order to accomplish this, we consider Interactive Generalised Semi-Markov Processes (IGSMPs). We start by formalizing the class of well-named IGSMP models and the class of Interactive Stochastic Timed Transition Systems (ISTTSs) which are both closed under CSP parallel composition and hiding. Then, we introduce a semantics for IGSMPs which maps well-named IGSMP models onto ISTTSs by recording spent lifetimes of delays. Finally, we show that two weakly bisimilar IGSMPs give rise to two weakly bisimilar semantic models and that our semantic mapping is compositional with respect to both CSP parallel composition and hiding.

Analysing sojourn times in queueing networks: A structural approach

Structural properties of generalised semi-Markov processes (GSMP) have been successfully studied in the literature. Examples are the celebrated commuting condition (CC), which is the key condition for unbiasedness of the infinitesimal perturbation analysis (IPA) gradient estimator, or the monotonicity condition (M), which implies stochastic order properties. When coming to queueing networks, these properties can be deduced from simple structural conditions on the topology of the network. Generally speaking, the results known so far apply to event times in GSMPs. Unfortunately, for multi or infinite server queueing networks, event times cannot be translated into customer related performance measures, such as sojourn times or waiting times. To overcome this drawback, we introduce in this paper a new technique, called “compound events”. Compound events enable us to define event times in such a way that results with respect to event times can be translated into results with respect to customer related performance measures. We model a generic queueing network by means of a GSMP with compound events and we establish conditions on the topology of the queueing network that imply that the corresponding GSMP satisfies structural conditions like (CC) and (M). These structural conditions then imply properties of event times, like continuity or stochastic monotonicity. Using compound events, these results can be translated into properties of customer times, like sojourn times or waiting times. Our results extend the area of applicability of IPA to the analysis of customer related performance measures in multi–server queueing networks. Furthermore, we obtain new results on stochastic ordering of customer related performance measures.

Simulation optimization of buffer allocations in production lines with unreliable machines

We use a recent simulation‐based optimization method, sample path optimization, to find optimal buffer allocations in tandem production lines where machines are subject to random breakdowns and repairs, and the product is fluid‐type. We explore some of the functional properties of throughput of such systems and exploit these properties to prove the almost sure convergence of our optimization technique, under a regularity condition on the steady state. Utilizing a generalized semi‐Markov process (GSMP) representation of the system, we derive recursive expressions to compute one‐sided directional derivatives of throughput, from a single simulation run. Finally, we give computational results for lines with up to 50 machines. We also compare results for smaller lines with the results from a more conventional method, stochastic approximation, whenever applicable. In these numerical studies, our method performed quite well on problems that are considered difficult by current computational standards.