Embedded Markov Chain Approach Research Articles

Analysis of renewal input batch service queue with impatient customers and multiple working vacations

ABSTRACTThis paper presents a renewal input single server batch service queue with impatient customers and multiple working vacations. An arriving customer may join the system with probability or balk with probability . The service is carried out in batches with a maximum threshold limit . The server takes a working vacation when there are no customers in the system, and continues taking multiple working vacations as long as there are no customers at working vacation completion epochs. Applying the embedded Markov chain approach and the displacement operator method, we find the probability distribution of the queue-length at pre-arrival epoch. With the help of the Markov renewal theory and semi-Markov processes, we compute the probability distribution of the queue-length at an arbitrary epoch. We also obtain the system-length distributions at both pre-arrival and arbitrary epochs and present a potential application of the model in cloud systems. Some corresponding results underneath special cases are found by putting suitable values. We present various performance measures and waiting-time distributions in the queue. Numerical results are demonstrated to emphasize the qualitative features of the model.

An embedded Markov chain approach to stock rationing under batch orders

We provide an extension to the embedded Markov chain approach of Fadiloglu and Bulut (2010) for the analysis of lot-per-lot inventory systems with backorders under rationing. This extension generalizes the method so that it can be used for the analysis under (Q,R) policy. We introduce a new embedded Markov chain of higher dimensionality that keeps track of the ordering process. We finally provide a recursive procedure to generate the steady-state probabilities from the chain and obtain the steady-state distribution.

Stationary Analysis and Optimal Control Under Multiple Working Vacation Policy in a GI/M(a,b)/1 Queue

This paper considers an infinite buffer renewal input queue with multiple working vacation policy wherein customers are served by a single server according to general bulk service (a, b)-rule (1 ≤ a ≤ b). If the number of waiting customers in the system at a service completion epoch (during a normal busy period) is lower than ‘a’, then the server starts a vacation. During a vacation if the number of waiting customers reaches the minimum threshold size ‘a’, then the server starts serving this batch with a lower rate than that of the normal busy period. After completion of a batch service during working vacation, if the server finds less than ‘a’ customers accumulated in the system, then the server takes another vacation, otherwise the server continues to serve the available batch with that lower rate. The maximum allowed size of a batch in service is ‘b’. The authors derive both queue-length and system-length distributions at pre-arrival epoch using both embedded Markov chain approach and the roots method. The arbitrary epoch probabilities are obtained using the classical argument based on renewal theory. Several performance measures like average queue and system-length, mean waiting-time, cost and profit optimization are studied and numerically computed.

Modeling and Performance Analysis of a Tracking-Area-List-Based Location Management Scheme in LTE Networks

In a cellular network, the network coverage is partitioned into nonoverlapping areas called registration areas to facilitate location management (LM) that keeps track of the location of a user equipment (UE) unit. In the second-generation Global System for Mobile Communications (GSM) and the third-generation Universal Mobile Telecommunications System (UMTS), a registration area is called a location area (LA) in the circuit-switched domain or a routing area (RA) in the packet-switched domain. In the fourth-generation Long-Term Evolution (LTE), a registration area is called a tracking area (TA). To tackle the drawbacks of the LA/RA-based LM scheme used in GSM and UMTS, which is intrinsically a static scheme, LTE adopts a dynamic LM scheme called a TA list (TAL)-based scheme. Under the TAL-based scheme, the network allocates to a UE unit a group of TAs referred to as a TAL instead of only a single TA. The UE can move freely within the allocated TAL without performing any location updates that are called TA updates (TAUs) in LTE. Only when moving into a TA that is not included in the allocated TAL does the UE need to perform a TAU, and afterward, the network will allocate, to the UE, a new TAL. The performance of the TAL-based scheme relies on the allocated TAL. If the allocated TAL is inappropriate with respect to the UE's mobility and traffic characteristics, the TAL-based scheme may produce adverse effects. To find an optimal TAL allocation method for global UE units that exhibit weak regularity in their mobility, this paper develops an embedded Markov chain approach to analyze the signaling cost of the TAL-based scheme. This paper distinguishes itself from existing studies in the following aspects. First, this paper follows LTE technical specifications, that is, the number of TAs in a TAL can vary and, furthermore, is upper bounded, whereas existing studies partially or completely ignore this stipulation. Second, this paper emphasizes the impact of a call handling model that dictates whether a TAU occurs after the completion of a call by considering two call handling models, i.e., a call plus location update and a call without location update. Third, as for the dependence among the cell residence time, the TA residence time, and the TAL residence time, this paper proposes the use of a fluid flow model to describe this dependence, which is simple but does not compromise accuracy. Analytical formulas for the total signaling cost of the TAL-based scheme due to TAU and paging operations, as well as formulas that are useful in designing an optimal paging scheme, are derived, and their accuracy is validated through Mont Carlo simulation. Then, numerical studies are carried out to investigate the impact of diverse parameters on the signaling cost, revealing that the network can allocate, to a UE unit, an optimal TAL that is consistent with the UE's mobility and traffic characteristics to minimize the signaling cost of the TAL-based scheme.

Computationally efficient optimization of stock pooling and allocation levels for two-demand-classes under general lead time distributions

ABSTRACTIn this article we develop a procedure for estimating service levels (fill rates) and for optimizing stock and threshold levels in a two-demand-class model managed based on a lot-for-lot replenishment policy and a static threshold allocation policy. We assume that the priority demand classes exhibit mutually independent, stationary, Poisson demand processes and non-zero order lead times that are independent and identically distributed. A key feature of the optimization routine is that it requires computation of the stationary distribution only once. There are two approaches extant in the literature for estimating the stationary distribution of the stock level process: a so-called single-cycle approach and an embedded Markov chain approach. Both approaches rely on constant lead times. We propose a third approach based on a Continuous-Time Markov Chain (CTMC) approach, solving it exactly for the case of exponentially distributed lead times. We prove that if the independence assumption of the embedded Markov chain approach is true, then the CTMC approach is exact for general lead time distributions as well. We evaluate all three approaches for a spectrum of lead time distributions and conclude that, although the independence assumption does not hold, both the CTMC and embedded Markov chain approaches perform well, dominating the single-cycle approach. The advantages of the CTMC approach are that it is several orders of magnitude less computationally complex than the embedded Markov chain approach and it can be extended in a straightforward fashion to three demand classes.

Equilibrium balking strategies in renewal input batch arrival queues with multiple and single working vacation

This paper investigates equilibrium threshold balking strategies of customers in a renewal input batch arrival queue with multiple and single working vacation of the server. The vacation period, service period during normal service and vacation period are considered to be independent and exponentially distributed. Upon arriving, the customers decide whether to join or balk the queue based on observation of the system-length and status of the server. The waiting time in the system is associated with a linear cost–reward structure for estimating the net benefit if a customer wishes to participate in the system. Equilibrium customer strategy is studied under four cases: fully observable, almost observable, almost unobservable and fully unobservable. Using embedded Markov chain approach and system cost analysis, we obtain the equilibrium threshold. The analysis of unobservable cases is based on the roots of the characteristics equations formed using the probability generating function of embedded pre-arrival epoch probabilities. Equilibrium balking strategy may be useful in quality of service for EPON (ethernet passive optical network) as a multiple working vacation model and accounting through gatekeeper based H.323 protocols as a single working vacation model.

Cost Analysis of a Hybrid-Movement-Based and Time-Based Location Update Scheme in Cellular Networks

A movement-based location update (MBLU) scheme is effective and has low operation complexity. On the other hand, cellular networks use periodic LUs that are intrinsically time-based LUs (TBLUs) to detect the presence of a user equipment (UE) to avoid wasting network resources to deliver incoming calls to an abnormally detached UE. When implemented in a cellular network, the MBLU scheme must incorporate TBLUs to detect the UE presence. Thus, a hybrid-movement-based and time-based LU (HMTBLU) scheme arises naturally. Under the HMTBLU scheme, an LU occurs either when the movement threshold for MBLUs is reached or when the time threshold for TBLUs is reached. In this paper, we develop an embedded Markov chain approach to calculate the signaling cost of the HMTBLU scheme. We derive analytical formulas for the signaling cost and formulas useful for designing an optimal paging scheme. The accuracy of these formulas is validated through simulation. Based on these formulas, we conduct a numerical study to investigate the influence of various parameters on the signaling cost. The derived formulas can guide the implementation of the HMTBLU scheme in cellular networks, including Long-Term Evolution (LTE). Moreover, the developed mathematical approach can be applied to studying other LU schemes such as the LU scheme used in LTE.

Embedded Markov chain approach to retrial queue with vacation, phase repair and multioptional services

The investigation deals with the steady-state behavior of a batch arrival retrial queue with multi-optional services and phase repair under Bernoulli vacation schedule. The customers enter the system in batches and are admitted following Bernoulli admission control policy. The incoming customers are forced to join the retrial group if they find the server unavailable. The customers are served in two phases viz. first essential service (FES) followed by second optional services (SOS). The server is unreliable and if fails, it is repaired in d-compulsory phases so as to become as good as before failure. The server may go for a vacation after each service completion following Bernoulli vacation schedule or it may continue serving the next customer. By applying the embedded Markov chain method, we establish the ergodicity condition for the system. The steady-state formulae for some queueing measures are established by evaluating the generating functions of queue length distribution. The cost function of the system has also been formulated. Finally, the effects of various parameters on the performance of the system have been examined numerically.

A preemptive priority retrial queue with two classes of customers and general retrial times

In this paper, we consider a continuous-time retrial queue with two classes of customers: priority customers and ordinary customers, where priority customers don’t queue and have an exclusive preemptive priority to receive their services over ordinary customers. If an arriving ordinary customer finds the server busy, it enters a retrial group (called orbit) according to FCFS discipline. Only the ordinary customer at the head of the retrial queue is allowed to access the server. Firstly, we obtain the necessary and sufficient condition for the system to be stable by embedded Markov chain approach. Secondly, using supplementary variable method, we obtain the stationary probability distribution and some performance measures of interest. Thirdly, we give the analysis of the sojourn time in the system of an arbitrary ordinary customer. Lastly, numerical examples are given to show the effect of system parameters on several performance measures.

Service Systems with Finite and Heterogeneous Customer Arrivals

We consider service systems with a finite number of customer arrivals, where customer interarrival times and service times are both stochastic and heterogeneous. Applications of such systems are numerous and include systems where arrivals are driven by events or service completions in serial processes as well as systems where servers are subject to learning or fatigue. Using an embedded Markov chain approach, we characterize the waiting time distribution for each customer, from which we obtain various performance measures of interest, including the expected waiting time of a specific customer, the expected waiting time of an arbitrary customer, and the expected completion time of all customers. We carry out extensive numerical experiments to examine the effect of heterogeneity in interarrival and service times. In particular, we examine cases where interarrival and service times increase with each subsequent arrival or service completion, decrease, increase and then decrease, or decrease and then increase. We derive several managerial insights and discuss implications for settings where such features can be induced. We validate the numerical results using a fluid approximation that yields closed-form expressions.

Cost Analysis of Movement-Based Location Management in PCS Networks: An Embedded Markov Chain Approach

In this paper, we develop an approach of embedded Markov chain to analyze the signaling cost of a movement-based location management (MBLM) scheme. This approach distinguishes itself from those developed in the literature in the following aspects. 1) It considers the location area (LA) architecture used by personal communication service (PCS) networks for location management. 2) It considers two different call handling models that determine after a call whether a location update should be performed. 3) It considers the effect of the call holding time on the call handling models. 4) It proposes to use a fluid flow model to describe the dependence between the cell and the LA residence time. We derive closed-form analytical formulas for the signaling cost, whose accuracy is manifested by a simulation. Based on the analytical formulas, we conduct a numerical study to evaluate the influence of various parameters on the signaling cost. The formulas can contribute to the implementation of the MBLM scheme in PCS networks including Fourth-Generation (4G) Long-Term Evolution. The modeling approach developed in this paper can be exploited to model other location management schemes.

Time-Dependent Queue-Size Distribution in the Finite GI/M/l Model with AQM-Type Dropping

A GI=M=1-type queueing system with finite buffer capacity and AQM-type packet dropping is investigated. Even when the buffer is not saturated an incoming packet can be dropped (lost) with probability dependent on the instanteneous queue size. The system of integral equations for time-dependent queue-size distribution conditioned by the number of packets present in the system initially is built using the embedded Markov chain approach. The solution of the corresponding system written for Laplace transforms is found using the linear algebra. Numerical examples, in which different-type dropping functions are investigated in some network-motivated traffic scenarios, are attached as well.

The Discrete-Time Bulk-ServiceGeo/Geo/1Queue with Multiple Working Vacations

This paper deals with a discrete-time bulk-serviceGeo/Geo/1queueing system with infinite buffer space and multiple working vacations. Considering an early arrival system, as soon as the server empties the system in a regular busy period, he leaves the system and takes a working vacation for a random duration at timen. The service times both in a working vacation and in a busy period and the vacation times are assumed to be geometrically distributed. By using embedded Markov chain approach and difference operator method, queue length of the whole system at random slots and the waiting time for an arriving customer are obtained. The queue length distributions of the outside observer’s observation epoch are investigated. Numerical experiment is performed to validate the analytical results.

An embedded Markov chain approach to stock rationing

We propose a new method for the analysis of lot-per-lot inventory systems with backorders under rationing. We introduce an embedded Markov chain that approximates the state-transition probabilities. We provide a recursive procedure for generating these probabilities and obtain the steady-state distribution.

Availability Analysis of an Intrusion Tolerant Distributed Server System With Preventive Maintenance

We consider availability models of an intrusion tolerant system, and investigate quantitative effects of preventive maintenance based on security patch releases. The stochastic behavior of the system is analyzed through an embedded Markov chain approach. More specifically, two semi-Markov models are formulated in continuous-time, and discrete-time scales. We derive the optimal preventive patch management times maximizing the steady-state system availability in respective models, and evaluate both the system availability, and the mean time to security failure. Numerical examples are presented for illustrating the optimal preventive maintenance policies, and performing sensitivity analysis of model parameters.

A polling model with an autonomous server

This paper considers polling systems with an autonomous server that remains at a queue for an exponential amount of time before moving to a next queue incurring a generally distributed switch-over time. The server remains at a queue until the exponential visit time expires, also when the queue becomes empty. If the queue is not empty when the visit time expires, service is preempted upon server departure, and repeated when the server returns to the queue. The paper first presents a necessary and sufficient condition for stability, and subsequently analyzes the joint queue-length distribution via an embedded Markov chain approach. As the autonomous exponential visit times may seem to result in a system that closely resembles a system of independent queues, we explicitly investigate the approximation of our system via a system of independent vacation queues. This approximation is accurate for short visit times only.

A BAYESIAN APPROACH TO FIND RANDOM-TIME PROBABILITIES FROM EMBEDDED MARKOV CHAIN PROBABILITIES

The embedded Markov chain approach is widely used in queuing theory, in particular in M/G/1 and GI/M/c queues. In these cases, one has to relate the embedded equilibrium probablities to the corresponding random-time probabilities. The classical method to do this is based on Markov renewal theory, a rather complex approach, especially if the population is finite or if there is balking. In this article we present a much simpler method to derive the random-time probabilities from the embedded Markov chain probabilities. The method is based on conditional probability. Our approach might also be applicable in such situations.

Optimal design and performance modelling of M/G/1/ K queueing systems

Approximating the performance measures of M/G/1/ K systems is a difficult, challenging, and important problem for applications in science and engineering. An approach based on a two-moment approximation of the process is presented and is contrasted with an embedded Markov chain approach, Gelenbe's approach, simulation, and finally, the statistics of M/M/1/ K systems. The closed form expressions for the different performance measures should be very handy. The use of the approximation in the performance modelling and design of M/G/1/ K systems is also explored in order to demonstrate the practical usefulness of the concepts contained within the paper.

M/G/1/N Queue with vacation time and limited service discipline

A variant of an M/ G/1 queuing model with finite waiting room is studied in this paper. In this system, every busy period of the server at the queue is followed by the execution of additional tasks. The time spent by the server to perform these tasks is called a vacation time away from the queue. The server will begin a vacation from the queue if either the queue has been emptied or M customers have been served during the visit. The embedded Markov chain approach is used to obtain the steady state queue length distribution. The Laplace-Stieltjes transforms of the busy period and cycle time distributions are given. Using a combination of the supplementary variables and sample biasing techniques, the waiting time distribution, blocking probability and general queue length distribution are derived. Finally the results for the case of infinite waiting places are provided.

On the waiting time distribution in a GI/G/1 queue with a Coxian–2 service time distribution

The purpose of this paper is to derive in an alternative way the result that the complementary waiting–time distribution function in the Gl/G/I queue is the sum of two exponential functions when the service time has a Coxian–2 distribution. The idea is to interpret this type of service–time distribution as the sum of a stochastic number of exponentially distributed phases. In this way the model can be seen as a special G/x/W/1 batch–arrival queue where the batch–size distribution is deduced from the Coxian service–time distribution. For the latter model we give an embedded Markov–chain approach. Because of the special form of the batch–size distribution the steady–state distribution of this Markov chain can be represented as the sum of two geometric terms of which the coefficients can be explicitly given. From this result the waiting–time distribution can be deduced immediately. Apart from its didactic interest the result can be useful to obtain simple approximations for more general GI/G/1 models.