Multi-dimensional Continuous-time Markov Chain Research Articles

Stability of Queueing Systems with Impatience, Balking and Non-Persistence of Customers

The operation of many queueing systems is adequately described by the structured multidimensional continuous-time Markov chains. The most well-studied classes of such chains are level-independent Quasi-Birth-and-Death processes, GI/M/1 type and M/G/1 type Markov chains, generators of which have the block tri-diagonal, lower- and upper-Hessenberg structure, respectively. All these classes assume that the matrices of transition rates are quasi-Toeplitz. This property greatly simplifies their analysis but makes them inappropriate for the study of many important systems, e.g., retrial queues with a retrial rate depending on the number of customers in orbit, queues with impatient customers, etc. The importance of such systems attracts significant interest to their analysis. However, in the literature, there is a methodological gap relating to the ergodicity condition of the corresponding Markov chains. To fulfill this gap and facilitate the analysis of a wide range of such systems, we show that under non-restrictive assumptions, the following hold true: (i) if the customers can balk or are impatient or non-persistent, then the Markov chain describing the behavior of the system belongs to the class of asymptotically quasi-Toeplitz Markov chains; (ii) this chain is ergodic; (iii) known algorithms can be applied for the calculation of the stationary distribution of the corresponding queueing system.

Analysis of Tandem Queue with Multi-Server Stages and Group Service at the Second Stage

In this paper, we consider a tandem dual queuing system consisting of multi-server stages. Stage 1 is characterized by an infinite buffer, one-by-one service of customers, and an exponential distribution of service times. Stage 2 is characterized by a finite buffer and a phase-type distribution of service times. Service at Stage 2 is provided to groups of customers. The service time of a group depends on the size of the group. The size is restricted by two thresholds. The waiting time of a customer at each stage is limited by a random variable with an exponential distribution, with the parameter depending on the stage. After service at Stage 1, a customer can depart from the system or try to enter Stage 2. If the buffer at this stage is full, the customer is either lost or returns for service at Stage 1. Customer arrivals are described by the versatile Markov arrival process. The system is studied via consideration of a multi-dimensional continuous-time Markov chain. Numerical examples, which highlight the influence of the thresholds on the system performance measures, are presented. The possibility of solving optimization problems is illustrated.

Queueing system with batch arrival of heterogeneous orders, flexible limited processor sharing and dynamical change of priorities

<abstract><p>A queueing system with the discipline of flexible limited sharing of the server is considered. This discipline assumes the admission, for a simultaneous service, of only a finite number of orders, as well as the use of a reduced service rate when the bandwidth required by the admitted orders is less than the total bandwidth of the server. The orders arrive following a batch-marked Markov arrival process, which is a generalization of the well-known $ MAP $ (Markov arrival process) to the cases of heterogeneous orders and batch arrivals. The orders of different types have different preemptive priorities. The possibility of an increase or a decrease in order priority during the service is suggested to be an effective mechanism to prevent long processing orders from being pushed out of service by just-arrived higher-priority orders. Under a fixed priority scheme and a mechanism of dynamic change of the priorities, the stationary analysis of this queueing system is implemented by considering a suitable multidimensional continuous-time Markov chain with a generator that has an upper Hessenberg structure. The possibility of the optimal restriction on the number of simultaneously serviced orders is numerically demonstrated.</p></abstract>

Analysis of a Multi-Server Queue with Group Service and Service Time Dependent on the Size of a Group as a Model of a Delivery System

In this paper, we consider a multi-server queue with a finite buffer. Request arrivals are defined by the Markov arrival process. Service is provided to groups of requests. The minimal and maximal group sizes are fixed. The service time of a group has a phase-type distribution with an irreducible representation depending on the size of the group. The requests are impatient. The patience time for an arbitrary request has an exponential distribution. After this time expires, the request is lost if all servers are busy or, if some server is idle, with a certain probability, all requests staying in the buffer start their service even if their number is below the required minimum. The behavior of the system is described by a multi-dimensional continuous-time Markov chain that does not belong to the class of level-independent quasi-birth-and-death processes. The algorithm for the computation of the stationary distribution of this chain is presented, and expressions for the computation of the queuing system’s performance characteristics are derived. The description of a delivery system operation in terms of the analyzed queuing model is given, and the problem of the optimization of its operation is numerically solved. Multi-server queues with a phase-type distribution for the group service time that are dependent on the size of the group, the account of request impatience, and the correlated arrival process have not previously been analyzed in the existing literature. However, they represent a precise model of many real-world objects, including delivery systems.

Performability analysis of multi-service call admission control schemes in LTE networks

Two important Quality-of-Service (QoS) measures for cellular networks are the fractions of New Calls (NCs) and Handoff Calls (HCs). These types of calls are usually blocked or dropped for two reasons: radio channels are occupied (performance reason) or failure of radio channels (availability reason). Call Admission Control (CAC) is a technique that accepts or rejects user requests in a network. This paper proposes two new CAC schemes for Long Term Evolution (LTE) networks supporting two classes of services (Real-Time (RT) and Non-Real-Time (NRT)). The proposed CAC schemes are composites, meaning that they consider performance changes associated with the failure and recovery of radio channels. We use the Probabilistic Model Checking (PMC) to perform a comparative analysis between the performability measures (combination of performance and availability measures) of the proposed composite CAC schemes and an existing pure performance model. First, we model the composite CAC schemes with multidimensional Continuous Time Markov Chain (CTMC). Then, we specify QoS requirements through the CTMC using the Continuous-time Stochastic Logic (CSL). Finally, we quantify the performability measures of the composite CAC schemes by checking CSL steady-state, transient, and path formulas using the PRISM model checker.

Analysis of a tandem queuing system with blocking and group service in the second node

ABSTRACT In this paper, we study a tandem queuing system in which the arrivals occur according to a versatile Markovian point process. The buffer sizes in the first and second nodes are, respectively, infinite and finite. We incorporate the impatience of the customers in both the nodes. The service times in both nodes are assumed to be of phase type. While the services in the first node are offered on a first-come-first-served basis, we assume that in the second node the services are provided in batches of varying sizes based on two pre-determined thresholds. The system is studied as a multi-dimensional continuous time Markov chain and a few illustrative numerical examples are reported.

Simulation of multidimensional diffusions with sticky boundaries via Markov chain approximation

In this study, we develop a novel method to simulate multidimensional diffusions with sticky boundaries for which the Euler scheme fails. We approximate the sticky diffusion process by a multidimensional continuous-time Markov chain (CTMC) that can be easily simulated. We provide two approaches to construct the CTMC. In the first approach, we approximate the infinitesimal generator of the sticky diffusion by finite difference using standard coordinate directions. In the second one, we match the local moments using the drift and the eigenvectors of the covariance matrix as transition directions. The first approach does not always guarantee a valid Markov chain, whereas the second one can. We show that both construction methods yield a first-order simulation scheme, which can capture the sticky behavior and is free from the curse of dimensionality. As applications, we use our method to simulate the limit of a queuing system with exceptional service policy and a multi-factor short rate model for low interest rate environment.

Queueing-Inventory with One Essential and m Optional Items with Environment Change Process Forming Correlated Renewal Process (MEP)

We consider a queueing inventory with one essential and m optional items for sale. The system evolves in environments that change randomly. There are n environments that appear in a random fashion governed by a Marked Markovian Environment change process. Customers demand the main item plus none, one, or more of the optional items, but were restricted to at most one unit of each optional item. Service time of the main item is phase type distributed and that of optional items have exponential distributions with parameters that depend on the type of the item, as well as the environment under consideration. If the essential item is not available, service will not be provided. The lead times of optional and main items have exponential distributions having parameters that depend on the type of the item. The condition for stability of the system is analyzed by considering a multi-dimensional continuous time Markov chain that represent the evolution of the system. Under this condition, various performance characteristics of the system are derived. In terms of these, a cost function is constructed and optimal control policies of the different types of commodities are investigated. Numerical results are provided to give a glimpse of the system performance.

Analysis of Single-Server Multi-Class Queue with Unreliable Service, Batch Correlated Arrivals, Customers Impatience, and Dynamical Change of Priorities

A single-server non-pre-emptive priority queueing system of a finite capacity with many types of customers is analyzed. Inter-arrival times can be correlated and batch arrivals are allowed. Possible unreliability of the server, implying the loss of a customer or the necessity of its service from the early beginning or some phase of the service, is taken into account. Initial priorities provided to various types of customers at the arrival moment can be varied (increased or decreased) after the random amount of time during the customer stay in the buffer. Such a type of queues arises in the modeling operation of various emergency care systems, information, and perishable goods delivering systems, etc. The stationary behavior of the system is described by the finite state multi-dimensional continuous-time Markov chain with the upper-Hessenberg block structure of the generator. The stationary distribution of the system states and some important characteristics of the system are calculated. The presented numerical examples illustrate opportunities to quantitatively evaluate the impact of the buffer capacity and customers’ mean arrival rate on the most important characteristics of the system. The possibility of solving optimization problems is briefly shown.

An Exact Analytical Model for an IoT Network with MMPP Arrivals

Analytical modeling of the Internet of Things (IoT) networks is challenging. This is due to the presence of a large number of devices in these networks and the complexity of the priorities between different types of traffic. Taking these aspects into account, the objective of this paper is to analyze the performance of an IoT network where the IoT devices work independently of one another. To this end, we developed a novel multi-dimensional Continuous-Time Markov Chain (CTMC) model with threshold-based preemption. In this model, each IoT device is modeled as a Markov Modulated Poisson Process (MMPP) that can transmit regular and alarm packets. Alarm packets have higher priority over regular packets. To measure access to the channel between alarm and regular packets, we introduced a threshold parameter where the threshold is the number of packets in the alarm queue that indicates when preemption starts. The performance measures include blocking probability, the average delay of regular packets and alarm packets, discard rate , and success probability of regular packets. Comprehensive numerical analysis was conducted. Our results indicate that impact of the threshold on performance measures is higher on the boundary values of the threshold. The model was proven to be efficient in analyzing the performance of IoT networks on a wide range of parameter values. These results may be used in the future to develop and assess a protocol that utilizes a scheduling algorithm with a dynamic preemption threshold to optimize the performance of the IoT network.

Cloud Firewall Under Bursty and Correlated Data Traffic: A Theoretical Analysis

Cloud firewalls stand as one of the major building blocks of the cloud security framework protecting the Virtual Private Infrastructure against attacks such as the Distributed Denial of Service (DDoS). In order to fully characterize the cloud firewall operation and gain actionable insights on the design of cloud security, performance models for the cloud firewall become imperative. In this article, we propose a multi-dimensional Continuous-Time Markov Chain model for the cloud firewall that takes into account the burstiness and correlation features of the legitimate and malicious data traffic. By adopting the Markov-Modulated Poisson process (MMPP) and the Interrupted Poisson Process (IPP), we identify the workload conditions under which the cloud firewall might be subject to a loss of availability. Furthermore, by comparing the IPP and Poisson attacks, we numerically verify that the cloud firewall is inherently vulnerable to a burstiness-aware attack which might seriously compromise its operation. Additionally, we characterize the joint harmful impact of burstiness and correlation on the cloud firewall that might lead to performance degradation. Finally, we design an elastic cloud firewall by proposing a MMPP-driven load balancing procedure that provisions virtual firewalls dynamically while fulfilling a Service Level Agreement (SLA) latency specification.

Queuing System with Two Types of Customers and Dynamic Change of a Priority

The use of priorities allows us to improve the quality of service of inhomogeneous customers in telecommunication networks, inventory and health-care systems. An important modern direction of research is to analyze systems in which priority of a customer can be changed during his/her stay in the system. We considered a single-server queuing system with a finite buffer, where two types of customers arrive according to a batch marked Markov arrival process. Type 1 customers have non-preemptive priority over type 2 customers. Low priority customers are able to receive high priority after the random amount of time. For each non-priority customer accepted into the buffer, a timer, which counts a random time having a phase type distribution, is switched-on. When the timer expires, the customer with some probability leaves the system unserved and with the complimentary probability gains the high priority. Such a type of queues is typical in many health-care systems, contact centers, perishable inventory, etc. We describe the behavior of the system by a multi-dimensional continuous-time Markov chain and calculate a number of the stationary performance measures of the system including the various loss probabilities as well as the distribution function of the waiting time of priority customers. The illustrative numerical examples giving insights into the system behavior are presented.

Channel allocation and ultra-reliable communication in CRNs with heterogeneous traffic and retrials: A dependability theory-based analysis

The current research efforts on Fifth Generation (5G) of wireless communication systems have identified the need for large extent improvements in accessibility and reliability of communication services. In this respect, Cognitive Radio (CR) has been envisioned as a key 5G enabler that allows dynamic spectrum access without causing interference to licensed (primary) users and can tackle the challenge of ultra reliable communication. Channel failures, which are generally caused due to hardware and software failure and/or due to intrinsic features such as fading and shadowing, can easily result in network performance degradation. In cognitive radio networks (CRNs), the connections of unlicensed (secondary) users are inherently vulnerable to breaks due to channel failures as well as licensed users’ arrivals. To explore the advantages of channel reservation and retrial phenomenon on performance improvement in error-prone channels, we propose and analyze dynamic spectrum access (DSA) scheme by also taking balking and reneging behavior into account. Moreover, since 5G networks should comprise heterogeneous applications that may have different Quality of Service (QoS), thus the present study facilitates the arrival of heterogeneous secondary users with access privilege variations. In addition, most previous works have studied the stationary performance of CRNs, however, those may not be adequate in practice, notably when the time horizon of operations is finite. This paper investigates the transient dynamics from the perspectives of dependability theory in CRNs. Furthermore, the whole system is modeled using a multi-dimensional continuous time Markov chain (CTMC) and numerical results illustrate the potential of the proposed scheme to achieve major gains in the performance of error-prone CRNs.

Analysis of retrial queue with heterogeneous servers and Markovian arrival process

Multi-server retrial queueing system with heterogeneous servers is analyzed. Requests arrive to the system according to the Markovian arrival process. Arriving primary requests and requests retrying from orbit occupy an available server with the highest service rate, if there is any available server. Otherwise, the requests move to the orbit having an infinite capacity. The total retrial rate infinitely increases when the number of requests in orbit increases. Service periods have exponential distribution. Behavior of the system is described by multi-dimensional continuous-time Markov chain which belongs to the class of asymptotically quasi-toeplitz Markov chains. This allows to derive simple and transparent ergodicity condition and compute the stationary probabilities distribution of chain states. Presented numerical results illustrate the dynamics of some system effectiveness indicators and the importance of considering of correlation in the requests arrival process.

Tighter Lyapunov Truncation for Multi-Dimensional Continuous Time Markov Chains with Known Moments

Continuous Time Markov chains (CTMCs) are widely used to model and analyze networked systems. A common analysis approach is to solve the system of balance equations governing the state transitions of a CTMC to obtain its steadystate probability distribution, and use the state probabilities to derive or compute various performance measures. In many systems, the state space of the underlying CTMC is infinite and multi-dimensional with state-dependent transitions; exact analysis of such models is challenging. For example, the exact probability distribution of the number of jobs in the Discriminatory Processor Sharing (DPS) system, first proposed by Kleinrock in 1967 [4], is still an open challenge. Likewise, obtaining the exact state probabilities of quasi-birth-and-death (QBD) processes with leveldependent transitions is known to be challenging [1]; QBDs are infinite state space multi-dimensional Markov chains in which states are organized into levels and transitions are skip-free between the levels.

Queueing Network with Moving Servers as a Model of Car Sharing Systems

We consider a queueing network with a finite number of nodes and servers moving between the nodes as a model of car sharing. The arrival process of customers to various nodes is defined by a marked Markovian arrival process. The customer that arrives at a certain node when there is no idle server (car) is lost. Otherwise, he/she is able to start the service. With known probability, which depends on the node and the number of available cars, this customer can balk the service and leave the system. The service time of a customer has an exponential distribution. Location of the server in the network after service completion is random with the known probability distribution. The behaviour of the network is described by a multi-dimensional continuous-time Markov chain. The generator of this chain is derived which allows us to compute the stationary distribution of the network states. The formulas for computing the key performance indicators of the system are given. Numerical results are presented. They characterize the dependence of some performance measures of the network and the nodes on the total number of cars (fleet size of the car sharing system) and correlation in the arrival process.

Analysis of an Adaptive Priority-Based Resource Sharing Scheme for Multiservice IoT Communications Over LTE-A Networks

At present, the revolution of the Internet of things (IoT) is an area of technology that is growing exponentially. IoT has a significant impact on consumer products and is expected to dominate in a wide variety of areas such as business, industry, and governments. In high-density deployments of IoT systems, providing sufficient networking and computing infrastructure with reliable and low-latency communication links is a challenging task. This is owing to the resource allocation methods applied at the cell edge that do not guarantee fair and efficient resource partitioning when receiving different types of IoT traffic. Deployment of an efficient and fair resource-partitioning scheme for IoT devices is necessary to reduce the effects of IoT traffic on network performance. The main contribution of this paper is to design efficient resource partitioning for different types of IoT device communications with an adaptive sharing priority (AS) scheme. Moreover, we analyze and compare the performance of conventional admission control schemes such as fixed sharing and complete sharing with our proposed scheme using a developed simulation as well as analytical modeling with a multidimensional continuous-time Markov chain. We also present and discuss the performance results of the AS scheme in terms of blocking probabilities, average waiting delay, and system utilization using different scenarios with different parameters. The results demonstrate that the AS scheme performs better than the two previously studied conventional schemes.

Analysis of the BMAP/PH/N queueing system with backup servers

• A multi-server queue with help of backup servers in case of long service is introduced. • Constructive ergodicity condition is obtained. • Various performance measures of the system are computed. • Numerical results giving insight into the system behavior are presented. We consider a novel multi-server queueing system that is potentially useful for optimizing real-world systems, in which the objectives of high performance and low power consumption are conflicting. The queueing model is formulated and investigated under the assumption that an arrival flow is defined by a batch Markovian arrival process and random values characterizing customer processing have the phase-type distribution. If the service time of some customer by a server exceeds a certain random bound, this server receives help from a so-called backup server from a finite pool of backup servers. The behavior of the system is described by a quite complicated multi-dimensional continuous-time Markov chain that is successfully analyzed in this paper. Examples of the potential use of the obtained results in managerial decisions are presented.

On compact solution vectors in Kronecker-based Markovian analysis

State based analysis of stochastic models for performance and dependability often requires the computation of the stationary distribution of a multidimensional continuous-time Markov chain (CTMC). The infinitesimal generator underlying a multidimensional CTMC with a large reachable state space can be represented compactly in the form of a block matrix in which each nonzero block is expressed as a sum of Kronecker products of smaller matrices. However, solution vectors used in the analysis of such Kronecker-based Markovian representations require memory proportional to the size of the reachable state space. This implies that memory allocated to solution vectors becomes a bottleneck as the size of the reachable state space increases. Here, it is shown that the hierarchical Tucker decomposition (HTD) can be used with adaptive truncation strategies to store the solution vectors during Kronecker-based Markovian analysis compactly and still carry out the basic operations including vector–matrix multiplication in Kronecker form within Power, Jacobi, and Generalized Minimal Residual methods. Numerical experiments on multidimensional problems of varying sizes indicate that larger memory savings are obtained with the HTD approach as the number of dimensions increases.

A Buffering and Switching Scheme for Admission Control in Cognitive Radio Networks

Cognitive radio users (i.e., secondary users) experience a lot of performance degradation due to the continuous appearance of primary users in the cognitive radio networks. To minimize the effect of this phenomenon, we propose a buffering and switching scheme for admission control to enhance the performance of the secondary users. In the proposed scheme, the secondary users are competing for the available channels (i.e., licensed and unlicensed) with two different kinds of users and utilizing different kinds of buffers to reduce the blocking probability for newly arriving users and dropping probability for ongoing connection. The system is modeled using a multi-dimensional continuous-time Markov chain, where the performance metrics are obtained. The numerical results are verified using a discrete event simulations.