Multi-dimensional Markov Chain Research Articles

Framework for Analysis of Queueing Systems with Correlated Arrival Processes and Simultaneous Service of a Restricted Number of Customers in Scenarios with an Infinite Buffer and Retrials

In this paper, we create a framework for the uniform algorithmic analysis of queueing systems with the Markov arrival process and the simultaneous service of a restricted number of customers, described by a multidimensional Markov chain. This chain behaves as the finite-state quasi-death process between successive service-beginning epochs, with jumps occurring at these epochs. Such a description of the service process generalizes many known mechanisms of restricted resource sharing and is well suited for describing various future mechanisms. Scenarios involving customers who cannot enter service upon arrival, access via waiting in an infinite buffer, and access via retrials are considered. We compare the generators of the multidimensional Markov chains describing the operation of queueing systems with a buffer and with retrials and show that the sufficient conditions for the ergodicity of these systems coincide. The computation of the stationary distributions of these chains is briefly discussed. The results can be used for performance evaluation and capacity planning of various queueing models with the Markov arrival process and a variety of different service mechanisms that provide simultaneous service to many customers.

A study of a loss system with priorities

The Erlang loss formula, also known as the Erlang B formula, has been known for over a century and has been used in a wide range of applications, from telephony to hospital intensive care unit management. It provides the blocking probability of arriving customers to a loss system involving a finite number of servers without a waiting room. Because of the need to introduce priorities in many services, an extension of the Erlang B formula to the case of a loss system with preemptive priority is valuable and essential. This paper analytically establishes the consistency between the global balance (steady state) equations for a loss system with preemptive priorities and a known result obtained using traffic loss arguments for the same problem. This paper, for the first time, derives this known result directly from the global balance equations based on the relevant multidimensional Markov chain. The paper also addresses the question of whether or not the well-known insensitivity property of the Erlang loss system is also applicable to the case of a loss system with preemptive priorities, provides explanations, and demonstrates through simulations that, except for the blocking probability of the highest priority customers, the blocking probabilities of the other customers are sensitive to the service time distributions and that a larger service time variance leads to a lower blocking probability of the lower priority traffic.

Analysis of Queueing System with Dynamic Rating-Dependent Arrival Process and Price of Service

We consider a multi-server queueing system with a visible queue and an arrival flow that is dynamically dependent on the system’s rating. This rating reflects the level of customer satisfaction with the quality and price of the provided service. A higher rating implies a higher arrival rate, which motivates the service provider to increase the price of the service. A steady-state analysis of this system using the proposed mechanism for changing the rating and a threshold strategy for changing the price is performed. This is carried out via the consideration of a suitably constructed multidimensional Markov chain. The impact of the variation in the threshold defining the strategy for changing the price on the key performance indicators is numerically illustrated. The results can be used to make managerial decisions, leading to an increase in the effectiveness of system operations.

Performance characteristics of the fork-join queuing system

Objectives. The problem of investigating a fork-join queuing system is considered. It is required to build the process of the system functioning, to find the condition for the existence of a stationary distribution, and propose algorithms for calculating the stationary distribution and the main stationary performance characteristics. The special interest of the study is to obtain the lower and upper bounds of the mean sojourn time of a customer in the system.Methods. Methods of probability theory, queuing theory and matrix theory are used.Results. The functioning of the system is described in terms of a multidimensional Markov chain. A constructive condition for the existence of a stationary distribution is found, and algorithms for calculating the stationary distribution and stationary performance characteristics of the system are proposed. Analytical expressions are obtained for the lower and upper bounds of the mean sojourn time of customers in the system.Conclusion. The functioning of the fork-join queuing system with a stationary Poisson flow has been studied. Each of the arriving customers forks into two tasks that go to two subsystems, each of which consists of a server and a buffer. We assume that the buffer to one of the servers is unlimited, and to the second server has a finite volume. Service times have, generally speaking, different phase distributions (PH-Phase type distributions). For this system, a condition for the existence of a stationary distribution is obtained, algorithms for calculating the stationary distribution and a number of stationary performance measures of the system are proposed. Analytical expressions for the lower and upper bounds of the mean sojourn time of a customer in the system from the moment it enters the system to the moment of synchronization, which is a critical performance indicator of the fork-join queues, are obtained. The results of the study can be used for modeling various computer and communication systems, in particular, systems that perform parallel computing, customer processing in distributed databases, and parallel disk access.

Efficient and accurate Lyapunov function-based truncation technique for multi-dimensional Markov chains with applications to discriminatory processor sharing and priority queues

Online service providers aim to satisfy the tail performance requirements of customers through Service-Level Objectives (SLOs). One approach to ensure tail performance requirements is to model the service as a Markov chain and obtain its steady-state probability distribution. However, obtaining the distribution can be challenging, if not impossible, for certain types of Markov chains, such as those with multi-dimensional or infinite state-space and state-dependent transitions. Examples include M/M/1 with Discriminatory Processor Sharing (DPS) and preemptive M/M/c with multiple priority classes and customer abandonment.To address this fundamental problem, we propose a Lyapunov function-based state-space truncation technique that leverages moments or bounds on moments of the state variables. This technique allows us to obtain tight truncation bounds while ensuring arbitrary probability mass guarantees for the truncated chain. We highlight the efficacy of our technique for multi-dimensional DPS and M/M/c priority queue with abandonment, demonstrating a substantial reduction in state space (up to 74%) compared to existing approaches. Additionally, we present three practical use cases that highlight the applicability of our truncation technique by analyzing the performance of the DPS system.

A Two-Stage PBFT Architecture With Trust and Reward Incentive Mechanism

Consensus algorithm is an essential ingredient of any blockchain system. Many different consensus mechanisms such as Practical Byzantine Fault Tolerance (PBFT), Proof-of-Work (PoW), Proof-of-Stake (PoS), and their many derivatives have been proposed over the years, but the complementary problems of performance and resilience to malicious behavior of the nodes have yet to be resolved in a satisfactory manner. In this work, we propose a consensus mechanism that integrates PoS with PBFT, which can effectively deal with dishonest nodes, both individual validators and leaders, whilst maintaining high performance. Our model incentivized truthful behavior by using trust score and reward mechanisms as crucial components of the block validation and ordering processes. The performance of the proposed scheme is evaluated using an analytical model that employs a semi-Markov process, defined by an ergodic multidimensional Markov chain with a finite number of states. The results show the efficiency of the proposed model in consensus-based decision making, even under a high likelihood of dishonest node behavior.

Markov chain modelling of ordered Rayleigh fading channels in non‐orthogonal multiple access wireless networks

AbstractA first‐order finite‐state Markov chain (FSMC) typically models the Rayleigh fading channel in the open literature because the first‐order FSMC is analytically tractable and can derive closed‐form results. Non‐orthogonal multiple access (NOMA) has been recognised as a novel wireless technology that addresses challenges in the next generation of mobile communications. According to the power‐domain NOMA protocol, channels in the NOMA wireless network are sorted by the channel gain. Then considering NOMA, there is insufficient information on how to further form a suitable model for ordered Rayleigh fading channels based on the first‐order FSMC. Given the mathematical statement on how to model the order statistics of multidimensional Markov chains for ordered Rayleigh fading channels, the authors consider these order statistics as a Markov chain, and propose specific processes of representing the state space and constructing the transition probability matrix accordingly. Numerical and simulation results validate the mathematical correctness and accuracy of these novel processes. In addition, for ordered Rayleigh fading channels, the performances of various methods of partitioning the entire signal‐to‐noise ratio range are compared. The performance comparison results are the same as those obtained for the individual unordered Rayleigh fading channel.

Analysis of Multi-Server Priority Queueing System with Hysteresis Strategy of Server Reservation and Retrials

A multi-server queueing system with two types of requests and preemptive priority of one type is considered as a model of a cell of a cognitive radio system under practical suggestions about the arrival flows. A hysteresis type strategy for server reservation is suggested to mitigate the effect of interruption of service of low priority requests. Under the arbitrarily fixed values of the sets of the thresholds defining this strategy, the behavior of the system is described by a level-dependent multi-dimensional Markov chain. Formulas for computation of values of performance characteristics of the system are derived. Numerical examples illustrating the dependence of the main performance characteristics on the thresholds defining the strategy of control and the numerical solution of the problem of the optimal choice of the thresholds are reported.

Asymptotic optimality of speed-aware JSQ for heterogeneous service systems

The Join-the-Shortest-Queue (JSQ) load-balancing scheme is known to minimise the average delay of jobs in homogeneous systems consisting of identical servers. However, it performs poorly in heterogeneous systems where servers have different processing rates. Finding a delay optimal scheme remains an open problem for heterogeneous systems. In this paper, we consider a speed-aware version of the JSQ scheme for heterogeneous systems and show that it achieves delay optimality in the fluid limit. One of the key issues in establishing this optimality result for heterogeneous systems is to show that the sequence of steady-state distributions indexed by the system size is tight in an appropriately defined space. The usual technique for showing tightness by coupling with a suitably defined dominant system does not work for heterogeneous systems. To prove tightness, we devise a new technique that uses the drift of exponential Lyapunov functions. Using the non-negativity of the drift, we show that the stationary queue length distribution has an exponentially decaying tail — a fact we use to prove tightness. Another technical difficulty arises due to the complexity of the underlying state-space and the separation of two time-scales in the fluid limit. Due to these factors, the fluid-limit turns out to be a function of the invariant distribution of a multi-dimensional Markov chain which is hard to characterise. By using some properties of this invariant distribution and using the monotonicity of the system, we show that the fluid limit is has a unique and globally attractive fixed point.

A queueing system with a batch Markovian arrival process and varying priorities

We consider herein a single-server queueing system with a finite buffer and a batch Markovian arrival process. Customers staying in the buffer may have a lower or higher priority. Immediately after arrival each of the customer is assigned the lowest priority and a timer is set for it, which is defined as a random variable distributed according to the phase law and having two absorbing states. After the timer enters one of the absorbing states, the customer may leave the system forever (get lost) or change its priority to the highest. When the timer enters another absorbing state, the customer is lost with some probability and the timer is set again with an additional probability. If a customer enters a completely full system, it is lost. Systems of this type can serve as mathematical models of many real-life medical care systems, contact centers, perishable food storage systems, etc. The operation of the system is described in terms of a multidimensional Markov chain, the stationary distribution and a number of performance characteristics of the system are calculated.

Modeling the Offloading Scheme With Unreliable VM-Related Service for MEC Networks

Multiaccess edge computing (MEC) allows terminal devices (TDs) with limited energy and computing capabilities to offload tasks to edge servers, which have more energy and computing capabilities to provide better service. In order to accommodate more users, an edge server can be divided into many virtual machines (VMs), but the interference among VMs could cause decreased service quality. At the same time, due to server hardware and software errors, there are random temporary failures regardless of whether a virtual machine is serving or not. This article uses a multidimensional Markov chain to establish a unified performance analysis model that jointly considers multiple types of terminals, different local service capabilities, different request arrival traffic, different virtual machine numbers, and different virtual machine failure rates. The statistical models of access blocking probability, resource availability, capacity, forced dropping probability, and retainability are presented. Furthermore, we analyze whether and how the local service should be performed when the virtual machine used fails and propose a backward offloading scheme. Analysis results show that some of quality is not monotonous. The model in this article provides a powerful and accurate analytical method for the preperformance simulation of actual engineering design.

Priority queueing system with many types of requests and restricted processor sharing

A priority queueing model with many types of requests and restricted processor sharing is considered. A novel discipline of requests admission and service is proposed. This discipline assumes restriction of the bandwidth (capacity) of the server and the number of requests that can receive service in the system at the same time. This discipline is some kind of realistic hybrid of the traditional discipline of service in a multi-server system and the discipline of the limited processor sharing. The requests of the highest priority can push out from the service the low priority requests. Therefore, the important problem is fitting of the number of requests that can receive service at the same time to the bandwidth of the server. This problem is solved via construction and analysis of a multi-dimensional Markov chain describing operation of the system under any fixed set of the system parameters.

Analysis of a priority queueing system with the enhanced fairness of servers scheduling

AbstractA multi-server queueing system with two types of customers is analyzed. Both types of customers have own reserved servers and also there is a pool of servers that can be used by both types of customers. Type 1 customers have almost preemptive priority with flexible use of a priority depending on the current number of customers of this type in an infinite buffer. Type 2 customers do not have an input buffer and are lost in cases of absence of available servers upon arrival or expelling (forced termination) from service by Type 1 customers. A multi-dimensional Markov chain, which defines the dynamics of the considered system under the fixed total number of servers, numbers of reserved servers, a pattern of the correlated arriving processes, service rates, and the thresholds defining the mechanism of expelling Type 2 customers from service, is analyzed. A numerical example of solving an optimization problem based on the obtained results is considered. Results can be used for the elaboration of enhanced protocols of servers scheduling in many real-world systems including cognitive radio networks with channels leasing.

A retrial queueing system with processor sharing and impatient customers

Objectives. The problem of constructing and investigating a mathematical model of a stochastic system with processor sharing, repeated calls, and customer impatience is considered. This system is formalized in the form of a queueing system. The operation of the queue is described in terms of multi-dimensional Markov chain. A condition for the existence of a stationary distribution is found, and algorithms for calculating the stationary distribution and stationary performance characteristics of the system are proposed.Methods. Methods of probability theory, queueing theory and matrix theory are used.Results. The steady state operation of a queueing system with repeated calls, processor sharing and two types of customers arriving in a marked Markovian arrival process is studied. The channel bandwidth is divided between two types of customers in a certain proportion, and the number of customers of each type simultaneously located on the server is limited. Customers of one of the types that have made all the channels assigned to them busy leave the system unserved with some probability and, with an additional probability, go to the orbit of infinite size, from where they make attempts to get service at random time intervals. Customers of the second type, which caused all the channels assigned to them to be busy, are lost. Customers in orbit show impatience: each of them can leave orbit forever if the time of its stay in orbit exceeds some random time distributed according to an exponential law. Service times of customers of different types are distributed according to the phase law with different parameters. The operation of the system is described in terms of a multi-dimensional Markov chain. It is proved that for any values of the system parameters this chain has a stationary distribution. Algorithms for calculating the stationary distribution and a number of performance measures of the system are proposed. The results of the study can be used to simulate the operation of a fixed capacity cell in a wireless cellular communication network and other real systems operating in the processor sharing mode.

A stochastic process on a network with connections to Laplacian systems of equations

AbstractWe study an open discrete-time queueing network. We assume data is generated at nodes of the network as a discrete-time Bernoulli process. All nodes in the network maintain a queue and relay data, which is to be finally collected by a designated sink. We prove that the resulting multidimensional Markov chain representing the queue size of nodes has two behavior regimes depending on the value of the rate of data generation. In particular, we show that there is a nontrivial critical value of the data rate below which the chain is ergodic and converges to a stationary distribution and above which it is non-ergodic, i.e., the queues at the nodes grow in an unbounded manner. We show that the rate of convergence to stationarity is geometric in the subcritical regime.

Self-Service System with Rating Dependent Arrivals

A multi-server infinite buffer queueing system with additional servers (assistants) providing help to the main servers when they encounter problems is considered as the model of real-world systems with customers’ self-service. Such systems are widely used in many areas of human activity. An arrival flow is assumed to be the novel essential generalization of the known Markov Arrival Process (MAP) to the case of the dynamic dependence of the parameters of the MAP on the rating of the system. The rating is the process defined at any moment by the quality of service of previously arrived customers. The possibilities of a customers immediate departure from the system at the entrance to the system and the buffer due to impatience are taken into account. The system is analyzed via the use of the results for multi-dimensional Markov chains with level-dependent behavior. The transparent stability condition is derived, as well as the expressions for the key performance indicators of the system in terms of the stationary probabilities of the Markov chain. Numerical results are provided.

Performance Analysis of A Queueing System with Server Arrival and Departure

In many systems, in order to fulfill demand (computing or other services) that varies over time, service capacities often change accordingly. In this paper, we analyze a simple two dimensional Markov chain model of a queueing system in which multiple servers can arrive to increase service capacity, and depart if a server has been idle for too long. It is well known that multi-dimensional Markov chains are in general difficult to analyze. Our focus is on an approximation method of stationary performance of the system via the Stein method. For this purpose, innovative methods are developed to estimate the moments of the Markov chain, as well as the solution to the Poisson equation with a partial differential operator.

Retrial BMAP/PH/N Queueing System with a Threshold-Dependent Inter-Retrial Time Distribution

In this paper, we study a multi-server queueing system with retrials and an infinite orbit. The arrival of primary customers is described by a batch Markovian arrival process (BMAP), and the service times have a phase-type (PH) distribution. Previously, in the literature, such a system was mainly considered under the strict assumption that the intervals between the repeated attempts from the orbit have an exponential distribution. Only a few publications dealt with retrial queueing systems with non-exponential inter-retrial times. These publications assumed either the rate of retrials is constant regardless of the number of customers in the orbit or this rate is constant when the number of orbital customers exceeds a certain threshold. Such assumptions essentially simplify the mathematical analysis of the system, but do not reflect the nature of the majority of real-life retrial processes. The main feature of the model under study is that we considered the classical retrial strategy under which the retrial rate is proportional to the number of orbital customers. However, in this case, the assumption of the non-exponential distribution of inter-retrial times leads to insurmountable computational difficulties. To overcome these difficulties, we supposed that inter-retrial times have a phase-type distribution if the number of customers in the orbit is less than or equal to some non-negative integer (threshold) and have an exponential distribution in the contrary case. By appropriately choosing the threshold, one can obtain a sufficiently accurate approximation of the system with a PH distribution of the inter-retrial times. Thus, the model under study takes into account the realistic nature of the retrial process and, at the same time, does not resort to restrictions such as a constant retrial rate or to rough truncation methods often applied to the analysis of retrial queueing systems with an infinite orbit. We describe the behavior of the system by a multi-dimensional Markov chain, derive the stability condition, and calculate the steady-state distribution and the main performance indicators of the system. We made sure numerically that there was a reasonable value of the threshold under which our model can be served as a good approximation of the BMAP/PH/N queueing system with the PH distribution of inter-retrial times. We also numerically compared the system under consideration with the corresponding queueing system having exponentially distributed inter-retrial times and saw that the latter is a poor approximation of the system with the PH distribution of inter-retrial times. We present a number of illustrative numerical examples to analyze the behavior of the system performance indicators depending on the system parameters, the variance of inter-retrial times, and the correlation in the input flow.

Studying the Effects of Training Data on Machine Learning-Based Procedural Content Generation

The exploration of Procedural Content Generation via Machine Learning (PCGML) has been growing in recent years. However, while the number of PCGML techniques and methods for evaluating PCG techniques have been increasing, little work has been done in determining how the quality and quantity of the training data provided to these techniques effects the models or the output. Therefore, little is known about how much training data would actually be needed to deploy certain PCGML techniques in practice. In this paper we explore this question by studying the quality and diversity of the output of two well-known PCGML techniques (multi-dimensional Markov chains and Long Short-term Memory Recurrent Neural Networks) in generating Super Mario Bros. levels while varying the amount and quality of the training data.

Leveraging Multi-Layer Level Representations for Puzzle-Platformer Level Generation

Procedural content generation via machine learning (PCGML) has been growing in recent years. However, many PCGML approaches are only explored in the context of linear platforming games, and focused on modeling structural level information. Previously, we developed a multi-layer level representation, where each layer is designed to capture specific level information. In this paper, we apply our multi-layer approach to Lode Runner, a game with non-linear paths and complex actions. We test our approach by generating levels for Lode Runner with a constrained multi-dimensional Markov chain (MdMC) approach that ensures playability and a standard MdMC sampling approach. We compare the levels sampled when using multi-layer representation against those sampled using the single-layer representation; we compare using both the constrained sampling algorithm and the standard sampling algorithm.