Product Form Solution Research Articles

Performance Modeling of Distributed Data Processing in Microservice Applications

Microservice applications are increasingly adopted in distributed data processing systems, such as in mobile edge computing and data mesh architectures. However, existing performance models of such systems fall short in providing comprehensive insights into the intricate interplay between data placement and data processing. To address these issues, this paper proposes a novel class of performance models that enables joint analysis of data storage access workflows, caching, and queueing contention. Our proposed models introduce a notion of access path for data items to model hierarchical data locality constraints. We develop analytical solutions to efficiently approximate the performance metrics of these models under different data caching policies, finding in particular conditions under which the underlying Markov chain admits a product-form solution. Extensive trace-driven simulations based on real-world datasets indicate that service and data placement policies based on our proposed models can respectively improve by up to 35% and 37% the average response time in edge and data mesh case studies compared to baseline resource allocation heuristics.

New directions in pass-and-swap queues

Recently, driven by redundancy systems and matching systems, there has been renewed interest in models with product form stationary distributions. By a “product form,” we mean that the stationary distribution can be expressed as a product of terms, each of which corresponds to a job in the system. Given the recent discovery of many such systems, it is natural to ask: how broad is this class of systems? In this paper, we consider extensions and generalizations of the recently-proposed pass-and-swap queue, which has a product-form stationary distribution. We make three main contributions. First, we identify sufficient conditions under which pass-and-swap queues can be connected in a closed network, while still preserving the product form. Second, we identify dimensions along which the pass-and-swap system can be extended while preserving the product-form stationary distribution. At the same time, we also identify cases in which generalizing the pass-and-swap queue causes the product-form nature of the stationary distribution to break. Finally, we identify questions that remain open and present a road map for future study.

Dynamic load balancing in energy packet networks

Energy Packet Networks (EPNs) model the interaction between renewable sources generating energy following a random process and communication devices that consume energy. This network is formed by cells and, in each cell, there is a queue that handles energy packets and another queue that handles data packets. We assume Poisson arrivals of energy packets and of data packets to all the cells and exponential service times. We consider an EPN model with a dynamic load balancing where a cell without data packets can poll other cells to migrate jobs. This migration can only take place when there is enough energy in both interacting cells, in which case a batch of data packets is transferred and the required energy is consumed (i.e. it disappears). We consider that data packet also consume energy to be routed to the next station. Our main result shows that the steady-state distribution of jobs in the queues admits a product form solution provided that a stable solution of a fixed point equation exists. We prove sufficient conditions for irreducibility. Under these conditions and when the fixed point equation has a solution, the Markov chain is ergodic. We also provide sufficient conditions for the existence of a solution of the fixed point equation. We then focus on layered networks and we study the polling rates that must be set to achieve a fair load balancing, i.e., such that, in the same layer, the load of the queues handling data packets is the same. Our numerical experiments illustrate that dynamic load balancing satisfies several interesting properties such as performance improvement or fair load balancing.

Product Form Solution of a Queuing-Inventory System with Lost Sales and Server Vacation

Product Form Solution of a Queuing-Inventory System with Lost Sales and Server Vacation

A Product-form Network for Systems with Job Stealing Policies

In queueing networks, product-form solutions are of fundamental importance to efficiently compute performance metrics in complex models of computer systems. The product-form property entails that the steady-state probabilities of the joint stochastic process underlying the network can be expressed as the normalized product of functions that only depend on the local state of the components. In many relevant cases, product-forms are the only way to perform exact quantitative analyses of large systems. In this work, we introduce a novel class of product-form queueing networks where servers are always busy. Applications include model of systems where successive refinements on jobs improve the processes quality but are not strictly required to obtain a result. To this aim, we define a job movement policy that admits instantaneous migrations of jobs from non-empty waiting buffers to empty ones. Thus, the resulting routing scheme is state-dependent. This class of networks maximizes the system throughput. This model can be implemented with arbitrary topology, including feedback, and both in an open and closed setting. As far as closed systems are concerned, we give a convolution algorithm and the corresponding mean value analysis to compute expected performance indices for closed models.

Product form solution for a queueing-inventory system with lost sales and phase type distribution of the vacation time

ABSTRACT In this paper, a continuous review s , S queueing-inventory system with lost sales is investigated. Customers arrive according to a Poisson process, and each arriving customer needs exactly one item from inventory for service. The server takes a single vacation once the inventory is depleted. It is assumed that the vacation time follows a phase type distribution, and the service time and the lead time follow exponential distributions. We obtain the product form solution for the steady-state distribution of the joint process of the queue length, the on-hand inventory level and the server’s status. Furthermore, the conditional distributions of the on-hand inventory level when the server is turned off due to a vacation and when it is turned on are derived, respectively. Some performance measures of the system are computed by using the steady-state distribution of system. Based on some performance measures, we develop a cost model and determine a total expected cost under an appropriate cost structure by using the genetic algorithm. The effects of various parameters on the optimal inventory policy and the optimal expected cost are investigated by numerical examples.

Migration–contagion processes

AbstractConsider the following migration process based on a closed network of N queues with $K_N$ customers. Each station is a $\cdot$ /M/ $\infty$ queue with service (or migration) rate $\mu$ . Upon departure, a customer is routed independently and uniformly at random to another station. In addition to migration, these customers are subject to a susceptible–infected–susceptible (SIS) dynamics. That is, customers are in one of two states: I for infected, or S for susceptible. Customers can swap their state either from I to S or from S to I only in stations. More precisely, at any station, each susceptible customer becomes infected with the instantaneous rate $\alpha Y$ if there are Y infected customers in the station, whereas each infected customer recovers and becomes susceptible with rate $\beta$ . We let N tend to infinity and assume that $\lim_{N\to \infty} K_N/N= \eta $ , where $\eta$ is a positive constant representing the customer density. The main problem of interest concerns the set of parameters of such a system for which there exists a stationary regime where the epidemic survives in the limiting system. The latter limit will be referred to as the thermodynamic limit. We use coupling and stochastic monotonicity arguments to establish key properties of the associated Markov processes, which in turn allow us to give the structure of the phase transition diagram of this thermodynamic limit with respect to $\eta$ . The analysis of the Kolmogorov equations of this SIS model reduces to that of a wave-type PDE for which we have found no explicit solution. This plain SIS model is one among several companion stochastic processes that exhibit both random migration and contagion. Two of them are discussed in the present paper as they provide variants to the plain SIS model as well as some bounds and approximations. These two variants are the departure-on-change-of-state (DOCS) model and the averaged-infection-rate (AIR) model, which both admit closed-form solutions. The AIR system is a classical mean-field model where the infection mechanism based on the actual population of infected customers is replaced by a mechanism based on some empirical average of the number of infected customers in all stations. The latter admits a product-form solution. DOCS features accelerated migration in that each change of SIS state implies an immediate departure. This model leads to another wave-type PDE that admits a closed-form solution. In this text, the main focus is on the closed stochastic networks and their limits. The open systems consisting of a single station with Poisson input are instrumental in the analysis of the thermodynamic limits and are also of independent interest. This class of SIS dynamics has incarnations in virtually all queueing networks of the literature.

A product-form network for systems with job stealing policies

In this paper, we introduce a new product-form queueing network model where servers are always busy. This is obtained by defining a job movement policy that admits instantaneous migrations of jobs from non-empty waiting buffers to empty ones. Thus, the resulting routing scheme is statedependent. This class of networks tends to maximize the system throughput and can be used to model situations where successive refinements on jobs improve the processes quality but are not strictly required to obtain a result.

Product form solutions of some queueing inventory problems in random environment

The public is often attracted by the offers given by the shopkeepers. These offers do make a crucial impact in the company's annual turnover. On a close watch it can be observed that these offers do change from time to time according to the seasonal variations. Here we investigate a single commodity Queueing-Inventory with 'offer' in random environment. The 'offer' can be anything given to the customers as incentives, who purchase the item. There are 'n' distinct environments. The offers in distinct environments are different. In some environments there may not be any offer. Here an environment dependent inventory control policy $ (s_i ,S) $ for $ 1\leq i \leq n $ is followed. Lead time, service and replenishment of the inventories are independent exponentially distributed random variables. It has been observed that when certain restrictions are imposed on the system, it admits asymptotic product form solution. Two such variants are discussed here. Long run behaviour of the system is analysed under the assumption of system stability. An optimization problem is constructed to compute the values of $ s_i $ and $ S $ that minimize the average system cost. The effect of environment in the optimum values of control variables, is also investigated.

Queueing Networks and Markov Chains Analysis with the Octave queueing package

Queueing networks and Markov chains are a widely used modeling notation that has been successfully applied to many kind of systems. In this paper we describe the queueing package, a free software package for product-form queueing networks and Markov chains analysis written in GNU Octave. The queueing package allows users to compute performance measures of Markov chains, single-station queueing systems and product- and some non product-form Queueing Network (QN) models. We present some practical examples showing how the queueing package can be used for reliability analysis, capacity planning and general systems modeling.

Multiservice Loss Models in C-RAN Supporting Compound Poisson Traffic

In this paper, a cloud radio access network (C-RAN) is considered where the baseband units form a pool of computational resource units (RUs) and are separated from the remote radio heads (RRHs). The RRHs are grouped into clusters based on their capacity in radio RUs. Each RRH serves different service-classes whose calls have different requirements in terms of radio and computational RUs and follow a compound Poisson process. This means that calls arrive in batches while each batch of calls follows a Poisson process. If the RUs’ requirements of an arriving call are met, then the call is accepted in the serving RRH for an exponentially distributed service time. Otherwise, call blocking occurs. We initially start our analysis with a single-cluster C-RAN and model it as a multiservice loss system, prove that the model has a product form solution, and determine time and call congestion probabilities via a convolution algorithm. Furthermore, the previous model is extended to include the more complex case of many clusters of RRHs.

An Algebraic Approach to Product-form Stationary Distributions for Some Reaction Networks

Exact results for product-form stationary distributions of Markov chains are of interest in different fields. In stochastic reaction networks (CRNs), stationary distributions are mostly known in special cases where they are of product-form. However, there is no full characterization of the classes of networks whose stationary distributions have product-form. We develop an algebraic approach to product-form stationary distributions in the framework of CRNs. Under certain hypotheses on linearity and decomposition of the state space for conservative CRNs, this gives sufficient and necessary algebraic conditions for product-form stationary distributions. Correspondingly, we obtain a semialgebraic subset of the parameter space that captures rates where, under the corresponding hypotheses, CRNs have product-form. We employ the developed theory to CRNs and some models of statistical mechanics, besides sketching the pertinence in other models from applied probability.

A Probabilistic Approach to Growth Networks

Single-class closed queueing networks, consisting of infinite-server and single-server queues with exponential service times and probabilistic routing, admit product-from solutions. Such solutions, although seemingly tractable, are difficult to characterize explicitly for practically relevant problems due to the exponential combinatorial complexity of its normalization constant (partition function). In “A Probabilistic Approach to Growth Networks,” Jelenković, Kondev, Mohapatra, and Momčilović develop a novel methodology, based on a probabilistic representation of product-form solutions and large-deviations concentration inequalities, which identifies distinct operating regimes and yields explicit expressions for the marginal distributions of queue lengths. From a methodological perspective, a fundamental feature of the proposed approach is that it provides exact results for order-one probabilities, even though the analysis involves large-deviations rate functions, which characterize only vanishing probabilities on a logarithmic scale.

Multiclass Energy Packet Networks with finite capacity energy queues

Energy Packet Network (EPN) consists of a queueing network formed by N blocks, where each of them is formed by one data queue, that handles the workload, and one energy queue, that handles packets of energy.We study an EPN model where the energy packets start the transfer. In this model, energy packets are sent to the data queue of the same block. An energy packet routes one workload packet to the next block if the data queue is not empty, and it is lost otherwise.We assume that the energy queues have a finite buffer size and if an energy packet arrives to the system when the buffer is full, jump-over blocking (JOB) is performed, and therefore with some probability it is sent to the data queue and it is lost otherwise.We first provide a value of the jump-over blocking probability such that the steady-state probability distribution of packets in the queues admits a product form solution. The product form is established for multiserver and multiclass data packet queues under FCFS, preemptive LCFS and PS discipline. Moreover, in the case of a directed tree queueing network, we show that the number of data packets in each subtree decreases as the JOB probability increases for each block.

Multiservice Loss Models in Single or Multi-Cluster C-RAN Supporting Quasi-Random Traffic

In this paper, a cloud radio access network (C-RAN) is considered where the baseband units form a pool of computational resource units and are separated from the remote radio heads (RRHs). Based on their radio capacity, the RRHs may form one or many clusters: a single cluster when all RRHs have the same capacity and multi-clusters where RRHs of the same radio capacity are grouped in the same cluster. Each RRH services the so-called multiservice traffic, i.e., calls from many service classes with various radio and computational resource requirements. Calls arrive in the RRHs according to a quasi-random process. This means that new calls are generated by a finite number of mobile users. Arriving calls require simultaneously computational and radio resource units in order to be accepted in the system, i.e., in the serving RRH. If their requirements are met, then these calls are served in the (serving) RRH for a service time which is generally distributed. Otherwise, call blocking occurs. We start with the single-cluster C-RAN and model it as a multiservice loss system, prove that the model has a product form solution, and determine time congestion probabilities via a convolution algorithm whose accuracy is validated with the aid of simulation. Furthermore, the previous model is generalized to include the more complex case of more than one clusters.

Smart Policies for Multisource Inventory Systems and General Tandem Queues with Order Tracking and Expediting

We study an inventory system with multiple supply sources and expediting options. The replenishment lead times from each supply source are stochastic, representing congestion and disruption. We construct a family of smart ordering and expediting policies that utilize real-time supply information. Such dynamic policies are generally difficult to evaluate, because the corresponding supply system is a tandem queue with state-dependent arrivals and routing, whose queue-length steady-state distribution is usually not in product form. Our main result is to identify two appealing special cases of the general policy, Policy-M and Policy-E, which possess simple product-form solutions and lead to closed-form performance measures. Policy-M retains full sourcing flexibility, but ignores upstream congestion in making expediting decisions. Policy-E only orders from the normal, farthest source, but makes expediting decisions based on both upstream and downstream information. A numerical study shows that the best Policy-M leads to a lower average cost than the best Policy-E in almost all cases. Also, implementing the best Policy-M parameters, the general policy only performs slightly better than Policy-M. These observations reveal the value of combining sourcing flexibility with some, but limited, dynamic expediting. Our findings are equally applicable to the equivalent tandem queue. They thus may aid dynamic routing and expediting decisions for online retailers and logistics providers, among others.

Scaling limits for closed product-form queueing networks

We consider a general class of closed product-form queueing networks, consisting of single-server queues and infinite-server queues. Even if a network is of product-form type, performance evaluation tends to be difficult due to the potentially large state space and the dependence between the individual queues. To remedy this, we analyze the model in a Halfin–Whitt inspired scaling regime, where we jointly blow up the traffic loads of all queues and the number of customers in the network. This leads to a closed-form limiting stationary distribution, which provides intuition on the impact of the dependence between the queues on the network’s behavior. We assess the practical applicability of our results through a series of numerical experiments, which illustrate the convergence and show how the scaling parameters can be chosen to obtain accurate approximations.

Response Time Distribution in a Tandem Pair of Queues with Batch Processing

Response time density is obtained in a tandem pair of Markovian queues with both batch arrivals and batch departures. The method uses conditional forward and reversed node sojourn times and derives the Laplace transform of the response time probability density function in the case that batch sizes are finite. The result is derived by a generating function method that takes into account that the path is not overtake-free in the sense that the tagged task being tracked is affected by later arrivals at the second queue. A novel aspect of the method is that a vector of generating functions is solved for, rather than a single scalar-valued function, which requires investigation of the singularities of a certain matrix. A recurrence formula is derived to obtain arbitrary moments of response time by differentiation of the Laplace transform at the origin, and these can be computed rapidly by iteration. Numerical results for the first four moments of response time are displayed for some sample networks that have product-form solutions for their equilibrium queue length probabilities, along with the densities themselves by numerical inversion of the Laplace transform. Corresponding approximations are also obtained for (non-product-form) pairs of “raw” batch-queues—with no special arrivals—and validated against regenerative simulation, which indicates good accuracy. The methods are appropriate for modeling bursty internet and cloud traffic and a possible role in energy-saving is considered.

Optimal hyper-scalable load balancing with a strict queue limit

Load balancing plays a critical role in efficiently dispatching jobs in parallel-server systems such as cloud networks and data centers. A fundamental challenge in the design of load balancing algorithms is to achieve an optimal trade-off between delay performance and implementation overhead (e.g. communication or memory usage). This trade-off has primarily been studied so far from the angle of the amount of overhead required to achieve asymptotically optimal performance, particularly vanishing delay in large-scale systems. In contrast, in the present paper, we focus on an arbitrarily sparse communication budget, possibly well below the minimum requirement for vanishing delay, referred to as the hyper-scalable operating region. Furthermore, jobs may only be admitted when a specific limit on the queue position of the job can be guaranteed.The centerpiece of our analysis is a universal upper bound for the achievable throughput of any dispatcher-driven algorithm for a given communication budget and queue limit. We also propose a specific hyper-scalable scheme which can operate at any given message rate and enforce any given queue limit, while allowing the server states to be captured via a closed product-form network, in which servers act as customers traversing various nodes. The product-form distribution is leveraged to prove that the bound is tight and that the proposed hyper-scalable scheme is throughput-optimal in a many-server regime given the communication and queue limit constraints. Extensive simulation experiments are conducted to illustrate the results.

Pass-and-swap queues

Order-independent (OI) queues, introduced by Berezner et al. (Queueing Syst 19(4):345–359, 1995), expanded the family of multi-class queues that are known to have a product-form stationary distribution by allowing for intricate class-dependent service rates. This paper further broadens this family by introducing pass-and-swap (P&S) queues, an extension of OI queues where, upon a service completion, the customer that completes service is not necessarily the one that leaves the system. More precisely, we supplement the OI queue model with an undirected graph on the customer classes, which we call a swapping graph, such that there is an edge between two classes if customers of these classes can be swapped with one another. When a customer completes service, it passes over customers in the remainder of the queue until it finds a customer it can swap positions with, that is, a customer whose class is a neighbor in the graph. In its turn, the customer that is ejected from its position takes the position of the next customer it can be swapped with, and so on. This is repeated until a customer can no longer find another customer to be swapped with; this customer is the one that leaves the queue. After proving that P&S queues have a product-form stationary distribution, we derive a necessary and sufficient stability condition for (open networks of) P&S queues that also applies to OI queues. We then study irreducibility properties of closed networks of P&S queues and derive the corresponding product-form stationary distribution. Lastly, we demonstrate that closed networks of P&S queues can be applied to describe the dynamics of new and existing load-distribution and scheduling protocols in clusters of machines in which jobs have assignment constraints.