Service Rate Control Research Articles

Discounted cost exponential semi-Markov decision processes with unbounded transition rates: a service rate control problem with impatient customers

Discounted cost exponential semi-Markov decision processes with unbounded transition rates: a service rate control problem with impatient customers

Optimal Control of Service Rates of Discrete-Time (s, Q) Queueing-Inventory Systems with Finite Buffer

Optimal Control of Service Rates of Discrete-Time (s, Q) Queueing-Inventory Systems with Finite Buffer

Adaptive service rate control of an M/M/1 queue with server breakdowns

Adaptive service rate control of an M/M/1 queue with server breakdowns

Event-based optimization of service rate control in retrial queues

This paper deals with the problem of service rate control for a retrial queue where the controller can observe only the number of total customers in the system. Service rates are adjustable based on the partial information (the arrival or departure event) instead of the perfect system state (customer distribution among orbit and server). The goal is to find the optimal service rates that minimize the long-run average cost. This problem is formulated as an event-based optimization instead of a state-based optimization. Using the sensitivity-based optimization theory, we obtain interesting structures of the optimal service rate control policy, which show that the optimal policy is a bang-bang control and even has a threshold form under some mild conditions. The necessary and sufficient condition of optimal policies is also derived. Furthermore, by the difference formula of the system performances under any two policies, we develop a policy iteration-type algorithm to find the optimal policy for the case with general cost functions. With different initial values of service rates, our algorithm is demonstrated to efficiently find the optimal service rates. Moreover, we study the difference of system performances from the corresponding state-based optimization by numerical experiments, which indicates the managerial insights about the value of information for decision-makers.

Queue-Learning: A Reinforcement Learning Approach for Providing Quality of Service

End-to-end delay is a critical attribute of quality of service (QoS) in application domains such as cloud computing and computer networks. This metric is particularly important in tandem service systems, where the end-to-end service is provided through a chain of services. Service-rate control is a common mechanism for providing QoS guarantees in service systems. In this paper, we introduce a reinforcement learning-based (RL-based) service-rate controller that provides probabilistic upper-bounds on the end-to-end delay of the system, while preventing the overuse of service resources. In order to have a general framework, we use queueing theory to model the service systems. However, we adopt an RL-based approach to avoid the limitations of queueing-theoretic methods. In particular, we use Deep Deterministic Policy Gradient (DDPG) to learn the service rates (action) as a function of the queue lengths (state) in tandem service systems. In contrast to existing RL-based methods that quantify their performance by the achieved overall reward, which could be hard to interpret or even misleading, our proposed controller provides explicit probabilistic guarantees on the end-to-end delay of the system. The evaluations are presented for a tandem queueing system with non-exponential inter-arrival and service times, the results of which validate our controller's capability in meeting QoS constraints.

Fair Scheduling and Rate Control for Service Function Chain in NFV Enabled Data Center

With the emerging paradigm of Virtual Network and Network Function Virtualization (NFV), the data center operator can flexibly manage the network to reduce Operating Expenditures (OPEX) and Capital Expenditures (CAPEX). Some new issues including the assignment of Virtual Network Functions (VNFs) and Service Function Chain (SFC) scheduling should be considered to apply in the communication. Mathematically, the SFC scheduling process can be formulated as a queuing system, and every server installed with VNF instances can be regarded as a service node, where batched requests are parallel submitted and processed one after another. In the management of SFC, the cloud service provider needs to resolve the problem of network congestion. Latency and throughput are two important but contradictory indexes in the network management, and both of them are deserved to be optimized during the scheduling of SFC requests. In this paper, different from most existing studies, we focus on the service rate control problem in the scheduling of SFC requests. Firstly, we formulate this problem as an integer programming problem. Through a cooperative game approach, a Nash bargaining based model is proposed to jointly optimize the latency and throughput, which is proven to provide fair performance guarantees by both theoretical analysis and simulation. To improve the scalability of the proposed algorithm, we also design a polynomial two-phase heuristic to perform Pareto optimization. Simulation evaluation shows that the proposed algorithm can implement balanced traffic scheduling and avoid excessive server latency caused by network congestion.

Dynamic pricing in a two-class queueing system with arrival and service rate control

Abstract Dynamic pricing in a two-class queueing system with adjustable arrival and service rates is considered. We initially take adjustable rates into account to maximize the long-run average social welfare and further establish matched dynamic prices. For the rate-setting problem, we apply sensitivity-based optimization theory and an iterative algorithm to investigate the optimal arrival and service rates of the two classes of customers. Next, we apply the results obtained from the rate-setting problem to find the expected delay time using a recursive algorithm and explicitly determine formulae for the optimal prices of the two classes of customers. Finally, we carry out numerical experiments to illustrate our findings and the performance between two classes of customers at different levels of the holding cost. It appears that for a low holding cost, the optimal prices for the two classes of customers are monotonically increasing in the number of customers regardless of the class, but for a high holding cost, the optimal prices for the customers who have low waiting cost may drop when the number of the other class rises.

Service Rate and Admission Control in an M/G/1 Queue with State-Dependent Arrival Rates

Admission and service rate control problems in queueing systems have been studied in the literature. For an exponential service time distribution, the optimality of the threshold-type policy has been proved. However, in production systems, the production time follows a general distribution, not an exponential one. In this paper, control of the service speed according to the number of customers is considered. The analytical results of an M/G/1 queue with arrival and service rates that depend on the number of customers in the system, which is called an Mn/Gn/1 queue, are used to compute the performance measure of service rate control. In particular, for the case in which the arrival rates are the same among queue-length intervals, a computation method for deriving stationary distributions is developed. Constant, uniform, exponential, and Bernoulli distributions on the service time are considered via numerical experiments. The results show that the optimal threshold depends on the type of distribution, even if the mean value of the service time is the same. In addition, when the reward rate is small, a case in which a non-threshold-type service rate control policy outperforms all threshold-type policies is identified.

The Effect of Flexible Service Rates in Appointment Scheduling Systems

Recent research offers overwhelming evidence that service workers, including healthcare providers in ambulatory care settings, exercise flexible service rates. While theoretical models of service rate control in steady-state queues explain service rate changes in response to queue length, there are no comparable models explaining the drivers of optimal service rate in appointment systems. In this paper, we analyze service rate control when the provider is responsible for serving a finite number of patients who arrive by prior appointments in a session of finite duration. Under general assumptions on random service times, we show that a provider should optimally increase the service rate in response to the queue length and the delay in starting a patient's service. In addition to the novel analytical finding on delay-driven speed-up behavior, we show that both congestion-driven and delay-driven speed-up behaviors persist even when the patient wait is costless. We also show that the provider may speed-up or slow-down while serving sequential arrivals in the appointment schedule, depending on the relative magnitudes of the cost of patient waiting and the cost of the provider's time. Finally, we explore implications of flexible service rates on the operational performance of appointment systems, and find that flexible service rates result in shorter, fairer, and less uncertain waiting times for patients in most situations.

Stabilizing the virtual response time in single-server processor sharing queues with slowly time-varying arrival rates

Motivated by the work of Whitt, who studied stabilization of the mean virtual waiting time (excluding service time) in a $GI_t/GI_t/1/FCFS$ queue, this paper investigates the stabilization of the mean virtual response time in a single-server processor sharing (PS) queueing system with a time-varying arrival rate and a service rate control (a $GI_t/GI_t/1/PS$ queue). We propose and compare a modified square-root (SR) control and a difference-matching (DM) control to stabilize the mean virtual response time of a $GI_t/GI_t/1/PS$ queue. Extensive simulation studies with various settings of arrival processes and service times show that the DM control outperforms the SR control for heavy-traffic conditions, and that the SR control performs better for light-traffic conditions.

Minimizing the Maximum Expected Waiting Time in a Periodic Single-Server Queue with a Service-Rate Control

We consider a single-server queue with unlimited waiting space, the first-come, first-served discipline, a periodic arrival-rate function, and independent and identically distributed service requirements, where the service-rate function is subject to control. We previously showed that a rate-matching control, whereby the service rate is made proportional to the arrival rate, stabilizes the queue-length process but not the (virtual) waiting-time process. To minimize the maximum expected waiting time (and stabilize the expected waiting time), we now consider a modification of the service-rate control involving two parameters: a time lag and a damping factor. We develop an efficient simulation search algorithm to find the best time lag and damping factor. That simulation algorithm is an extension of our recent rare-event simulation algorithm for the GIt/GI/1 queue to the GIt/GIt/1 queue, allowing the time-varying service rate. To gain insight into these controls, we establish a heavy-traffic limit with periodicity in the fluid scale. This produces a diffusion control problem for the stabilization, which we solve numerically by the simulation search in the scaled family of systems with ρ ↑ 1. The state space collapse in that theorem shows that there is a time-varying Little’s law in heavy traffic, implying that the queue length and waiting time cannot be simultaneously stabilized in this limit. We conduct simulation experiments showing that the new control is effective for stabilizing the expected waiting time for a wide range of model parameters, but we also show that it cannot stabilize the expected waiting time perfectly.

OPTIMAL CONTROL POLICIES FOR AN M/M/1 QUEUE WITH A REMOVABLE SERVER AND DYNAMIC SERVICE RATES

AbstractWe consider an M/M/1 queue with a removable server that dynamically chooses its service rate from a set of finitely many rates. If the server is off, the system must warm up for a random, exponentially distributed amount of time, before it can begin processing jobs. We show under the average cost criterion, that work conserving policies are optimal. We then demonstrate the optimal policy can be characterized by a threshold for turning on the server and the optimal service rate increases monotonically with the number in system. Finally, we present some numerical experiments to provide insights into the practicality of having both a removable server and service rate control.

Dynamic Pricing Control for Open Queueing Networks

Pricing control is an important problem in service systems and it aims to control customer behaviors through an economic way, instead of administrative commands. In this paper, we study a dynamic pricing and service rate control problem in an open Jackson network with limited capacity. The goal is to determine the optimal admission prices and the optimal service rates at every state such that the long-run average social welfare is maximized. The original problem is decomposed into a rate-setting problem plus a price-setting problem. To solve the rate-setting problem, we derive a difference formula based on the sensitivity-based optimization theory. When the cost rate function is convex in service rates and the value rate function is concave in arrival rates, we decompose the rate-setting problem into a series of convex optimization subproblems. When the rate functions have linear structure, these subproblems are even simpler and a bang–bang control is optimal. For the price-setting problem, we determine the state-dependent prices so as to induce the optimal arrival rates obtained by the rate-setting problem. We propose a recursive algorithm to numerically compute the conditional expected delays at every state. Finally, we conduct numerical experiments to explore the optimality properties and some useful insights for this dynamic pricing control problem.

Energy-Aware Multilevel Control System for a Network of Linux Software Routers: Design and Implementation

The Linux operating system provides many well-developed tools that support the concept of energy-efficient networking. This paper outlines the results of research focused on the design and implementation of a control system reducing power consumption of IP-traffic processing in a network of Linux-based software routers. It is demonstrated how the standard ACPI-compliant system components can be adjusted to meet the requirements of adaptive energy-aware network control and what performance tradeoffs may there be expected. In particular, it is demonstrated how the abstraction layers provided by the Linux kernel can be used to exploit energy-saving mechanisms of packet processing servers. Formulations of the routing optimization and service rate control problems are presented and discussed. The results of the extensive experimental studies are presented as well.

Optimal Control of Service Rate in a Service Facility System Maintenance and Retrial Demands

This paper, deals with a problem of optimally controlling service rates for a retrial service facility system. We consider a finite capacity service facility system with Poisson arrivals and exponentially distributed service times. Here, the customer who sees the server busy, joins the orbit and reattempts the facility with exponential distributed time interval. The system is formulated as Markov Decision Process and we find the optimum service rates to be employed at each instant of time. Linear Programming method is implemented in the context of minimizing the long-run expected cost rate.

Cross-layer Dynamic Subcarrier Allocation with Adaptive Service Rate Control in SC-FDMA System

In this paper, an improved utility-based cross-layer dynamic subcarrier allocation (DSA) algorithm is proposed for single carrier frequency division multiple access (SC-FDMA) system, which adopts adaptive service rate control (ASRC) to eliminate the service rate waste and improve the spectral efficiency in heterogeneous network including non-real-time traffic and real-time traffic. In this algorithm, furthermore, a first in first out (FIFO) queuing model with finite space is established on the cross-layer scheduling framework. Simulation results indicate that by taking the service rate constraint as the necessary condition for optimality, the ASRC algorithm can effectively eliminate the service rate waste without compromising the scheduling performance. Moreover, the ASRC algorithm is able to further improve the quality of service (QoS) performance and transmission throughput by contributing an attractive performance trade-off between real-time and non-real-time applications.

Optimal Control of State-Dependent Service Rates in a MAP/M/1 Queue

In this paper, we study the optimal control of service rates in a queueing system with a Markovian arrival process (MAP) and exponential service times. The service rate is allowed to be state dependent, i.e., we can adjust the service rate according to the queue length and the phase of the MAP. The cost function consists of holding cost and operating cost. The goal is to find the optimal service rates that minimize the long-run average total cost. To achieve that, we use the matrix-analytic methods (MAM) together with the sensitivity-based optimization (SBO) theory. A performance difference formula is derived, which can quantify the difference of the long-run average total cost under any two different settings of service rates. Based on the difference formula, we show that the long-run average total cost is monotone in the service rate and the optimal control is a bang–bang control. We also show that, under some mild conditions, the optimal control policy of service rates is of a quasi-threshold-type. By utilizing the MAM theory, we propose a recursive algorithm to compute the value function related quantities. An iterative algorithm to efficiently find the optimal policy, which is similar to policy iteration, is proposed based on the SBO theory. We further study some special cases of the problem, such as the optimality of the threshold-type policy for the M/M/1 queue. Finally, a number of numerical examples are presented to demonstrate the main results and explore the impact of the phase of the MAP on the optimization in the MAP/M/1 queue.

Service Rate Control of Tandem Queues With Power Constraints

In this paper, we study the optimal control of service rates of a tandem queue with power constraints. The service rate of a server is determined by the power allocated to that server. The total power of the system is fixed. The system cost is comprised of two parts, the holding cost reflecting the congestion of queues and the operating cost reflecting the power consumed at servers. The optimization objective is to find the optimal power allocation policy among servers, which can minimize the system average cost. We formulate this problem as a Markov decision process with a constrained action space. Sensitivity-based optimization theory is applied to study this problem. The necessary and sufficient condition of optimal service rates, and the optimality of the vertexes of the feasible domain are derived when the operating cost has a linear or concave form. An iterative algorithm is further developed to find the optimal service rates. This algorithm may work well even when the cost function has a general form. The extension to general tandem queues with many servers is also studied. Finally, we conduct numerical experiments under different parameter settings to demonstrate the main idea of this paper.

Stochastic modeling of a high-performance cluster with hysteretic control of service rate

Stochastic modeling of a high-performance cluster with hysteretic control of service rate

An approximating diffusion control problem for dynamic admission and service rate control in a [formula omitted] queue

We study a diffusion control problem that is motivated by the dynamic admission and service rate control problem for a G∕M∕N+G queue. The objective is to minimize long run average cost. Because the original queueing control problem is not tractable, we solve the approximating diffusion control problem that arises under the QED heavy traffic regime and show that its optimal solution has two components: (1) a threshold control that regulates the diffusion and (2) a feedback-type drift rate control.