Optimal Admission Control Policy Research Articles

Admission Control in Multi-server Systems Under Binary Reward Structure

We study a multi-server queueing system where a customer is satisfied (and generates a unit revenue) if their queueing time is at most a given constant. If the queueing time of the admitted customer exceeds this constant, the customer gets served, but is unsatisfied and generates no revenue. Such queueing systems arise in the context of modeling service systems where excessive delays are of concern. A key challenge is how to design an admission control policy to maximize the number of satisfied customers per unit time in the long run, assuming that we can observe the number of customers in the system at any time. We call this the binary reward structure system and show that a threshold-type admission policy is optimal. The optimal threshold policy has to be computed numerically. Hence we propose a square-root admission policy to approximate the optimal admission control policy, and compare the performance of these two policies. We derive an analytical upper bound on the performance of optimal admission control policy by deriving an optimal admission policy assuming we have full information over the queueing time of the admitted customers. This is equivalent to a queueing system where customers abandon the queue (i.e., leave without service) if their queueing time exceeds the given constant. We demonstrate that the optimal policy that includes customer abandonment, or alternatively, the optimal policy under full information, the optimal threshold policy, and the square-root admission policy, all exhibit identical performance in the asymptotic regions of the parameter space. Our numerical results indicate that the worst optimality gap of the square-root admission policy is within 3.9% of the optimal revenue, and implementing the square-root admission policy in the observable queueing system leads to a revenue loss that is at most 5.6% of the maximum possible revenue rate in the full information system. We also compare the binary reward structure with the more common linear reward structure where the system incurs holding cost per unit queueing time per customer. In addition, we also show that the analysis based on queueing time is applicable to the system time as well.

Structure-Aware Reinforcement Learning for Node-Overload Protection in Mobile Edge Computing

Mobile Edge Computing (MEC) involves placing computational capability and applications at the edge of the network, providing benefits such as reduced latency, reduced network congestion, and improved performance of applications. The performance and reliability of MEC degrades significantly when the edge server(s) in the cluster are overloaded. In this work, an adaptive admission control policy to prevent edge node from getting overloaded is presented. This approach is based on a recently-proposed low complexity RL (Reinforcement Learning) algorithm called SALMUT (Structure-Aware Learning for Multiple Thresholds), which exploits the structure of the optimal admission control policy in multi-class queues for an average-cost setting. We extend the framework to work for node overload-protection problem in a discounted-cost setting. The proposed solution is validated using several scenarios mimicking real-world deployments in two different settings — computer simulations and a docker testbed. Our empirical evaluations show that the total discounted cost incurred by SALMUT is similar to state-of-the-art deep RL algorithms such as PPO (Proximal Policy Optimization) and A2C (Advantage Actor Critic) but requires an order of magnitude less time to train, outputs easily interpretable policy, and can be deployed in an online manner.

Admission Control Policies in Loss Networks

This article addresses the admission control of a loss queueing network, or shortly loss network, of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$N$ </tex-math></inline-formula> parallel multiserver stations with no waiting rooms. <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$M$ </tex-math></inline-formula> classes of customers arrive at a Poisson rate and require an exponential service time with common mean. A customer can be served by any free server of a station in the set of stations determined by the class of the customer. Admission of a customer to a station brings different rewards that depend on customer class and the preference order of that station. Our objective is to find the optimal admission control policy that maximizes the average reward. We adopt a research strategy that evolves from simpler networks to more complicated ones. First, we consider a one-station <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$M$ </tex-math></inline-formula> -class loss network whose results form a fundamental basis for the analysis of more complicated networks. Second, we consider a two-station <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$M$ </tex-math></inline-formula> -class loss network for which we establish the existence of an optimal threshold admission policy. Finally, we consider a general <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$N$ </tex-math></inline-formula> -station <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$M$ </tex-math></inline-formula> -class loss network and propose an iterative approximation policy (IAP) and a mixed-integer linear programming model that is proven to compute an upper bound. We demonstrate the near-optimal performance of the proposed IAP and the tightness of the upper bound on several numerical instances motivated by real-life networks, such as healthcare emergency service networks and emergency call centers. Note to Practitioners—This article is mainly motivated by real-time scheduling of emergency service networks, such as emergency healthcare and telecommunication networks. In such networks, emergent demands need to be served promptly and are lost otherwise. They can be served by multiple hospitals/operator groups but have their own preferences. The decision maker has to decide dynamically whether to keep the capacity of a hospital/operator for its own customers to meet customer preference or serve the incoming demand to improve the resource utilization. We set the problem as the admission control of a multiclass multistation loss network, study the properties of the optimal policy, and propose an iterative approximation policy that is proven to be near-optimal for large-size loss networks. The proposed policy is found to improve by up to 16% and 52% the no-control policy admitting the customer to the most preferred available station and the no-overflow policy with admission to only the most preferred station, respectively. Managerial insights and extensions are also discussed.

Optimal admission control under premature discharge decisions for operational effectiveness

AbstractThe shortage of inpatient beds causes surgery cancellations, treatment delays, and inappropriate ward arrangements. That is why studying how to make admission decisions on the wards of crowded units under premature discharge decisions is essential. In this study, we set up an infinite‐horizon total discounted cost Markov decision process model to minimize the total expected cost to the hospital. By analyzing the structural properties of the optimal value function, we find that there is an optimal admission control policy of the threshold type for low‐acuity emergency patients (type‐2). Numerical experiments are done to examine how optimal dynamic policy depends on key system parameters. Furthermore, we evaluate the performance of the optimal dynamic policy with a benchmark policy without premature discharge decisions. The results indicate that the optimal admission control policy helps hospitals in maintaining low costs and enables patients with high‐acuity diseases to enter primary wards more quickly.

On the optimality of a maintenance queueing system

We consider a finite single-server maintenance queue with multiple types of customers. The difference between customers' types is defined by the offered rewards. We show that the optimal admission control policy for maximizing the long-run average reward per unit time has a trunk reservation structure. Meanwhile, if the equipment is off, there exists a threshold of the queue length, above which the optimal repair speed is increasing in the queue length and below which the optimal repair speed is 0.

Modeling and analysis of a cooperative service network

With the advances in technology and changes in customers’ attitude towards different service delivery formats, it is important for the service providers to deliver online services in addition to the traditional face-to-face services. In the cooperative service network presented in this study, service providers cooperate to serve online service requests received by the network in addition to their own customers. Designing and managing the cooperative network effectively increase the utilization of the involved servers, provide an adequate service for the external customers, and increase the profit for both the network and service providers. From the operational perspective, the number and utilization of the members to be included in the network and the price that will be paid to each member for a directed request are the main design questions. In order to answer these questions, we present a stochastic model that captures the dynamics of customer arrivals, assignment, and admission control. To establish this model, we first derive the solution of the dynamic admission control problem for the servers who decide how to admit their own customers and the external online customers using a Markov decision process. We then analyze the operation of the whole network with the servers who use the optimal admission control policy and obtain the system performance measures depending on the members’ operational parameters. These results are used to determine the optimal number of servers in the network and the service price to be paid to the participating servers in order to maximize the obtained profit. We show that a cooperative service network is an effective way of utilizing the idle capacity of the servers while providing an adequate service level for the external online customers and increasing the profit for both the network and service providers.

Joint admission, production sequencing, and production rate control for a two-class make-to-order manufacturing system

Today's manufacturing industry faces a number of challenges related to the rapid delivery of products with a high degree of variety. Striking a balance between the effectiveness in capacity utilization and the rapidness in order-fulfillment is a substantial challenge for manufacturing companies. This work aims to provide a theoretical basis from which to address this practical question. To this end, we address the problem of coordinating the admission, production sequencing, and production rate controls for a two-class make-to-order manufacturing system. Formulating the problem as a Markov decision process model, we identify the structural properties of optimal control policies under both discounted and average profit criteria. We show that the cμ rule is optimal for production sequencing and the optimal admission and production rate control policies can be characterized by the state-dependent threshold levels, provided that the production times are not associated with customer class. We also show that the optimal production rates are monotone in the system state, as in the case of a single class queueing system, and that the lower priority class can be preferred to the higher priority class in order admission under a certain condition on the system parameters. Our numerical study demonstrates that a considerable economic benefit can be achieved if the production rate is dynamically controlled between the minimum and maximum rates rather than fixed to the mean rate of these values. Finally, we present a heuristic policy that is described by linear switching functions for the control of order admission and a selection rule for the control of production rate. We compare the performance of our heuristic to the optimal policy using a numerical experiment, revealing that the heuristic provides near optimal solutions to test example problems and is robust to the system parameters.

Equilibrium points and equilibrium sets of some $$\textit{GI}/M/1$$ queues

Queues can be seen as a service facility where quality of service (QoS) is an important measure for the performance of the system. In many cases, the queue implements the optimal admission control (either discounted or average) policy in the presence of holding/congestion cost and revenue collected from admitted customers. In this paper, users offer an arrival rate at stationarity that depends on the QoS they experience. We study the interaction between arriving customers and such a queue under two different QoS measures—the asymptotic rate of the customers lost and the fraction of customers lost in the long run. In particular, we investigate the behaviour of equilibrium points and equilibrium sets associated with this interaction and their interpretations in terms of business cycles. We provide sufficient conditions for existence of equilibrium sets for M/M/1 queue. These conditions further help us to identify the relationship among system parameters for which equilibrium sets exist. Next, we consider $$\textit{GI}/M/1$$ queues and provide a sufficient condition for existence of multiple optimal revenue policies. We then specialize these results to study the equilibrium sets of (i) a D/M/1 queue and (ii) a queue where the arrival rate is locally continuous. The equilibrium behaviour in the latter case is more interesting as there may be multiple equilibrium points or sets. Motivated by such queues, we introduce a weaker version of monotonicity and investigate the existence of generalized equilibrium sets.

Optimal Admission Control Policy Based on Memetic Algorithm in Distributed Real Time Database System

Recently distributed real-time database systems are intended to manage large volumes of dispersed data. To develop distributed real-time data processing, a reality and stay competitive well defined protocols and algorithms must be required to access and manipulate the data. An admission control policy is a major task to access real-time data which has become a challenging task due to random arrival of user requests and transaction timing constraints. This paper proposes an optimal admission control policy based on deep reinforcement algorithm and memetic algorithm which can efficiently handle the load balancing problem without affecting the Quality of Service (QoS) parameters. A Markov decision process (MDP) is formulated for admission control problem, which provides an optimized solution for dynamic resource sharing. The possible solutions for MDP problem are obtained by using reinforcement learning and linear programming with an average reward. The deep reinforcement learning algorithm reformulates the arrived requests from different users and admits only the needed request, which improves the number of sessions of the system. Then we frame the load balancing problem as a dynamic and stochastic assignment problem and obtain optimal control policies using memetic algorithm. Therefore proposed admission control problem is changed to memetic logic in such a way that session corresponds to individual elements of the initial chromosome. The performance of proposed optimal admission control policy is compared with other approaches through simulation and it depicts that the proposed system outperforms the other techniques in terms of throughput, execution time and miss ratio which leads to better QoS.

An Optimal Flow Admission and Routing Control Policy for Resource Constrained Networks.

Overloaded network devices are becoming an increasing problem especially in resource limited networks with the continuous and rapid increase of wireless devices and the huge volume of data generated. Admission and routing control policy at a network device can be used to balance the goals of maximizing throughput and ensuring sufficient resources for high priority flows. In this paper we formulate the admission and routing control problem of two types of flows where one has a higher priority than the other as a Markov decision problem. We characterize the optimal admission and routing policy, and show that it is a state-dependent threshold type policy. Furthermore, we conduct extensive numerical experiments to gain more insight into the behavior of the optimal policy under different systems’ parameters. While dynamic programming can be used to solve such problems, the large size of the state space makes it untractable and too resource intensive to run on wireless devices. Therefore, we propose a fast heuristic that exploits the structure of the optimal policy. We empirically show that the heuristic performs very well with an average reward deviation of 1.4% from the optimal while being orders of magnitude faster than the optimal policy. We further generalize the heuristic for the general case of a system with n () types of flows.

An Optimal Dynamic Admission Control Policy and Upper Bound Analysis in Wireless Sensor Networks

Data admission control is an important issue in wireless sensor networks (WSNs) because of the limited transmission coverage area and the limited battery capability of each sensor node. To achieve the optimum benefit of the sensor network, a reward when a data package arriving at a sensor is accepted (not rejected) for transmission is considered, but a holding cost per unit time for the accepted data package waiting in the sensor for transmission is also incurred. For the sensor with sleep and active phases, a dynamic data admission control model is developed in this paper. By constructing a suitable Markov decision process (MDP), we verify that the optimal admission control policy of when to admit or reject an arriving data packet is a control limit policy, in order to achieve the maximum expected a discounted reward in the sleep and active phase, respectively. Furthermore, we provide a formula for how to calculate the upper bound of these threshold values. For the identified model with the optimal expected discount reward, the energy consumption of the sensors in active and sleep phases, as well as the energy consumption switching from active to sleep per unit time and vice versa is investigated. Extensive simulation is implemented. The results show that the problem is effectively solved by an optimal scheme with high energy efficiency. The results of this paper can be applied to designing optimal sensor nodes in wireless sensor networks.

Cognitive Radio Networks: Analysis of a Paid-Sharing Approach Based on Admission Control Decisions

Cognitive radio networks have emerged in the last decade as a solution for two problems: spectrum underutilization and spectrum scarcity. The main idea is to manage the radio spectrum more efficiently, where secondary users (SUs) are allowed to exploit the spectrum holes in primary user’s (PUs) frequency bands. We consider a paid-sharing approach where SUs pay for spectrum utilization. A challenging aspect in these mechanisms is how to proceed when a PU needs certain amount of bandwidth and the free capacity is insufficient. We assume a preemptive system where PUs have strict priority over SUs; when a PU arrives to the system and there are not enough free channels to accommodate the new user, one or more SUs will be deallocated. The affected SUs will then be reimbursed, implying some cost for the PUs service provider (SP). This paper bears on the design and analysis of an optimal SU admission control policy; i.e. that maximizes the long-run profit of the SP. We model the optimal revenue problem as a Markov Decision Process and we use dynamic programming and further techniques such as sample-path analysis to characterize properties of the optimal admission control policy. We introduce different changes to one of the best known dynamic programming algorithms incorporating the knowledge of the characterization. In particular, those proposals accelerate the rate of convergence of the algorithm when is applied in the considered context. Our results are validated through numerical examples.

Data-driven optimization methodology for admission control in critical care units.

The decision of whether to admit a patient to a critical care unit is a crucial operational problem that has significant influence on both hospital performance and patient outcomes. Hospitals currently lack a methodology to selectively admit patients to these units in a way that patient health risk metrics can be incorporated while considering the congestion that will occur. The hospital is modeled as a complex loss queueing network with a stochastic model of how long risk-stratified patients spend time in particular units and how they transition between units. A Mixed Integer Programming model approximates an optimal admission control policy for the network of units. While enforcing low levels of patient blocking, we optimize a monotonic dual-threshold admission policy. A hospital network including Intermediate Care Units (IMCs) and Intensive Care Units (ICUs) was considered for validation. The optimized model indicated a reduction in the risk levels required for admission, and weekly average admissions to ICUs and IMCs increased by 37% and 12%, respectively, with minimal blocking. Our methodology captures utilization and accessibility in a network model of care pathways while supporting the personalized allocation of scarce care resources to the neediest patients. The interesting benefits of admission thresholds that vary by day of week are studied.

Optimal Admission Control for Many-Server Systems with QED-Driven Revenues

We consider Markovian many-server systems with admission control operating in a Quality-and-Efficiency-Driven (QED) regime, where the relative utilization approaches unity while the number of servers grows large, providing natural Economies-of-Scale. In order to determine the optimal admission control policy, we adopt a revenue maximization framework, and suppose that the revenue rate attains a maximum when no customers are waiting and no servers are idling. When the revenue function scales properly with the system size, we show that a nondegenerate optimization problem arises in the limit. Detailed analysis demonstrates that the revenue is maximized by nontrivial policies that bar customers from entering when the queue length exceeds a certain threshold of the order of the typical square-root level variation in the system occupancy. We identify a fundamental equation characterizing the optimal threshold, which we extensively leverage to provide broadly applicable upper/lower bounds for the optimal threshold, establish its monotonicity, and examine its asymptotic behavior, all for general revenue structures. For linear and exponential revenue structures, we present explicit expressions for the optimal threshold.

Admission and inventory control of a single-component make-to-order production system with replenishment setup cost and lead time

We consider the admission control and inventory management problems of a single-component make-to-order production system. Components are purchased from suppliers in batches of fixed size subject to stochastic lead times and setup costs. A control policy specifies when a batch of components is purchased, and whether the demand for each MTO production is accepted upon arrival. We formulate the problem as a Markov decision process (MDP) model, and characterize the structure of optimal admission control and inventory replenishment policies. We show that a state dependent base-stock policy is optimal for the inventory replenishment, although the MDP value function is not necessarily convex. We also show that the optimal admission control can be identified as a lattice dependent policy. A sensitivity analysis is conducted to show how the optimal policy changes as a function of the system parameters. To effectively coordinate admission and inventory control decisions, we propose simple, implementable, and yet effective heuristic policies. Our extensive numerical results suggest that the proposed heuristics can greatly help firms to effectively coordinate their admission and inventory control activities.

An Optimal Joint Call Admission Control Policy in Heterogeneous Wireless Networks

In heterogeneous wireless networks (HWNs), a scheme called Joint Call Admission Control (JCAC) is demanding for deciding whether or not an incoming service request will be accepted. In this paper, we propose an optimal JCAC policy for HWNs based on semi-Markov decision process (SMDP) to achieve the optimal resource management scheme in terms of minimal the network cost and acquire the required quality of service (QoS) of mobile user. Furthermore, an improved value iteration algorithm with multi-dimensional threshold structure is presented. Numerical results show that the proposed JCAC policy is the overall optimal policy. The proposed algorithm is effective for high QoS performance by reducing the probabilities of dropping and blocking calls. The optimal JCAC policy is ratified by the current trend in the design of next generation wireless networks (NGWNs).

Who's Next? Scheduling Personalization Services with Variable Service Times

Online personalization has become quite prevalent in recent years, with firms able to derive additional profits from such services. As the adoption of such services grows, firms implementing such practices face some operational challenges. One important challenge lies in the complexity associated with the personalization process and how to deploy available resources to handle such complexity. The complexity is exacerbated when a site faces a large volume of requests in a short amount of time, as is often the case for e-commerce and content delivery sites. In such situations, it is generally not possible for a site to provide perfectly personalized service to all requests. Instead, a firm can provide differentiated service to requests based on the amount of profiling information available about the visitor. We consider a scenario where the revenue function is concave, capturing the diminishing returns from personalization effort. Using a batching approach, we determine the optimal scheduling policy (i.e., time allocation and sequence of service) for a batch that accounts for the externality cost incurred when a request is provided service before other waiting requests. The batching approach leads to sunk costs incurred when visitors wait for the next batch to begin. An optimal admission control policy is developed to prescreen new request arrivals. We show how the policy can be implemented efficiently when the revenue function is complex and there are a large number of requests that can be served in a batch. Numerical experiments show that the proposed approach leads to substantial improvements over a linear approximation of the concave revenue function. Interestingly, we find that the improvements in firm profits are not only (or primarily) due to the different service times that are obtained when using the nonlinear personalization function—there is a ripple effect on the admission control policy that incorporates these optimized service times, which contributes even more to the additional profits than the service time optimization by itself.

Dynamic admission control for two customer classes with stochastic demands and strict due dates

We study a dynamic capacity allocation problem with admission control decisions of a company that caters for two demand classes with random arrivals, capacity requirements and strict due dates. We formulate the problem as a Markov decision process (MDP) in order to find the optimal admission control policy that maximises the expected profit of the company. Such a formulation suffers a state-space explosion. Moreover, it involves an additional dimension arising from the multiple possible order sizes that customers can request which further increases the complexity of the problem. To reduce the cardinality of possible policies, and, thus, the computational requirements, we propose a threshold-based policy. We formulate an MDP to generate such a policy. To deal with the curse of dimensionality, we develop threshold-based approximate algorithms based on the state-reduction heuristics with aggregation proposed previously. Our results reveal that for the majority of instances considered the optimal policy has a threshold structure. We then demonstrate the superiority of the proposed threshold-based approximate algorithms over two benchmark policies in terms of the generated profits and the robustness of the solutions to changes in operational conditions. Finally, we show that our proposed policies are also robust to changes in actual demand from its estimation.

Utilization-based admission control for aperiodic tasks under EDF scheduling

Utilization-based admission control policies for processors executing aperiodic tasks are often favored due to their low overhead. This paper investigates utilization-based admission control for uniprocessor systems executing aperiodic tasks following the earliest deadline first priority assignment. We first propose a new constant time utilization-based admission control policy, CTAC. We prove that CTAC is safe and will admit any task instance that can be admitted by the best existing utilization-based admission control policy, ASL, given the same processor state. We also introduce an optimal utilization-based admission control policy, OPAC. It is proved that given the same processor state, a newly arriving task instance that can be admitted by any other safe utilization-based admission control policy can also be admitted by OPAC. We also show that OPAC cannot be implemented in constant time. Simulation results show that CTAC indeed outperforms ASL with constant time complexity and achieves performance close to OPAC in terms of various metrics.

Optimal admission control for two station tandem queues with loss

We consider a two-station tandem queue with a buffer size of one at the first station and a finite buffer size at the second station. Silva et al. (2013) gave a criterion determining the optimal admission control policy for this model. In this paper, we improve the results of Silva et al. (2013) and also solve the problem conjectured by Silva et al. (2013).