Switching-POMDP based admission control policies for service systems with distributed architecture

Abstract

Many network systems with distributed structure today such as streaming media systems and resource-sharing systems can be modeled as the distributed network service system with multiple service nodes. Admission control technology is an essential way to enhance such systems. Model-based optimization approach as Markov decision process (MDP) is a good way to be applied to analyze and compute the optimal admission control policy that maximizes performance of system. However, due to the “curse of dimensionality”, computing such optimal admission control policy for practical distributed systems is a rather difficult task. Therefore, we describe the admission control process of the distributed network service system as a switching partially observable Markov decision process (SPOMDP) with two-level structure. The upper level decides whether to switch the operation mode of system, and the lower level decides to admit or block new service requests. According to the partially observable Markov decision process (POMDP) model, a distributed admission control algorithm is presented which the service nodes in system make decisions without the knowledge of other service nodes. The randomized policy is applied to optimize system performance, and the policy-gradient iteration algorithm is used to compute the optimal admission control policy. Then, an operation mode switching mechanism is presented to detect the change of system and determine the switch epoch of operation mode. Through the numerical experiments, we demonstrate the efficiency of the presented approach.