Stochastic Approximations and Adaptive Control of a Discrete-Time Single-Server Network with Random Routing

Abstract

This paper considers a discrete-time system composed of K infinite capacity queues that compete for the use of a single server. Customers arrive in independent and identically distributed (i.i.d.) batches and are served according to a server allocation policy. Upon completing service, customers either leave the system or are routed instantaneously to another queue according to some random mechanism. As an alternative to simply randomized strategies, a policy based on a stochastic approximation algorithm is proposed to drive a long-run average cost to a given value. The underlying motivation can be traced back to implementation issues associated with constrained optimal strategies.A version of the ordinary differential equation (ODE) method as given by Metivier and Priouret is developed for proving almost sure convergence of this algorithm. This is done by exploiting the recurrence structure of the system under nonidling policies. A probabilistic representation of solutions to an associated Poisson equatio...