Universality of Power-of-d Load Balancing Schemes

Abstract

We consider a system of N parallel queues with unit exponential service rates and a single dispatcher where tasks arrive as a Poisson process of rate λ( N ). When a task arrives, the dispatcher assigns it to a server with the shortest queue among d ( N ) ≤ N randomly selected servers. This load balancing policy is referred to as a power-of- d ( N ) or JSQ( d ( N )) scheme, and subsumes the Join-the-Shortest Queue (JSQ) policy as a crucial special case for d ( N ) = N . We construct a coupling to bound the difference in the queue length processes between the JSQ policy and an arbitrary value of d ( N ). We use the coupling to derive the fluid limit in the regime where λ( N )/ N → λ < 1 and d ( N )→ ∞ as N → ∞, along with the corresponding fixed point. The fluid limit turns out not to depend on the exact growth rate of d ( N ), and in particular coincides with that for the JSQ policy. We further leverage the coupling to establish that the diffusion limit in the regime where ( N --λ( N ))/ √ N → β > 0 and d ( N )/ √ N log N → ∞ as N → ∞ corresponds to that for the JSQ policy. These results indicate that the stochastic optimality of the JSQ policy can be preserved at the fluid-level and diffusion-level while reducing the overhead by nearly a factor O( N ) and O(√ N ), respectively.