Degree of Queue Imbalance

Abstract

In this paper, we argue that heavy-traffic delay optimality is a coarse metric that does not necessarily imply good delay performance. Specifically, we show that any load balancing scheme is heavy-traffic delay optimal as long as it satisfies a fairly weak condition. This condition only requires that in the long-term the dispatcher favors, even slightly, shorter queues over longer queues. Hence, the empirical delay performance of heavy-traffic delay optimal schemes can range from very good (that of join-shortest-queue) to very bad (arbitrarily close to the performance of random routing). To overcome this limitation, we introduce a new metric called degree of queue imbalance, which measures the queue length difference between all the servers in steady-state. Given a heavy-traffic delay optimal load balancing scheme, we can characterize the resultant degree of queue imbalance. This, in turn, allows us to explicitly differentiate between good and poor load balancing schemes. Thus, this paper implies that good load balancing should not only be designed to be heavy-traffic delay optimal, but also have a low degree of queue imbalance.