Redundancy-d: The Power of d Choices for Redundancy

Abstract

Redundancy is an important strategy for reducing response time in multi-server distributed queueing systems. This strategy has been used in a variety of settings, but only recently have researchers begun analytical studies. The idea behind redundancy is that customers can greatly reduce response time by waiting in multiple queues at the same time, thereby experiencing the minimum time across queues. Redundancy has been shown to produce significant response time improvements in applications ranging from organ transplant waitlists to Google’s BigTable service. However, despite the growing body of theoretical and empirical work on the benefits of redundancy, there is little work addressing the questions of how many copies one needs to make to achieve a response time benefit, and the magnitude of the potential gains. In this paper we propose a theoretical model and dispatching policy to evaluate these questions. Our system consists of k servers, each with its own queue. We introduce the Redundancy-d policy, under which each incoming job makes copies at a constant number of servers, d, chosen at random. Under the assumption that a job’s service times are exponential and independent across servers, we derive the first exact expressions for mean response time in Redundancy-d systems with any finite number of servers, as well as expressions for the distribution of response time which are exact as the number of servers approaches infinity. Using our analysis, we show that mean response time decreases as d increases, and that the biggest marginal response time improvement comes from having each job wait in only d = 2 queues. The e-companion is available at https://doi.org/10.1287/opre.2016.1582 .