Haste makes waste: The On–Off algorithm for replica selection in key–value stores

Abstract

In current large-scale distributed key–value stores, the tail latency of the key–value accesses generated by end-user requests is crucial to the response time of these requests. To cut the tail latency, the replica selection algorithm, which selects a replica server for each key by a client and thus determines the latency of each key–value access, is crucial. Naturally, all of current replica selection algorithms send keys out immediately to reduce the tail latencies of key–value accesses. In this paper, we find that sending out keys in haste makes waste of the chance to select a better replica server certain time later, and suggest to await for a better replica server available when all current replica servers are bad. To realize this idea, we develop the On–Off algorithm, which recognizes bad replica servers according to the feedback information and put them into the OFF state. Special attention is paid on the time interval replica servers stay at the OFF state. Obviously, the On–Off algorithm adds awaiting time at clients, but can greatly reduce the dominated queuing delays at replica servers. In total, the On–Off algorithm improves the 99th latency by about 29% under the default simulation configuration and outperforms the C3 algorithm proposed recently under kinds of scenarios.