Optimal Rate Control for Latency-constrained High Throughput Big Data Applications

Abstract

High performance distributed systems such as distributed stream processing systems and message-passing parallel programs are often deployed on platforms that make use of vanilla TCP/IP communication, which in turn uses the conventional Nagle’s algorithm for congestion control. Recent research in Reinforcement Learning (RL) techniques to either replace or control the conventional TCP approach shows promise in achieving a greater degree of performance, especially when the demand for network resources in a multi-tenant platform is highly dynamic and infeasible to model. Existing results are, however, focused on RL for general Internet communication, with the learning objective being some combination of throughput, loss, and latency, and predominately use a continuous action space to adjust the packet rate at the sender. In this work, we propose a coefficient-free RL objective that perfectly matches the data transmission rate to the underlying communication system’s bottleneck, which naturally deters packet loss and thereby converges to the ideal throughput even in lock-free and latency-constrained Big Data applications where packets are dropped due to load shedding or exceeding latency thresholds. Our results compare favorably to other state-of-the-art objective functions using an RL framework, e.g., providing up to 48% reduction in packet loss while obtaining up to a 4% increase in overall throughput when packet latency is highly constrained.