Abstract
Large service providers use load balancers to dispatch millions of incoming connections per second towards thousands of servers. There are two basic yet critical requirements for a load balancer: <i>uniform load distribution</i> of the incoming connections across the servers, which requires to support advanced load balancing mechanisms, and <i>per-connection-consistency</i> (PCC), i.e, the ability to map packets belonging to the same connection to the same server even in the presence of changes in the number of active servers and load balancers. Yet, simultaneously meeting these requirements has been an elusive goal. Today’s load balancers minimize PCC violations at the price of non-uniform load distribution. This paper presents Cheetah, a load balancer that supports advanced load balancing mechanisms <i>and</i> PCC while being scalable, memory efficient, fast at processing packets, and offers comparable resilience to clogging attacks as with today’s load balancers. The Cheetah LB design guarantees PCC for <i>any</i> realizable server selection load balancing mechanism and can be deployed in both stateless and stateful manners, depending on operational needs. We implemented Cheetah on both a software and a Tofino-based hardware switch. Our evaluation shows that a stateless version of Cheetah guarantees PCC, has negligible packet processing overheads, and can support load balancing mechanisms that reduce the flow completion time by a factor of <inline-formula> <tex-math notation="LaTeX">$2-3 \times $ </tex-math></inline-formula>.
Highlights
T HE vast majority of services deployed in a datacenter need load balancers (LBs) to spread incoming connection requests over the set of servers running these services
We note that implementing other types of LB mechanisms such as least-loaded in the data-plane is non-trivial in P4 since one would need to extract a minimum from an array in O(1)
The CHEETAH LB design allows datacenter operators to unleash the power of arbitrary load balancing mechanisms while guaranteeing PCC, i.e, the ability to grow/shrink the LB and Direct IP (DIP) pools without disrupting existing connections
Summary
T HE vast majority of services deployed in a datacenter need load balancers (LBs) to spread incoming connection requests over the set of servers running these services. When only the number of LBs change, hash-based load balancing mechanisms guarantee PCC as packets reach the correct server even when sent to a different LB [5], [7]. Unlike prior work, which relies on hash computations to mitigate PCC violations, the design of CHEETAH completely decouples the load balancing logic from PCC support. This in turn allows an operator to guarantee PCC regardless of the “connection-to-server” mapping produced by the chosen load balancing logic. The number of processor cycles per packet for both our stateless and stateful implementation of CHEETAH is comparable to existing stateless implementations and 3.5x fewer cycles per packet than existing stateful LBs
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
