MP-CREDIT: Multi-path credit for high-speed data center transports

Abstract

In high-speed interconnection networks, credit-based flow control is widely deployed at the link layer to ensure data lossless. In this case, each credit schedules data on a one-hop fixed link. When employing credit in higher layers, e.g. transport layer, credit needs to be transmitted through multiple hops to realize end-to-end scheduling. In credit-based transports, each credit is transmitted to the data sender through a credit path, in return, each data packet is sent to the data receiver through a data path. In modern high-speed data centers that deploy topologies with multiple equivalent paths, the multi-hop credit can be routed into diverse paths. For the credit-based transports that require strong path-consistency between the data path and the credit path, properly routing credits becomes significant to improve data transmission efficiency. However, existing solutions lack consideration for exploiting multiple paths, this limits them to be single-path transports and affects their performance.To optimize the end-to-end latency and fully utilize the network resources, this paper presents MP-CREDIT, the first multi-path credit protocol designed for strong path-consistency transports. MP-CREDIT leverages credit spray to uniformly balance the traffic on multiple paths, and employs sequence-free feedback control to remedy credit out-of-order. Besides, we develop path probing for MP-CREDIT to handle path failures. Compared to state-of-the-art credit-based transport, MP-CREDIT reduces average flow completion times to 0.53x in the realistic workloads, greatly improves the throughput to 1.28x with permutation traffic, and achieves much better fairness in asymmetry network.