Abstract
This paper evaluates the real-time distribution of data over Ethernet for the upgraded LHCb data acquisition cluster at CERN. The system commissioning ends in 2021 and its total estimated input throughput is 32 Terabits per second. After the events are assembled, they must be distributed for further data selection to the filtering farm of the online trigger. High-throughput and very low overhead transmissions will be an essential feature of such a system. In this work RoCE (Remote Direct Memory Access over Converged Ethernet) high-throughput Ethernet protocol and Ethernet flow control algorithms have been used to implement lossless event distribution. To generate LHCb-like traffic, a custom benchmark has been implemented. It was used to stress-test the selected Ethernet networks and to check resilience to uneven workload distribution. Performance tests were made with selected evaluation clusters. 100 Gb/s and 25 Gb/s links were used. Performance results and overall evaluation of this Ethernet-based approach are discussed.
Highlights
The present CERN LHCb upgrade [1, 2] involves a significant increase in the throughput in the filtering farm [3,4,5]
We presented a feasibility study, whether or not the Ethernet networks can handle the LHCb workload dispatch in the LHCb filtering farm
Our purpose was to check if Ethernet with flow control protocols is performant enough for the many-to-many traffic, with different link speeds with different numbers of data receivers and consumers
Summary
The present CERN LHCb upgrade [1, 2] involves a significant increase in the throughput in the filtering farm [3,4,5]. One of the tested scenarios was to use a network to dispatch workloads between the data producers (that is, the event building nodes doing assembly) and the data consumers (that is, the filtering nodes doing selection). This many-to-many high-throughput event distribution traffic must be lossless. In the event of the temporary busyness of some of the data selection servers, the workloads should be redirected to non-busy nodes For this specific use-case, Ethernet was considered as a potential alternative to InfiniBand.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
