EJ-FAT Joint ESnet JLab FPGA Accelerated Transport Load Balancer

Abstract

To increase the science rate for high data rates/volumes, Thomas Jefferson National Accelerator Facility (JLab) has partnered with Energy Sciences Network (ESnet) to define an edge to data center traffic shaping / steering transport capability featuring data event-aware network shaping and forwarding. The keystone of this ESnet JLab FPGA Accelerated Transport (EJFAT) is the joint development of a dynamic compute work Load Balancer (LB) of UDP streamed data. The LB is a suite consisting of a Field Programmable Gate Array (FPGA) executing the dynamically configurable, low fixed latency LB data plane featuring real-time packet redirection at high throughput, and a control plane running on the FPGA host computer that monitors network and compute farm telemetry in order to make dynamic decisions for destination compute host redirection / load balancing. The LB provides for three forms of scaling. It provides horizontal scale by adding more FPGAs for increased bandwidth. Second it scales out to the number of core compute hosts independent of the number of source DAQs. Thirdly it allows for a flexible number of CPUs and threads per host, treating each receiving thread as an independent LB destination. The LB provides seamless integration of edge / core computing to support direct experimental data processing. Immediate use will be at JLab science programs and others such as the EIC (Electron Ion Collider). Data centers of the future will need high throughput and low latency for both live streamed and recorded data for running experiment data acquisition analysis and data center use cases. EJ-FAT is a development for production use within DOE. When completed, it will have an operational impact for integrated research infrastructure as called for in [8], [9], and [10]. It demonstrates a new load balancing architecture, when compared with prior solutions like Server Load Balancing.