Abstract
HPC network technologies like Infiniband, TrueScale or OmniPath provide low- latency and high-throughput communication between hosts, which makes them attractive options for data-acquisition systems in large-scale high-energy physics experiments. Like HPC networks, DAQ networks are local and include a well specified number of systems. Unfortunately traditional network communication APIs for HPC clusters like MPI or PGAS exclusively target the HPC community and are not suited well for DAQ applications. It is possible to build distributed DAQ applications using low-level system APIs like Infiniband Verbs, but it requires a non-negligible effort and expert knowledge.At the same time, message services like ZeroMQ have gained popularity in the HEP community. They make it possible to build distributed applications with a high-level approach and provide good performance. Unfortunately, their usage usually limits developers to TCP/IP- based networks. While it is possible to operate a TCP/IP stack on top of Infiniband and OmniPath, this approach may not be very efficient compared to a direct use of native APIs.NetIO is a simple, novel asynchronous message service that can operate on Ethernet, Infiniband and similar network fabrics. In this paper the design and implementation of NetIO is presented and described, and its use is evaluated in comparison to other approaches. NetIO supports different high-level programming models and typical workloads of HEP applications. The ATLAS FELIX project [1] successfully uses NetIO as its central communication platform.The architecture of NetIO is described in this paper, including the user-level API and the internal data-flow design. The paper includes a performance evaluation of NetIO including throughput and latency measurements. The performance is compared against the state-of-the- art ZeroMQ message service. Performance measurements are performed in a lab environment with Ethernet and FDR Infiniband networks.
Highlights
High-performance networking is crucial for large data-acquisition (DAQ) systems
A project in the ALICE experiment is evaluating the use of nanomsg [13], a system similar to ZeroMQ, and developing an OFI (OpenFabrics Interface) transport for nanomsg that allows it to be run on Infiniband, OmniPath and other high performance interconnects [14]
The NetIO library was designed with the goal to make HPC network technologies like Infiniband accessible for DAQ systems in high-energy physics (HEP) experiments
Summary
High-performance networking is crucial for large data-acquisition (DAQ) systems. Such systems are composed of thousands of individual components that communicate with each other. (ii) The transport layer in NetIO can be provided by different back-ends, so that NetIO can communicate natively on Ethernet, Infiniband, and other network technologies This allows users of NetIO to leverage technologies from the HPC domain for DAQ systems, which have been traditionally dominated by Ethernet. This paper will describe the architecture and design of NetIO, including how the library uses the network stack and tunes for the different workload scenarios (low-latency and highthroughput). A project in the ALICE experiment is evaluating the use of nanomsg [13], a system similar to ZeroMQ, and developing an OFI (OpenFabrics Interface) transport for nanomsg that allows it to be run on Infiniband, OmniPath and other high performance interconnects [14]
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
