Abstract
The data acquisition (DAQ) system of the CMS experiment at the CERN Large Hadron Collider assembles events at a rate of 100 kHz, transporting event data at an aggregate throughput of 100 GB/s to the high level trigger (HLT) farm. The HLT farm selects interesting events for storage and offline analysis at a rate of around 1 kHz. The DAQ system has been redesigned during the accelerator shutdown in 2013/14. The motivation is twofold: Firstly, the current compute nodes, networking, and storage infrastructure will have reached the end of their lifetime by the time the LHC restarts. Secondly, in order to handle higher LHC luminosities and event pileup, a number of sub-detectors will be upgraded, increasing the number of readout channels and replacing the off-detector readout electronics with a <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$\mu {\hbox {TCA}}$</tex></formula> implementation. The new DAQ architecture will take advantage of the latest developments in the computing industry. For data concentration, 10/40 Gb/s Ethernet technologies will be used, as well as an implementation of a reduced TCP/IP in FPGA for a reliable transport between custom electronics and commercial computing hardware. A Clos network based on 56 Gb/s FDR Infiniband has been chosen for the event builder with a throughput of <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$\sim 4~\hbox{Tb/s}$</tex> </formula> . The HLT processing is entirely file based. This allows the DAQ and HLT systems to be independent, and to use the HLT software in the same way as for the offline processing. The fully built events are sent to the HLT with 1/10/40 Gb/s Ethernet via network file systems. Hierarchical collection of HLT accepted events and monitoring meta-data are stored into a global file system. This paper presents the requirements, technical choices, and performance of the new system.
Highlights
T HE Compact Muon Solenoid (CMS) experiment [1], [2] at CERN’s Large Hadron collider is one of two large general-purpose experiments exploring a wide range of physics at the TeV scale
The data acquisition (DAQ) system [4], [2] reads out the detector and assembles full events at 100 kHz, passing the assembled events to the high-level trigger, a software system based on the full CMS reconstruction software, running on a farm of computers
At a nominal event size of 1 MB, the CMS DAQ system for Run-1 was designed to handle a throughput of 100 GB/s, making it the highest throughput DAQ system in high-energy physics to date
Summary
T HE Compact Muon Solenoid (CMS) experiment [1], [2] at CERN’s Large Hadron collider is one of two large general-purpose experiments exploring a wide range of physics at the TeV scale. At a nominal event size of 1 MB, the CMS DAQ system for Run-1 was designed to handle a throughput of 100 GB/s, making it the highest throughput DAQ system in high-energy physics to date. Some of the CMS sub-systems are currently upgrading their detectors and/or on-line systems and are building new off-detector read-out electronics based on the μTCA standard. In this paper we present the final design of the new DAQ system that is currently being installed, and report on performance measurements including first measurements in the production system. In each of the sections we report on the requirements, design and on latest performance measurements
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have