Low Level Trigger Research Articles

Progress report on the online processing upgrade at the NA62 experiment

A new FPGA-based low-level trigger processor has been installed at the NA62 experiment. It is intended to extend the features of its predecessor due to a faster interconnection technology and additional logic resources available on the new platform. With the aim of improving trigger selectivity and exploring new architectures for complex trigger computation, a GPU system has been developed and a neural network on FPGA is in progress. They both process data streams from the ring imaging Cherenkov detector of the experiment to extract in real time high level features for the trigger logic. Description of the systems, latest developments and design flows are reported in this paper.

The integrated low-level trigger and readout system of the CERN NA62 experiment

The integrated low-level trigger and data acquisition (TDAQ) system of the NA62 experiment at CERN is described. The requirements of a large and fast data reduction in a high-rate environment for a medium-scale, distributed ensemble of many different sub-detectors led to the concept of a fully digital integrated system with good scaling capabilities. The NA62 TDAQ system is rather unique in allowing full flexibility on this scale, allowing in principle any information available from the detector to be used for triggering. The design concept, implementation and performances from the first years of running are illustrated.

Real-time heterogeneous stream processing with NaNet in the NA62 experiment

The use of GPUs to implement general purpose computational tasks, known as GPGPU since fifteen years ago, has reached maturity. Applications take advantage of the parallel architectures of these devices in many different domains.Over the last few years several works have demonstrated the effectiveness of the integration of GPU-based systems in the high level trigger of various HEP experiments. On the other hand, the use of GPUs in the DAQ and low level trigger systems, characterized by stringent real-time constraints, poses several challenges.In order to achieve such a goal we devised NaNet, a FPGA-based PCI-Express Network Interface Card design capable of direct (zero-copy) data transferring with CPU and GPU (GPUDirect) while online processing incoming and outgoing data streams. The board provides as well support for multiple link technologies (1/10/40GbE and custom ones).The validity of our approach has been tested in the context of the NA62 CERN experiment, harvesting the computing power of last generation NVIDIA Pascal GPUs and of the FPGA hosted by NaNet to build in real-time refined physics-related primitives for the RICH detector (i.e. the Cerenkov rings parameters) that enable the building of more stringent conditions for data selection in the low level trigger.

A readout digitization system for the ATLAS muon spectrometer phase-I tRPC trigger upgrade

The front-end digitization electronics for thin-Resistive Plate Chambers to be installed as part of the ATLAS Muon Spectrometer Phase-I upgrade project is described. The design is based on the CERN HPTDC and GOL chips. In this paper, the design of the digitization system is presented, including power, connection interfaces to front-end amplifier-shaper-discriminator board and to the low level trigger processor board. System performance tests were conducted and validated with a prototype which includes one HPTDC/GOL pair.

Development of Network Interface Cards for TRIDAQ systems with the NaNet framework

NaNet is a framework for the development of FPGA-based PCI Express (PCIe) Network Interface Cards (NICs) with real-time data transport architecture that can be effectively employed in TRIDAQ systems. Key features of the architecture are the flexibility in the configuration of the number and kind of the I/O channels, the hardware offloading of the network protocol stack, the stream processing capability, and the zero-copy CPU and GPU Remote Direct Memory Access (RDMA). Three NIC designs have been developed with the NaNet framework: NaNet-1 and NaNet-10 for the CERN NA62 low level trigger and NaNet3 for the KM3NeT-IT underwater neutrino telescope DAQ system. We will focus our description on the NaNet-10 design, as it is the most complete of the three in terms of capabilities and integrated IPs of the framework.

A high-speed DAQ framework for future high-level trigger and event building clusters

Modern data acquisition and trigger systems require a throughput of several GB/s and latencies of the order of microseconds. To satisfy such requirements, a heterogeneous readout system based on FPGA readout cards and GPU-based computing nodes coupled by InfiniBand has been developed. The incoming data from the back-end electronics is delivered directly into the internal memory of GPUs through a dedicated peer-to-peer PCIe communication. High performance DMA engines have been developed for direct communication between FPGAs and GPUs using “DirectGMA (AMD)” and “GPUDirect (NVIDIA)” technologies. The proposed infrastructure is a candidate for future generations of event building clusters, high-level trigger filter farms and low-level trigger system. In this paper the heterogeneous FPGA-GPU architecture will be presented and its performance be discussed.

GPU-based low-level trigger system for the standalone reconstruction of the ring-shaped hit patterns in the RICH Cherenkov detector of NA62 experiment

This project aims to exploit the parallel computing power of a commercial Graphics Processing Unit (GPU) to implement fast pattern matching in the Ring Imaging Cherenkov (RICH) detector for the level 0 (L0) trigger of the NA62 experiment. In this approach, the ring-fitting algorithm is seedless, being fed with raw RICH data, with no previous information on the ring position from other detectors. Moreover, since the L0 trigger is provided with a more elaborated information than a simple multiplicity number, it results in a higher selection power. Two methods have been studied in order to reduce the data transfer latency from the readout boards of the detector to the GPU, i.e., the use of a dedicated NIC device driver with very low latency and a direct data transfer protocol from a custom FPGA-based NIC to the GPU. The performance of the system, developed through the FPGA approach, for multi-ring Cherenkov online reconstruction obtained during the NA62 physics runs is presented.

Latency study of the High Performance Time to Digital Converter for the ATLAS Muon Spectrometer trigger upgrade

The High Performance Time to Digital Converter (HPTDC), a multi-channel ASIC designed by the CERN Microelectronics group, has been proposed for the digitization of the thin-Resistive Plate Chambers (tRPC) in the ATLAS Muon Spectrometer Phase-1 upgrade project. These chambers, to be staged for higher luminosity LHC operation, will increase trigger acceptance and reduce or eliminate the fake muon trigger rates in the barrel-endcap transition region, corresponding to pseudo-rapidity range 1<|η|<1.3. Low level trigger candidates must be flagged within a maximum latency of 1075 ns, thus imposing stringent signal processing time performance requirements on the readout system in general, and on the digitization electronics in particular. This paper investigates the HPTDC signal latency performance based on a specially designed evaluation board coupled with an external FPGA evaluation board, when operated in triggerless mode, and under hit rate conditions expected in Phase-I. This hardware based study confirms previous simulations and demonstrates that the HPTDC in triggerless operation satisfies the digitization timing requirements in both leading edge and pair modes.

Reconfigurable ASIC for a low level trigger system in Cherenkov Telescope Cameras

A versatile and reconfigurable ASIC is presented, which implements two different concepts of low level trigger (L0) for Cherenkov telescopes: the Majority trigger (sum of discriminated inputs) and the Sum trigger concept (analogue clipped sum of inputs). Up to 7 input signals can be processed following one or both of the previous trigger concepts. Each differential pair output of the discriminator is also available as a LVDS output. Differential circuitry using local feedback allows the ASIC to achieve high speed (500 MHz) while maintaining good linearity in a 1 Vpp range. Experimental results are presented. A number of prototype camera designs of the Cherenkov Telescope Array (CTA) project will use this ASIC.

Graphical processors for HEP trigger systems

General-purpose computing on GPUs is emerging as a new paradigm in several fields of science, although so far applications have been tailored to employ GPUs as accelerators in offline computations. With the steady decrease of GPU latencies and the increase in link and memory throughputs, time is ripe for real-time applications using GPUs in high-energy physics data acquisition and trigger systems. We will discuss the use of online parallel computing on GPUs for synchronous low level trigger systems, focusing on tests performed on the trigger of the CERN NA62 experiment. Latencies of all components need analysing, networking being the most critical. To keep it under control, we envisioned NaNet, an FPGA-based PCIe Network Interface Card (NIC) enabling GPUDirect connection. Moreover, we discuss how specific trigger algorithms can be parallelised and thus benefit from a GPU implementation, in terms of increased execution speed. Such improvements are particularly relevant for the foreseen LHC luminosity upgrade where highly selective algorithms will be crucial to maintain sustainable trigger rates with very high pileup.

NaNet-10: a 10GbE network interface card for the GPU-based low-level trigger of the NA62 RICH detector.

A GPU-based low level (L0) trigger is currently integrated in the experimental setup of the RICH detector of the NA62 experiment to assess the feasibility of building more refined physics-related trigger primitives and thus improve the trigger discriminating power. To ensure the real-time operation of the system, a dedicated data transport mechanism has been implemented: an FPGA-based Network Interface Card (NaNet-10) receives data from detectors and forwards them with low, predictable latency to the memory of the GPU performing the trigger algorithms. Results of the ring-shaped hit patterns reconstruction will be reported and discussed.

Graphics Processors in HEP Low-Level Trigger Systems

Usage of Graphics Processing Units (GPUs) in the so called general-purpose computing is emerging as an effective approach in several fields of science, although so far applications have been employing GPUs typically for offline computations. Taking into account the steady performance increase of GPU architectures in terms of computing power and I/O capacity, the real-time applications of these devices can thrive in high-energy physics data acquisition and trigger systems. We will examine the use of online parallel computing on GPUs for the synchronous low-level trigger, focusing on tests performed on the trigger system of the CERN NA62 experiment. To successfully integrate GPUs in such an online environment, latencies of all components need analysing, networking being the most critical. To keep it under control, we envisioned NaNet, an FPGA-based PCIe Network Interface Card (NIC) enabling GPUDirect connection. Furthermore, it is assessed how specific trigger algorithms can be parallelized and thus benefit from a GPU implementation, in terms of increased execution speed. Such improvements are particularly relevant for the foreseen Large Hadron Collider (LHC) luminosity upgrade where highly selective algorithms will be essential to maintain sustainable trigger rates with very high pileup.

Graphics Processing Units for HEP trigger systems

General-purpose computing on GPUs (Graphics Processing Units) is emerging as a new paradigm in several fields of science, although so far applications have been tailored to the specific strengths of such devices as accelerator in offline computation. With the steady reduction of GPU latencies, and the increase in link and memory throughput, the use of such devices for real-time applications in high-energy physics data acquisition and trigger systems is becoming ripe. We will discuss the use of online parallel computing on GPU for synchronous low level trigger, focusing on CERN NA62 experiment trigger system. The use of GPU in higher level trigger system is also briefly considered.

NaNet: a configurable NIC bridging the gap between HPC and real-time HEP GPU computing

NaNet is a FPGA-based PCIe Network Interface Card (NIC) design with GPUDirect and Remote Direct Memory Access (RDMA) capabilities featuring a configurable and extensible set of network channels. The design currently supports both standard—Gbe (1000BASE-T) and 10GbE (10Base-R)—and custom—34 Gbps APElink and 2.5 Gbps deterministic latency KM3link—channels, but its modularity allows for straightforward inclusion of other link technologies. The GPUDirect feature combined with a transport layer offload module and a data stream processing stage makes NaNet a low-latency NIC suitable for real-time GPU processing. In this paper we describe the NaNet architecture and its performances, exhibiting two of its use cases: the GPU-based low-level trigger for the RICH detector in the NA62 experiment at CERN and the on-/off-shore data transport system for the KM3NeT-IT underwater neutrino telescope.

NaNet: a low-latency NIC enabling GPU-based, real-time low level trigger systems

We implemented the NaNet FPGA-based PCIe Gen2 GbE/APElink NIC, featuring GPUDirect RDMA capabilities and UDP protocol management offloading. NaNet is able to receive a UDP input data stream from its GbE interface and redirect it, without any intermediate buffering or CPU intervention, to the memory of a Fermi/Kepler GPU hosted on the same PCIe bus, provided that the two devices share the same upstream root complex. Synthetic benchmarks for latency and bandwidth are presented. We describe how NaNet can be employed in the prototype of the GPU-based RICH low-level trigger processor of the NA62 CERN experiment, to implement the data link between the TEL62 readout boards and the low level trigger processor. Results for the throughput and latency of the integrated system are presented and discussed.

GPUs for real-time processing in HEP trigger systems

We describe a pilot project for the use of Graphics Processing Units (GPUs) for online triggering applications in High Energy Physics (HEP) experiments. Two major trends can be identified in the development of trigger and DAQ systems for HEP experiments: the massive use of general-purpose commodity systems such as commercial multicore PC farms for data acquisition, and the reduction of trigger levels implemented in hardware, towards a pure software selection system (trigger-less). The very innovative approach presented here aims at exploiting the parallel computing power of commercial GPUs to perform fast computations in software both at low- and high-level trigger stages. General-purpose computing on GPUs is emerging as a new paradigm in several fields of science, although so far applications have been tailored to the specific strengths of such devices as accelerator in offline computation. With the steady reduction of GPU latencies, and the increase in link and memory throughputs, the use of such devices for real-time applications in high-energy physics data acquisition and trigger systems is becoming very attractive. We discuss in details the use of online parallel computing on GPUs for synchronous low-level trigger with fixed latency. In particular we show preliminary results on a first test in the NA62 experiment at CERN. The use of GPUs in high-level triggers is also considered, the ATLAS experiment (and in particular the muon trigger) at CERN will be taken as a study case of possible applications.

NaNet: a flexible and configurable low-latency NIC for real-time trigger systems based on GPUs

NaNet is an FPGA-based PCIe X8 Gen2 NIC supporting 1/10 GbE links and the custom 34 Gbps APElink channel. The design has GPUDirect RDMA capabilities and features a network stack protocol offloading module, making it suitable for building low-latency, real-time GPU-based computing systems. We provide a detailed description of the NaNet hardware modular architecture. Benchmarks for latency and bandwidth for GbE and APElink channels are presented, followed by a performance analysis on the case study of the GPU-based low level trigger for the RICH detector in the NA62 CERN experiment, using either the NaNet GbE and APElink channels. Finally, we give an outline of project future activities.

First performance studies of a pixel-based trigger in the CMS experiment

An important tool for the discovery of new physics at LHC is the design of a low level trigger with an high power of background rejection. The contribution of pixel detector to the lowest level trigger at CMS is studied focusing on low-energy jet identification, matching the information from calorimeters and pixel detector. In addition, primary vertex algorithms are investigated. The performances are evaluated in terms of, respectively, QCD rejection and multihadronic jets final states efficiency.

Timing compensation for an excimer laser solid-state pulsed power module (SSPPM)

For certain applications, it is critical to minimize variations in the (throughput) timing between trigger and the output pulse of a magnetic modulator. A circuit is described that maintains a relatively constant delay over a large operating voltage range (600-1150 V) and temperature range (25 /spl deg/C-65 /spl deg/C) range, The circuit operates by sampling the charging voltage and magnetic switch temperature just prior to the start switch trigger. Those parameters are then used to calculate the appropriate amount of delay to add into the low-level trigger chain to ensure that the delay stays constant over the voltage and temperature operating range. Although other approaches can be conceived and implemented, this particular design is relatively simple and inexpensive and meets the desired performance goals. Data presented show that the ideal correction function is nonlinear in nature and, as a result, simple linear approximations are limited in their ability to minimize the timing variations. Improvements to the original circuit use a multiple, piece-wise, linear approach in order to obtain better performance. The results are that an initial timing variation of almost 3 /spl mu/s has been reduced to a total variation of less than 100 ns.

High peak power, burst mode operation of a multikilowatt CO2 laser for deep penetration welding

A significant advancement in laser welding of heavy metal sections is presented. Operation of a photoinitiated, impulse-enhanced, electrically excited multikilowatt continuous wave (cw) laser in the pulse periodic regime results in significant improvement in weld penetration depth. High average power pulsed operation is made possible through the use of a low level trigger for a digitally controlled hydrogen thyratron pulser circuit. Peak optical powers on the order of 3 times the cw level are produced while maintaining multikilowatt average output power. A comparison of cw and pulsed welding of X70 pipeline steel is included. Burst mode welding with this system is found to yield up to a 50% increase in weld penetration with a superior weld profile when compared to cw welding at the same average power level.