Event Building Research Articles

Data reduction for low energy nuclear physics experiments using data frames

Low energy nuclear physics experiments are transitioning towards fully digital data acquisition systems. Realizing the gains in flexibility afforded by these systems relies on equally flexible data reduction techniques. In this paper, methods utilizing data frames and in-memory techniques to work with data, including data from self-triggering, digital data acquisition systems, are discussed within the context of a Python package, sauce. It is shown that data frame operations can encompass common analysis needs and allow interactive data analysis. Two event building techniques, dubbed referenced and referenceless event building, are shown to provide a means to transform raw list mode data into correlated multi-detector events. These techniques are demonstrated in the analysis of two example data sets.

Codebase release 2.0 for sauce

Low energy nuclear physics experiments are transitioning towards fully digital data acquisition systems. Realizing the gains in flexibility afforded by these systems relies on equally flexible data reduction techniques. In this paper, methods utilizing data frames and in-memory techniques to work with data, including data from self-triggering, digital data acquisition systems, are discussed within the context of a Python package, sauce. It is shown that data frame operations can encompass common analysis needs and allow interactive data analysis. Two event building techniques, dubbed referenced and referenceless event building, are shown to provide a means to transform raw list mode data into correlated multi-detector events. These techniques are demonstrated in the analysis of two example data sets.

Photon classification with Gradient Boosted Trees at CLAS12

Dihadron semi-inclusive deep inelastic scattering (SIDIS) of 10.6 GeV longitudinally polarized electrons off the proton has been measured using the CLAS12 detector at Jefferson Lab. Two separate channels, π + π 0 and π - π 0, were analyzed, requiring the reconstruction of diphoton pairs. In this analysis, we addressed the problem of false neutral particles being reconstructed by CLAS12's event builder, polluting the otherwise physical combinatorial background underneath the π 0 peak. A photon classifier using a Gradient Boosted Trees (GBTs) architecture was trained with Monte Carlo simulations to reduce the amount of background π 0's. We show that the nearest-neighbor features learned by the model lead to a substantial increase in signal vs. background discrimination compared to previous CLAS12 π^0 analyses. The machine learning approach recovers several times more dihadron statistics for the dataset.

The CMS Orbit Builder for the HL-LHC at CERN

The CMS Orbit Builder for the HL-LHC at CERN

MiniDAQ-3: Providing concurrent independent subdetector data-taking on CMS production DAQ resources

The data acquisition (DAQ) of the Compact Muon Solenoid (CMS) experiment at CERN, collects data for events accepted by the Level-1 Trigger from the different detector systems and assembles them in an event builder prior to making them available for further selection in the High Level Trigger, and finally storing the selected events for offline analysis. In addition to the central DAQ providing global acquisition functionality, several separate, so-called “MiniDAQ” setups allow operating independent data acquisition runs using an arbitrary subset of the CMS subdetectors. During Run 2 of the LHC, MiniDAQ setups were running their event builder and High Level Trigger applications on dedicated resources, separate from those used for the central DAQ. This cleanly separated MiniDAQ setups from the central DAQ system, but also meant limited throughput and a fixed number of possible MiniDAQ setups. In Run 3, MiniDAQ-3 setups share production resources with the new central DAQ system, allowing each setup to operate at the maximum Level-1 rate thanks to the reuse of the resources and network bandwidth. Configuration management tools had to be significantly extended to support the synchronization of the DAQ configurations needed for the various setups. We report on the new configuration management features and on the first year of operational experience with the new MiniDAQ-3 system.

Benchmarking Data Acquisition event building network performance for the ATLAS HL-LHC upgrade

The ATLAS experiment’s data acquisition (DAQ) system will be extensively updated to take full advantage of the High-Luminosity LHC (HL-LHC) upgrade, allowing it to record data at unprecedented rates. The detector will be read out at 1 MHz, generating over 5 TB/s of data. This design poses significant challenges for the Ethernet-based network, which will have to transport 20 times more data than during Run 3. The increased data rate, data sizes and number of servers will exacerbate the TCP Incast effect observed in the past, making it impossible to fully exploit the capabilities of the network and limiting the performance of the processing farm. We present exhaustive and systematic experiments to define buffering requirements in network equipment to minimise the effects of TCP Incast and reduce the impact on processing applications. Both deep and shallow buffer switches were stress-tested using DAQ traffic patterns in a test environment at approximately 10% of the expected HL-LHC DAQ system size. As the desired HL-LHC system hardware is not currently available and the laboratory size is significantly smaller, the tests aim to extrapolate buffer requirements to the expected operating point. A novel analytical formula and new simulation models have been developed to cross-validate the results. The results of these evaluations will contribute to the decision-making process for the acquisitions of network hardware for the HL-LHC DAQ.

Functional characterization of modules for the Silicon Tracking System of the CBM experiment

The CBM experiment is a fixed-target heavy-ion physics experiment at the future FAIR facility, designed to explore the QCD phase diagram at high baryon-net densities and moderate temperatures. To address its complex physics program, the experiment is conceived to operate at beam–target interaction rates up to 10 MHz with self-triggered front-end electronics and free-streaming readout architecture. The Silicon Tracking System (STS) of CBM is the main charged-particle tracker, capable of delivering time-stamped hits for online tracking and event building, and momentum measurement inside a superconducting dipole magnet. The functional unit of the STS is the detector module, a device assembled from a double-sided silicon microstrip sensor connected via a stack of microcables to a pair of front-end boards carrying the readout electronics. The assembly and successful operation of STS modules, produced with close-to-final components, was an important milestone achieved in recent years. The systematic study of the modules’ performance is also necessary for optimizing the assembly procedure and ensuring high production yield. This work reports a series of tests, which aim to evaluate the module’s operation and establish the basic criteria for quality control. It summarizes the procedures and results of different measurements performed with the modules, starting with noise estimation and charge amplitude calibration, resulting in the linearization of the channel’s ADC and a homogeneous threshold distribution. Those results were verified using a radioactive source. Additionally, first conclusions concerning the assembly yield in terms of number of functional channels were obtained. These studies represent a fundamental step towards the series production readiness for the final STS detector.

Design and Commissioning of the First 32-Tbit/s Event-Builder

The LHCb experiment is a forward spectrometer, designed to study beauty and charm quarks Physics at the LHC. To exploit of the higher luminosity that will be delivered during Run3, the full experiment needed a substantial upgrade, from the detector to the Data Acquisition (DAQ) and High Level Trigger (HLT). In this paper, we will focus on the new DAQ system for the LHCb experiment; that represents a substantial paradigm shift compared to the previous one, and to similar systems used by similar experiments in the past and present times. To overcome the inefficiencies introduced by a local selection implemented directly with the readout hardware, the Run3 system is designed to perform a full software reconstruction of all the produced events. To achieve this, both the DAQ and the HLT need to process the ~ 30 MHz full event-rate. In particular, this paper will introduce the final design of the system; it will provide a focus on the hardware and software design of the Event Building (EB) and how we integrated technologies designed for the High Performance Computing (HPC) world - like InfiniBand HDR (200 Gb/s)- into the DAQ system; we will present performance measurements of the full EB system under different operational conditions; and we will provide a feedback from EB operation during the beginning of the data-taking.

Trigger Timing Interface for the Read-Out Upgrade of the Belle II DAQ

To improve data throughput of the Belle II data acquisition we are upgrading the CPU-based COPPER system with a PCIe40 board carrying Arria 10 FPGA. Since one of the main functionalities of the new system is event building in FPGA, the read-out system must be synchronized with the front-end electronics. This task is performed by the bidirectional trigger timing distribution system. During system commissioning, we prepared several versions of the interface to this system. In the initial version of the interface, we ported the code from Xilinx FPGAs to Arria 10. This revision also introduces monitoring of the status for multiple channels and a ring buffer to distribute trigger information to all channels in parallel. To improve stability under external noise, we implemented a clock-data recovery using an independent on-board oscillator as a reference clock in the next revision of the interface. We are also developing a version utilizing a high-speed serial transceiver to replace CAT-7 RJ45 cables with optical fibers. The system commissioning started in 2021 with a few detectors and will be completed after the long shutdown 1 of SuperKEKB in 2023. In this paper, we present the architectures of the interface to the trigger timing system implemented in the PCIe40 board and the system performance in the experiment.

Event Building Algorithm in a Distributed Stream Processing Data Acquisition Platform: D-Matrix

Event building is an important task for data acquisition (DAQ) systems in high-energy physics experiments. D-Matrix is designed as a generic distributed stream processing DAQ platform, containing hardware description language (HDL)-based modules and software modules. In the D-Matrix system, the data transfer is abstracted into the form of a stream, and the event building is implemented through a cascade of various reusable stream processing modules. These stream processing modules will determine their work modes automatically based on a set of attributes, which describe the abstracted data stream characteristics, such as spatiotemporal organization and granularity. This article introduces the event building architecture, the physics data structure abstraction, and the algorithmic implementation of specific event building modules in the D-Matrix system. An application of the D-Matrix event building architecture in the CSR external-target experiment is presented in the end.

Data Acquisition Upgrade at KOTO

KOTO is building a new data acquisition system which is a pyramid style architecture, composed of all new homemade optical fiber centers modules with high-speed ports that gather and build the event from over 4000 channels. It will be able to handle up to 30,000 triggers per second with minimal loss in the near future, which is a factor of six more than current rates. This will allow KOTO to handle a higher intensity KL beam, as well as open up the possibility of adding new physics triggers such as one for KL → π 0 e + e −. Additionally, the new architecture simplifies our data handling and allows for greater flexibility in adding late level trigger conditions since the event building is done before the data is sent to the server. This system is robust, flexible and easily scalable in the future and provides a foundation for moving forward to KOTO II.

The DAQ and clock distribution system of CMS MIP Timing Detector

The Compact Muon Solenoid (CMS) detector at the CERN Large Hadron Collider (LHC) is undergoing an extensive Phase-II upgrade program to cope with the challenging conditions of the High-Luminosity LHC (HL-LHC). A new timing detector is designed to measure minimum ionizing particles (MIPs) with a time resolution of 30–60 ps during the entire HL-LHC phase. A common data acquisition (DAQ) system will collect data from readout chips, reconstruct timing information, and send data to the event builder. The MIP timing detector (MTD) DAQ system is built around the state-of-the-art ATCA-form-factor Serenity board with two high-speed FPGAs. The precision clock is synchronized to the LHC collision rate of 40 MHz and is received at the subsystem and transmitted to the detector via high-speed data links. The detector system with a full-readout chain has been tested using prototypes of the DAQ, on-detector electronics, and sensors, showing that the system can successfully achieve timing resolution below 30 ps.

Talmy’s typology in serializing languages: Variations on a VP

Two types of resultative constructions that are unevenly distributed across languages (Talmy 2000) can be identified based on the lexicalization of manner and result meaning in the verbal main predicate: resultative secondary predication lexicalizes the manner component, while so called means constructions lexicalize the result component instead. However, this typology has been based primarily on non-serializing languages such as English and certain Romance varieties, in which the secondary manner or result predicate is necessarily non-verbal. This contrasts with resultatives in serializing languages, in which both the manner and result component are realized by verbal predicates, making it difficult to determine the underlying syntactic status of the respective predicates. By investigating the morphosyntactic and semantic properties of resultative serial verb constructions (RSVCs) in two serializing languages, Mandarin and Samoan, I argue that RSVCs are neither a uniform nor special phenomenon (contra Larson 1991; Slobin 2004), but show the same split observed in non-serializing languages. This observation has further implications on a unified configurational analysis of manner and result meaning within a syntactic account of event and argument structure building.

A Trigger Interface Board for the Large and Medium Sized telescopes of the Cherenkov Telescope Array

This paper presents the Trigger Interface Board (TIB), a system in charge of managing the trigger and clock signals in the cameras for the Large- and Medium-Sized telescopes in the Cherenkov Telescope Array (CTA). The TIB includes interfaces with the neighboring telescopes and with other subsystems in the camera. It receives trigger pulses from different origins and decides when those trigger signals should cause the camera readout. The decision is based on slow control parameters, the TIB internal state, the busy state of the camera, and the coincidence in time with other telescopes in a hardware stereo trigger scheme. Depending on the trigger origin, the TIB also generates an event data fragment, that is delivered to the camera Event Builder as part of the full raw event data. Additionally, it provides camera data-taking information to CTA central instances for monitoring purposes. The design of the TIB is based on two main components, namely an FPGA and a microcomputer. This structure provides fast digital processing capabilities, high-level software features for slow control and TCP/IP communication, and, especially, great flexibility to develop functionalities by working on the firmware and software levels separately.

Design of Data Acquisition System for TOF Detectors in the CEE Experiment

The CSR External-target Experiment(CEE) system under development may reach an average data rate of 2 GB/s after being triggered. In response to the design requirements of the entire spectrometer, a generic stream processing data acquisition architecture D-Matrix is put forward in the article, and a specific scheme of D-Matrix in the Time-of-Flight detector(TOF) subsystem is proposed. The DAQ of the TOF system uses high-speed serial communication methods such as optical fiber and PCIe bus to complete data transmission from Front-End Electronics(FEE) to the servers, and deploys and cascades stream processing nodes with a unified interface to complete event building, command routing, Real-time status monitoring and displaying and data storage tasks. In addition, the DAQ of the TOF system has strong versatility in hardware and logic design, it also can be compatible with other sub-detector systems, and can be further applied to the global system of CEE experiments.

Commissioning and testing of pre-series triple GEM prototypes for CBM-MuCh in the mCBM experiment at the SIS18 facility of GSI

Large area triple GEM chambers will be employed in the first two stations of the MuCh system of the CBM experiment at the upcoming Facility for Antiproton and Ion Research FAIR in Darmstadt/Germany. The GEM detectors have been designed to take data at an unprecedented interaction rate (up to 10 MHz) in nucleus-nucleus collisions in CBM at FAIR. Real-size trapezoidal modules have been installed in the mCBM experiment and tested in nucleus-nucleus collisions at the SIS18 beamline of GSI as a part of the FAIR Phase-0 program. In this report, we discuss the design, installation, commissioning, and response of these GEM modules in detail. The response has been studied using the self-triggered readout electronics. In free-streaming data, the first attempt on an event building based on the timestamps of hits has been carried out, resulting in the observation of clear spatial correlations between the GEM modules in the mCBM setup for the first time. Accordingly, a time resolution of ∼15 ns have been obtained for the GEM detectors. Gain uniformity & cluster characteristics have also been studied.

Data-Taking Network for COMET Phase-I

An experiment to search for mu-e conversion named COMET is being constructed at J-PARC. The experiment will be carried out using a two-stage approach of Phase-I and Phase-II. The data taking system of Phase-I is developed based on common network technology. The data taking system consists of two kinds of networks. One is a front-end network. Its network bundles around twenty front-end devices that have a 1-Gb optical network port. And a front-end computer accepts data from the devices via its network. The other is a back-end network that collects all event fragments from the front-end computers using a 10-Gb network. We used a low price 1Gb/10Gb optical network switch for the front-end network. And direct connection between the front-end PC and an event building PC using 10-Gb optical network devices was used for the back-end network. The event building PC has ten 10-Gb network ports. And each network port of the event building PC is connected to the front-end PC's port without using a network switch. We evaluated data taking performance with an event building on these two kinds of networks. The event building throughput of the front-end network achieved 337 MiB/s. And the event building throughput of the back-end networks achieved 1.2 GiB/s. It means that we could reduce the construction cost of the data taking network using this structure without deteriorating performance. Moreover, we evaluated the writing speed of the local storage RAID disk system connected to a back-end PC by a SAS interface, and a long-distance network copy from the experiment location to the lasting storage.

New Software-Based Readout Driver for the ATLAS Experiment

To maintain sensitivity to new physics in the coming years of Large Hadron Collider (LHC) operations, A Toroidal LHC ApparatuS (ATLAS) collaboration has been working on upgrading a portion of the front-end (FE) electronics and replacing some parts of the detector with new devices that can operate under the much harsher background conditions of future LHC runs. The legacy FE of the ATLAS detector sent data to the data acquisition (DAQ) system via the so-called Read Out Drivers (RODs) custom-made VMEbus boards devoted to data processing, configuration, and control. The data were then received by the Read Out System (ROS), which was responsible for buffering them during the High-Level Trigger (HLT) processing. From Run 3 onward, all new trigger and detector systems will be read out using new components, replacing the combination of the ROD and the ROS. This new path will feature an application called the Software Read Out Driver (SW ROD), which will run on a commodity server receiving FE data via the Front-End Link eXchange (FELIX) system. The SW ROD will perform event fragment building and buffering as well as serving the data on request to the HLT. The SW ROD application has been designed as a highly customizable high-performance framework providing support for detector-specific event building and data processing algorithms. The implementation that will be used for Run 3 of the LHC is capable of building event fragments at a rate of 100 kHz from an input stream consisting of up to 120 MHz of individual data packets. This document will cover the design and the implementation of the SW ROD application and will present the results of performance measurements executed on the server models selected to host SW ROD applications during Run 3.

A Generic Streaming Software Platform Design for High-Energy Physics Data Acquisition Systems

The data acquisition (DAQ) system is a vital component in the high-energy physics experiment. To reduce duplication of work during development and improving efficiency, D-Matrix-S, a generic platform, has been developed as the software part of a software and hardware combination streaming DAQ system. D-Matrix-S is based on stream processing. Its philosophy is to abstract different tasks in the stream processing and encapsulate them as reusable modules with standard intermodule connectors. Furthermore, customizable DAQ systems could be built by calling appropriate modules and connecting them freely. D-Matrix-S adopts a layered design method during its development. The underlying construction is divided into stream source layers, stream transport layer, and stream processing layer. In the source layer, all types of data sources are streamed to help build a unified transmission model. The transport layer in a software environment achieves efficient and flexible stream transmission through dual shared memory space and plug-in transceivers. As for the processing layer, some common services for high-energy physics experiments such as configurable, generic event building, and multiroot, tree-like slow controlling are provided by the combination of standard modules. This article presents the design philosophy of a generic DAQ system and optimization made to improve flexibility and provides an application of the D-Matrix-S platform in the end.

Simple and Scalable Streaming: The GRETA Data Pipeline

The Gamma Ray Energy Tracking Array (GRETA) is a state of the art gamma-ray spectrometer being built at Lawrence Berkeley National Laboratory to be first sited at the Facility for Rare Isotope Beams (FRIB) at Michigan State University. A key design requirement for the spectrometer is to perform gamma-ray tracking in near real time. To meet this requirement we have used an inline, streaming approach to signal processing in the GRETA data acquisition system, using a GPU-equipped computing cluster. The data stream will reach 480 thousand events per second at an aggregate data rate of 4 gigabytes per second at full design capacity. We have been able to simplify the architecture of the streaming system greatly by interfacing the FPGA-based detector electronics with the computing cluster using standard network technology. A set of highperformance software components to implement queuing, flow control, event processing and event building have been developed, all in a streaming environment which matches detector performance. Prototypes of all high-performance components have been completed and meet design specifications.