Read-Out Drivers Research Articles

Operational Experience and Evolution of the ATLAS Tile Hadronic Calorimeter Read-Out Drivers

TileCal is the central hadronic calorimeter of the ATLAS experiment at the Large Hadron Collider (LHC). It is a sampling detector where scintillating tiles are embedded in steel absorber plates. The tiles are grouped forming cells, which are read-out on both sides by photomultiplier tubes (PMTs). The PMT digital samples are transmitted to the Read-Out Drivers (ROD) located in the back-end system for the events accepted by the Level 1 trigger system. The ROD is the core element of the back-end electronics and it represents the interface between the front-end electronics and the ATLAS overall Data AcQuisition (DAQ) system. It is responsible for energy and time reconstruction, trigger and data synchronization, busy handling, data integrity checking and lossless data compression. The TileCal ROD is a standard 9U VME board equipped with DSP based Processing Units mezzanine cards. A total of 32 ROD modules are required to read-out the entire TileCal detector. Each ROD module has to process the data from up to 360 PMTs in real time in less than 10 microseconds. The commissioning of the RODs was completed in 2008 before the first LHC collisions. Since then, several hardware and firmware updates have been implemented to accommodate the RODs to the evolving ATLAS Trigger and DAQ conditions adjusted to follow the LHC parameters. The initial ROD system, the different updates implemented and the operational experience during the LHC Run 1 and Run 2 are presented.

New Software-Based Readout Driver for the ATLAS Experiment

To maintain sensitivity to new physics in the coming years of Large Hadron Collider (LHC) operations, A Toroidal LHC ApparatuS (ATLAS) collaboration has been working on upgrading a portion of the front-end (FE) electronics and replacing some parts of the detector with new devices that can operate under the much harsher background conditions of future LHC runs. The legacy FE of the ATLAS detector sent data to the data acquisition (DAQ) system via the so-called Read Out Drivers (RODs) custom-made VMEbus boards devoted to data processing, configuration, and control. The data were then received by the Read Out System (ROS), which was responsible for buffering them during the High-Level Trigger (HLT) processing. From Run 3 onward, all new trigger and detector systems will be read out using new components, replacing the combination of the ROD and the ROS. This new path will feature an application called the Software Read Out Driver (SW ROD), which will run on a commodity server receiving FE data via the Front-End Link eXchange (FELIX) system. The SW ROD will perform event fragment building and buffering as well as serving the data on request to the HLT. The SW ROD application has been designed as a highly customizable high-performance framework providing support for detector-specific event building and data processing algorithms. The implementation that will be used for Run 3 of the LHC is capable of building event fragments at a rate of 100 kHz from an input stream consisting of up to 120 MHz of individual data packets. This document will cover the design and the implementation of the SW ROD application and will present the results of performance measurements executed on the server models selected to host SW ROD applications during Run 3.

FELIX: the Detector Interface for the ATLAS Experiment at CERN

The Front-End Link eXchange (FELIX) system is an interface between the trigger and detector electronics and commodity switched networks for the ATLAS experiment at CERN. In preparation for the LHC Run 3, to start in 2022, the system is being installed to read out the new electromagnetic calorimeter, calorimeter trigger, and muon components being installed as part of the ongoing ATLAS upgrade programme. The detector and trigger electronic systems are largely custom and fully synchronous with respect to the 40.08 MHz clock of the Large Hadron Collider (LHC). The FELIX system uses FPGAs on server-hosted PCIe boards to pass data between custom data links connected to the detector and trigger electronics and host system memory over a PCIe interface then route data to network clients, such as the Software Readout Drivers (SW ROD), via a dedicated software platform running on these machines. The SW RODs build event fragments, buffer data, perform detector-specific processing and provide data for the ATLAS High Level Trigger. The FELIX approach takes advantage of modern FPGAs and commodity computing to reduce the system complexity and effort needed to support data acquisition systems in comparison to previous designs. Future upgrades of the experiment will introduce FELIX to read out all other detector components.

The ATLAS Diamond Beam Monitor: Luminosity detector at the LHC

After the first three years of the LHC running, the ATLAS experiment extracted its pixel detector system to refurbish and re-position the optical readout drivers and install a new barrel layer of pixels. The experiment has also taken advantage of this access to install a set of beam monitoring telescopes with pixel sensors, four each in the forward and backward regions. These telescopes are based on chemical vapor deposited (CVD) diamond sensors to survive in this high radiation environment without needing extensive cooling. This paper describes the lessons learned in construction and commissioning of the ATLAS Diamond Beam Monitor (DBM). We show results from the construction quality assurance tests and commissioning performance, including results from cosmic ray running in early 2015.

A parallel FPGA implementation for real-time 2D pixel clustering for the ATLAS Fast Tracker Processor

The parallel 2D pixel clustering FPGA implementation used for the input system of the ATLAS Fast TracKer (FTK) processor is presented. The input system for the FTK processor will receive data from the Pixel and micro-strip detectors from inner ATLAS read out drivers (RODs) at full rate, for total of 760Gbs, as sent by the RODs after level-1 triggers. Clustering serves two purposes, the first is to reduce the high rate of the received data before further processing, the second is to determine the cluster centroid to obtain the best spatial measurement. For the pixel detectors the clustering is implemented by using a 2D-clustering algorithm that takes advantage of a moving window technique to minimize the logic required for cluster identification. The cluster detection window size can be adjusted for optimizing the cluster identification process. Additionally, the implementation can be parallelized by instantiating multiple cores to identify different clusters independently thus exploiting more FPGA resources. This flexibility makes the implementation suitable for a variety of demanding image processing applications. The implementation is robust against bit errors in the input data stream and drops all data that cannot be identified. In the unlikely event of missing control words, the implementation will ensure stable data processing by inserting the missing control words in the data stream. The 2D pixel clustering implementation is developed and tested in both single flow and parallel versions. The first parallel version with 16 parallel cluster identification engines is presented. The input data from the RODs are received through S-Links and the processing units that follow the clustering implementation also require a single data stream, therefore data parallelizing (demultiplexing) and serializing (multiplexing) modules are introduced in order to accommodate the parallelized version and restore the data stream afterwards. The results of the first hardware tests of the single flow implementation on the custom FTK input mezzanine (IM) board are presented. We report on the integration of 16 parallel engines in the same FPGA and the resulting performances. The parallel 2D-clustering implementation has sufficient processing power to meet the specification for the Pixel layers of ATLAS, for up to 80 overlapping pp collisions that correspond to the maximum LHC luminosity planned until 2022.

Software design of the ATLAS Muon Cathode Strip Chamber ROD

The ATLAS Cathode Strip Chamber system consists of two end-caps with 16 chambers each. The CSC Readout Drivers (RODs) are purpose-built boards encapsulating 13 DSPs and around 40 FPGAs. The principal responsibility of each ROD is for the extraction of data from two chambers at a maximum trigger rate of 75 KHz. In addition, each ROD is in charge of the setup, control and monitoring of the on-detector electronics. This paper introduces the design of the CSC ROD software. The main features of this design include an event flow schema that decentralizes the different dataflow streams, which can thus operate asynchronously at its own natural rate; an event building mechanism that associates data transferred by the asynchronous streams belonging to the same event; and a sparcification algorithm that discards uninteresting events and thus reduces the data occupancy volume. The time constraints imposed by the trigger rate have made paramount the use of optimization techniques such as the curiously recurrent template pattern and the programming of critical code in assembly language. The behaviour of the CSC RODs has been characterized in order to validate its performance.

The TileCal Optical Multiplexer Board 9U

Abstract TileCal is the hadronic calorimeter of the ATLAS experiment at LHC/CERN. The system contains roughly 10,000 channels of read-out electronics, whose signals are gathered and digitized in the front-end electronics and then transmitted to the counting room through two redundant optical links. Then, the data is received in the back-end system by the Optical Multiplexer Board (OMB) 9U which performs a CRC check to the redundant data to avoid Single Event Upsets errors. A real-time decision is taken on the event-to-event basis to transmit single data to the Read-Out Drivers (RODs) for processing. Due to the low dose level expected during the first years of operations in ATLAS it was decided not to use a redundant system and currently the front-end electronics is directly connected to the RODs. However, the increasing luminosity of the LHC will force to use the redundant read-out and the OMB system will be installed. Moreover, the OMB can be used as a ROD injector to emulate the front-end electronics for ROD software tests during detector maintenance periods taking advantage of its location in the data acquisition chain. First we will give a detailed description of the main components of the board and the different operation modes. Then, the production and qualification tests will be explained including a detailed description of the test-bench, software and validation protocols.

Calibration and signal reconstruction in the ATLAS Tile Hadronic calorimeter

TileCal is the central hadronic calorimeter of the ATLAS detector which is one of the four experiments installed at the Large Hadron Collider (LHC) at CERN. In order to calibrate the full read-out path in TileCal different calibration systems are present. The full calibration process relies on three subsystems: the Charge Injection System (CIS), the Laser System, and the Cesium. The signal reconstruction determines the amplitude and the time of the deposited energy. In TileCal the Optimal Filter (OF) algorithm is used for this purpose; in particular the signal is reconstructed in the Read-Out Drivers (ROD) using the Digital Signal Processor (DSP).

The ATLAS tile calorimeter ROD injector and multiplexer board

The ATLAS Tile Calorimeter is a sampling detector composed by cells made of iron-scintillator tiles. The calorimeter cell signals are digitized in the front-end electronics and transmitted to the Read-Out Drivers (RODs) at the first level trigger rate. The ROD receives triggered data from up to 9856 channels and provides the energy, phase and quality factor of the signals to the second level trigger. The back-end electronics is divided into four partitions containing eight RODs each. Therefore, a total of 32 RODs are used to process and transmit the data of the TileCal detector. In order to emulate the detector signals in the production and commissioning of ROD modules a board called ROD Injector and Multiplexer Board (RIMBO) was designed. In this paper, the RIMBO main functional blocks, PCB design and the different operation modes are described. It is described the crucial role of the board within the TileCal ROD test-bench in order to emulate the front-end electronics during the validation of ROD boards as well as during the evaluation of the ROD signal reconstruction algorithms. Finally, qualification and performance results for the injection operation mode obtained during the Tile Calorimeter ROD production tests are presented.

ATLAS TileCal Read Out Driver production

The production tests of the 38 ATLAS TileCal Read Out Drivers (RODs) arepresented in this paper. The hardware specifications and firmware functionality of the RODsmodules, the test-bench and the test procedure to qualify the boards are described. Finally theperformance results, the temperature studies and high rate tests are shown and discussed.

Algorithms for the ROD DSP of the ATLAS Hadronic Tile Calorimeter

In this paper we present the performance of two algorithms currently running in the Tile Calorimeter Read-Out Driver boards for the commissioning of ATLAS. The first algorithm presented is the so called Optimal Filtering. It reconstructs the deposited energy in the Tile Calorimeter and the arrival time of the data. The second algorithm is the MTag which tags low transverse momentum muons that may escape the ATLAS muon spectrometer first level trigger.Comparisons between online (inside the Read-Out Drivers) and offline implementations are done with an agreement around 99% for the reconstruction of the amplitude using the Optimal Filtering algorithm and a coincidende of 93% between the offline and online tagged muons for the MTag algorithm. The processing time is measured for both algorithms running together with a resulting time of 59.2 μs which, although above the 10 μs of the first level trigger, it fulfills the requirements of the commissioning trigger ( ∼ 1 Hz). We expect further optimizations of the algorithms which will reduce their processing time below 10 μs.

Optical readout in a multi-module system test for the ATLAS pixel detector

The innermost part of the ATLAS experiment at the LHC, CERN, will be a pixel detector, which is presently under construction. The command messages and the readout data of the detector are transmitted over an optical data path. The readout chain consists of many components which are produced at several locations around the world, and must work together in the pixel detector. To verify that these parts are working together as expected a system test has been built up. It consists of detector modules, optoboards, optical fibres, Back of Crate cards, Readout Drivers, and control computers. In this paper, the system test setup and the operation of the readout chain are described. Also, some results of tests using the final pixel detector readout chain are given.

The read-out driver for the ATLAS MDT muon precision chambers

The 1172 Monitored Drift Tube (MDT) precision chambers of the muon spectrometer of the ATLAS experiment at the LHC will be read out by 204 MDT Read-Out Drivers (MRODs). The MRODs receive event data via 1136 optical links (up to 6 per MROD), build event fragments at a maximum rate of 100 kHz, output these to the ATLAS data-acquisition system and take care of monitoring and error checking, handling and flagging. The evolution of the design of the MROD (a 9U VME64 module in which this functionality is implemented using FPGAs and ADSP-21160 Digital Signal Processors programmed in C++) is reviewed and critical issues are discussed.

Transistorized 10-Mc Decade Scaler

A space-saving, standard, transistorized 10-Mc decade scaler of modular construction has been developed to replace assorted earlier types of scalers. For high reliability, silicon planar transistors are used throughout, except for readout drivers. Two complete scalers are contained in a standard, rack mounting frame of 3-1/2-inch front panel height. A power supply module on the rear panel powers both scalers. Each scaler comprises nine modules. One module contains a tunnel diode pulse height discriminator with a threshold variable from 0.1 to 2 volts, and temperature coefficient less than 1 mv/°C. Another module contains a diode gate for gating the discriminator output, and the reset circuitry for the scaler modules. A third module contains a 10-Mc decade counting unit (DCU). Six identical 1-Mc DCU's complete the scaler. Ten-digit front panel readout of the DCU's is provided by neon lights in column configuration. External DCU readout is provided by four-line binary-coded decimal signals in 1-2-4-8 code.