Event Building Research Articles

The CMS event builder demonstrator based on Myrinet

The data acquisition system for the CMS experiment at the Large Hadron Collider (LHC) will require a large and high performance event building network. Several switch technologies are currently being evaluated in order to compare different architectures for the event builder. One candidate is Myrinet. This paper describes the demonstrator which has been set up to study a small-scale (8/spl times/8) event builder based on a Myrinet switch. Measurements are presented on throughput, overhead and scaling for various traffic conditions. Results are shown on event building with a push architecture.

Event-building and PC farm based level-3 trigger at the CDF experiment

In the technical design report the event building process at Fermilab's CDF experiment is required to function at an event rate of 300 events/sec. The events are expected to have an average size of 150 kBytes (kB) and are assembled from fragments of 16 readout locations. The fragment size from the different locations varies between 12 kB and 16 kB. Once the events are assembled they are fed into the Level-3 trigger which is based on processors running programs to filter events using the full event information. Computing power on the order of a second on a Pentium II processor is required per event. The architecture design is driven by the cost and is therefore based on commodity components: VME processor modules running VxWorks for the readout, an ATM switch for the event building, and Pentium PCs running Linux as an operation system for the Level-3 event processing. Pentium PCs are also used to receive events from the ATM switch and further distribute them to the processing nodes over multiple 100 Mbps Ethernets. Studies with a prototype of up to 10 VME readout modules and up to 4 receiving PCs are presented. This system is also a candidate for the CMS experiment at CERN.

The general purpose data acquisition system MBS

The GSI data acquisition system MBS (Multi Branch System) has become, since its introduction in 1994, the standard system at GSI (37 local and 9 portable systems). It is also used at numerous outside laboratories (29 systems). Compared to phase I of the system, the current version MBS 3.0 offers four new major enhancements: 1. Support of event builder hierarchies. 2. Support of message oriented busses and networks. 3. Multipleparallel event builders. 4. A multi-threaded graphical user interface (Java). The software supports various hardware setups and topologies.

Flexible processing framework for online event data and software triggering

The BABAR experiment is the particle detector at the new PEP-II factory facility at the Stanford Linear Accelerator Center. The experiment's goal is the detailed study of the fundamental phenomenon of charge-parity symmetry breaking in B meson decays. Within the detector's online system, the Online Event Processing subsystem (OEP) provides for near real-time processing of data delivered from the data acquisition system's event builder. It supports four principal tasks: the final software trigger, rapid-feedback fast data quality monitoring, event displays, and the final steps in online calibrations. In order to accommodate the expected 2000 Hz event rate, the OEP subsystem runs on computing farm, receiving events on each node into a shared memory buffer system and coordinating processes performing analysis on this data. Analysis tasks are assigned administrative characteristics providing for protection of data integrity and ensuring that indispensable tasks such as triggering are always performed, while others are performed as resources are available. Data access in OEP uses BABAR's standard offline reconstruction framework. Thus, notably, the Level 3 trigger software could be developed in advance in the full offline simulation environment and yet be suitable for near-real-time online running with minimal changes.

MAR/sub a/BOOU - a MBS and ROOT based online/offline utility

MAR/sub a/BOOU is a system for data acquisition and evaluation developed at the Tandem Accelerator Laboratory of the Universities of Munich. It consists of a front-end system for data readout, event building, and data transport based on the Multi Branch System MBS, and a back-end system responsible for setup, run control, histogramming, data analysis, and data storage written within the ROOT framework. The ROOT part includes a GUI to control the MBS front-end, a histogram presenter to visualize the data, and class libraries to describe the experiment.

Data acquisition system for the Belle experiment

The Belle II experiment, a new generation B-factory experiment at KEK aiming to search for new physics beyond the standard model, is scheduled to start in 2016. The maximum trigger rate is expected to reach as high as 30 kHz with the raw event size of more than 1MB, and the data acquisition is a big challenge. The Belle II DAQ system is a synchronous system based on a pipelined trigger flow control. Data from all sub detectors except for the Pixel Detector (PXD) digitized by the front-end are transferred to unified readout modules (COPPER) via a high speed optical link (Belle2link). The data are then collected by the readout PCs via Ethernet and the event building is performed in multiple steps. The built events are fed into High Level Trigger (HLT) to perform a software trigger. For processing data of PXD, an FPGA-based readout system (ONSEN) is used. ONSEN performs a data reduction by exploiting the feed-back of track information obtained by HLT. In order to test the full functionality of our DAQ design, we developed a down-scaled version of the Belle II DAQ system. The system was first tested with dummy data to check its basic performance and then used in the combined beam test of Silicon Vertex Detector (SVD) and PXD in the DESY test beam performed in January 2014. A slow control and real time data monitoring are also implemented in this test. All components of this complex readout chain successfully cooperated to collect the beam data. We present the BelleII DAQ system and the current status of the development.

The NA48 event-building PC farm

The NA48 experiment at the CERN SPS aims to measure the parameter Re(/spl epsiv/'//spl epsiv/) of direct CP violation in the neutral kaon system with an accuracy of 2x10/sup -4/. Based on the requirements of: high event rates (up to 10 kHz) with negligible dead time; support for a variety of detectors with very wide variation in the number of readout channels; data rates of up to 150 MByte/s sustained over the beam burst; level-3 filtering and remote data logging in the CERN computer center; the collaboration has designed and built a modular pipelined data flow system with 40 MHz sampling rate. The architecture combines custom-designed components with commercially available hardware for cost effectiveness and flexibility. To increase the available data bandwidth and to add filtering and monitoring capabilities, the original custom-built event builder hardware has been replaced by a farm of 24 Intel PentiumII based PCs running the Linux operating system during the shutdown between the 1997 and 1998 data taking periods. During the data taking period 1998 the system has been successfully operated taking ca. 70 Terabyte of data.

Time-slot interchanger for self-routing event building

An event builder based on a sorting network has been developed. The key components of the event builder are a comparator used as a node of the network, and a time slot interchanger (TSI), which arranges input data fragments. Since the data fragments determine their path in the network, no complicated software control is needed for the event builder. The sorting network has scalability for the size of the readout system, because of its recursive characteristics and capability of downsizing. In order to deduce these features, a time slot interchanger (TSI) was newly developed. The throughput of the data and scalability were examined for various speeds of the system clocks and the number of input channels. The TSI has sufficient scalability of the speed and the network-size.

Experience with fibre channel in the environment of the ATLAS DAQ prototype “−1” project

Fibre Channel equipment has been evaluated in the environment of the ATLAS DAQ prototype “−1”. Fibre Channel PCI and PMC cards have been tested on PowerPC-based VME processor boards running LynxOS and on Pentium-based personal computers running Windows NT. The performance in terms of overhead and bandwidth has been measured in point-to-point, arbitrated loop and fabric configuration with a Fibre Channel switch. The possible use of the equipment for event building in the ATLAS DAQ prototype “−1” has been studied.

Dead-time free pixel readout architecture for ATLAS front-end IC

A low-power, sparse-scan, readout architecture has been developed for the ATLAS pixel front-end electronics. The architecture supports a dual discriminator and extracts the time over threshold (TOT) information along with a 2-D spatial address of the hits and associates them with a unique 7-bit beam crossing number. The IC implements level-1 trigger filtering along with event building (grouping together all hits in a beam crossing) in the end of column (EOC) buffer. The events are transmitted over a 40 MHz serial data link with the protocol supporting buffer overflow handling by appending error flags to events. This mixed-mode full custom IC is implemented in 0.8 /spl mu/ HP process to meet the requirements for the pixel readout in the ATLAS inner detector. The circuits have been tested and the IC has been found to provide dead-time-less ambiguity-free readout at 40 MHz data rate.

The charged trigger system of NA48 at CERN

The NA48 charged trigger is a mixed hardware and software real-time processing system intended to detect the interesting configurations of K0 charged decays. It achieves real-time event building, track reconstruction and kinematics computation on drift chamber data at an event rate of 100kHz and within a maximum decision latency of 100μs. The system uses data driven, FPGA-based coordinate builders, a hardware event builder based on a crossbar switch, and a farm of up to 16 event processors for its software part. It has been installed and operated at CERN since 1995. After a description of the constraints and architecture of the various subsystems, the paper gives an account of the results and performance of the system based on the 1996/1997 runs. More specifically, the replacement of the present DSP-based implementation of the processing farm by RISC processors will be discussed.

Self-routing event building on sorting network

Event building system based on sorting network has been developed and discussed. The event builder comprises a comparator with subsidiary functions as a network node, FIFOs as a time slot interchanger at the input stage and data buffers at the output stage. The event building system gathers data without any complex controls by the software. The throughput of the data and scalability were examined for various speeds of the system clocks. The system has a scalability of the speed for the network-size. Though the throughput is limited by the speed of the device we chose, the speed can be increased by choosing a faster device.

The second generation of DAQ-Systems at COSY

At the Julich Cooler Synchrotron COSY several experiments are running based on a modular, flexible system design in hardware and software. Future data acquisition experiments should use PC-technologies as far as possible. For the intelligent units initially passive backplanes with Single Bord Computers (SBCs) have been chosen as defined by the PCI Industrial Computer Manufacturing Group (PICMG). For future applications the CompactPCI is considered which combines the IBM PC electrical and software standards with the Eurocard mechanical standards. Connection of the front-end modular systems (CAMAC, VME, FASTBUS) is performed by transparent controllers with PCI compliant interfaces, already available on the market or home made developments. As far as operating systems are concerned UNIX like systems are used. An object oriented, client-server based Experiment Messaging System (EMS) establishes the communication on the application layer between the experiment control, the event builder and different front-end systems. The design criteria for the second generation DAQ-Systems, a first experiment layout, interconnectivity questions and necessary component developments will be described, as well as the problems we have found turning a PC into a diskless system. Important results of performance measurements concerning the DAQ requirements will be given.

Analysis of nonuniform data flow over switch type event builders

Switch topologies are discussed in terms of graph theory. It is shown that nonblocking circuit switches can handle any requested data flow if the switch-controller is intelligent enough. It is also shown that most event builders do not need nonblocking switches to get high performance.

Data collection modules for the PHENIX experiment

The data acquisition (DAQ) system for the PHENIX experiment is designed as a pipeline system with simultaneous triggering and readout. The maximum average level-1 (LVL1) trigger rate is 25 kHz. The DAQ system consists of Front-End Modules (FEMs), a level-1 (LVL1) trigger, data collection modules (DCMs), timing systems, slow controllers and an event builder (EVE). The data collection modules have the responsibility of collecting uncompressed LVL1 trigger event fragments from the FEMs. The DCMs provide buffering for up to five LVL1 events. The DCMs also perform zero suppression, error checking, data reformatting and outputting data to the event builder. In addition to the FEM data, the DCMs also receive primitives from LVL1 trigger system. These primitives are used for alignment checking on the FEM data packet. Additional trigger primitives can also be generated together with the FEM Data. The DCM is hosted in a VME crate. VME is used as a means for maintenance and slow control. Data collection within the crate is done through a private data-way.

Global traffic control system for unbalanced data flow on nonsquare event builder in BELLE experiment

It has been stated that an unbalanced data flow cannot be handled by blocking switches such as a barrel shifter. It is not true for a square switch, which has the same number of destination nodes as source nodes. However, if the switch is not square, a simple barrel shifter is not adequate. We have studied the minimal switch configuration for the event builder system in a BELLE experiment in KEK. The performance in the actual experiment was measured.

The FINUDA data acquisition system a C/C++ DAQ based on ROOT as event monitor

A parallel scalable Data Acquisition based on VME, has been developed to be used in the FINUDA experiment, scheduled to run at the DA/spl Phi/NE machine at Frascati in 1998. The acquisition software runs on embedded RTPC 8067 processors using the LynxOS operating system. The read-out of event fragments is coordinated by a suitable trigger supervisor. Data read by different controllers are transported via dedicated bus to a Global Event Builder running on a PowerPC machine. Commands from and to VME processors are sent via socket based network protocols. The Online Monitor, froot, is based on an object oriented approach in the framework of the ROOT package.

The charged trigger system of NA48 at CERN

The NA48 charged trigger is a mixed hardware and software real time processing system intended to detect the interesting configurations of K/sup 0/ charged decays. It achieves real-time event building, track reconstruction and kinematics computation on drift chamber data at an event rate of 100 kHz and within a maximum decision latency of 100 /spl mu/s. The system uses data driven, FPGA-based coordinate builders, a hardware event builder based on a crossbar switch, and a farm of up to 16 event processors for its software part. It has been installed and operated at CERN since 1995. After a description of the constraints and architecture of the various subsystems, the paper will give an account of the results and performance of the system based on the 1996/1997 runs. More specifically, the replacement of the present DSP-based implementation of the processing farm by RISC processors will be discussed.

An overall MWPC-readout system based on chamber mounted ASICs

At the Julich Cooler Synchrotron, COSY, the magnetic spectrometer ANKE (Apparatus for Studies of Nucleon and Kaon Ejectils) is installed. Reaction products emitted by an internal target in the forward direction are measured. About 11000 channels are read out. This requirement justified the design of a VLSI based readout system. For this, an analog chip (RAL 118)-a low power high speed eight channel amplifier and discriminator-is used. Digital delay, zero suppression and address encoding are performed by the digital chip RAL 111 from the Rutherford Appleton Laboratory. A scalable and flexible array of chamber mounted front end boards may be adapted to instrument detectors of any size. Readout is performed by serializing hit addresses over a high speed optical link to a CAMAC readout module which serves for additional data reduction, subevent building and a number of test functions. Furthermore, subevent data are transferred using a front panel ECLbus (FERAbus*) to a crate controller with PCI-interface. An industrial PC with diskless UNIX-compatible operating system (NetBSD) is responsible for controlling the CAMAC crate, processing the event data and connecting to a central event builder.

PHENIX on-line distributed computing system architecture

PHENIX is one of the two large experiments at the Relativistic Heavy Ion Collider (RHIC) currently under construction at Brookhaven National Laboratory. The detector consists of 11 subdetectors, that are further subdivided into 29 units (“granules”) that can be operated independently, which includes simultaneous data taking with independent data streams and independent triggers.The detector has 250 000 channels and is read out by front-end modules, where the data is buffered in a pipeline while awaiting the level 1 trigger decision. Zero suppression and calibration is done after the level 1 accept in custom built data collection modules (DCMs) with DSPs before the data is sent to an event builder (design throughput of 2 Gb/sec) and higher level triggers.The On-line Computing Systems Group (ONCS) has two responsibilities. Firstly, it is responsible for receiving the data from the event builder, routing it through a network of workstations to consumer processes and archiving it at a data rate of 20 MB/sec.Secondly, it is also responsible for the overall configuration, control and operation of the detector and data acquisition chain, which comprises the software integration for several thousand custom built hardware modules. The software must furthermore support the independent operation of the above mentioned granules, which includes the coordination of processes that run in 60–100 VME processors and workstations.ONCS has adapted the Shlaer—Mellor Object Oriented Methodology for the design of the top layer software. CORBA is used as communication layer between the distributed objects, which are implemented as asynchronous finite state machines.We will give an overview of the PHENIX online system with the main focus on the system architecture, software components and integration tasks of the On-line Computing group ONCS and report on the status of the current prototypes.