VME Crate Research Articles

Cooling and Timing Tests of the ATLAS Fast TracKer VME Boards

The Fast TracKer (FTK) is an ATLAS trigger upgrade built for full-event, low-latency, high-rate tracking. The FTK core, made of 9U VME boards, performs the most demanding computational task. The associative memory board (AMB) serial link processor and the auxiliary card (AUX), plugged on the front and back sides of the same VME slot, constitute the processing unit (PU), which finds tracks using hits from eight layers of the inner detector. The PU works in pipeline with the second stage board (SSB), which finds 12-layer tracks by adding extra hits to the identified tracks. In the designed configuration, 16 PUs and four SSBs are installed in a VME crate. The high power consumption of the AMB, AUX, and SSB (respectively, of about 250, 70, and 160 W per board) required the development of a custom cooling system. Even though the expected power consumption for each VME crate of the FTK system is high compared with a common VME setup, the 8 FTK core crates will use ≈60 kW, which is just a fraction of the power and the space needed for a CPU farm performing the same task. We report on the integration of 32 PUs and eight SSBs inside the FTK system, on the infrastructures needed to run and cool them, and on the tests performed to verify the system processing rate and the temperature stability at a safe value.

Performance Evaluation of the ATLAS IBL Calibration

The Insertable-B-Layer (IBL) has recently been commissioned at the ATLAS Experiment, adding 12 million channels to the existing Pixel Detector. The front-end chips (FE-I4) are connected to newly designed readout hardware situated in a VME crate. In order to take data under uniform conditions, one needs to periodically tune the detector in short breaks between data-taking sessions to accommodate for radiation damage and aging effects. Tuning involves a variety of components, ranging from high-level steering and analysis software (PixLib) running on commodity hardware, to embedded components situated inside the VME crate that feature only a minimal or no operating system at all. Understanding the interactions between these components is key in debugging and optimizing the tuning procedures to become more efficient.We therefore implement an instrumentation framework aimed at all major components. It features a uniform interface to the user and is able to take instrumentation data with µs- precision. A central server application is used to gather the instrumentation data of a tuning session. It processes the data and saves it into an SQLite database for later analysis.

Prototype of time digitizing system for BESIII endcap TOF upgrade

The prototype of a time digitizing system for the BESIII endcap TOF (ETOF) upgrade is introduced in this paper. The ETOF readout electronics has a distributed architecture. Hit signals from the multi-gap resistive plate chamber (MRPC) are signaled as LVDS by front-end electronics (FEE) and are then sent to the back-end time digitizing system via long shield differential twisted pair cables. The ETOF digitizing system consists of two VME crates, each of which contains modules for time digitization, clock, trigger, fast control, etc. The time digitizing module (TDIG) of this prototype can support up to 72 electrical channels for hit information measurement. The fast control (FCTL) module can operate in barrel or endcap mode. The barrel FCTL fans out fast control signals from the trigger system to the endcap FCTLs, merges data from the endcaps and then transfers to the trigger system. Without modifying the barrel TOF (BTOF) structure, this time digitizing architecture benefits from improved ETOF performance without degrading the BTOF performance. Lab experiments show that the time resolution of this digitizing system can be lower than 20 ps, and the data throughput to the DAQ can be about 92 Mbps. Beam experiments show that the total time resolution can be lower than 45 ps.

Instrument front-ends at Fermilab during Run II**Work supported by Fermi Research Alliance, LLC under Contract No. DE-AC02-07CH11359 with the United States Department of Energy.

The optimization of an accelerator relies on the ability to monitor the behavior of the beam in an intelligent and timely fashion. The use of processor-driven front-ends allowed for the deployment of smart systems in the field for improved data collection and analysis during Run II. This paper describes the implementation of the two main systems used: National Instruments LabVIEW running on PCs, and WindRiver's VxWorks real-time operating system running in a VME crate processor.

BES III Time-of-Flight Readout System

A time-of-flight (TOF) readout system has been developed for a Beijing Spectrometer (BES III). This paper will present an overview of the entire TOF readout system design and its performance in the operation. In a BES III TOF detector, a total of 448 photomultiplier tubes (PMTs) have been installed in barrel and end-cap sections. The corresponding 448 channels' readout electronics aimed precisely to measure the arrival time of PMT signals from the TOF detector is mainly housed in two VME64xP crates, each crate containing 14 FEE modules, 14 FEE_rear modules, one Fast Control module (FastCtrl), one Clock module, and one Power-PC interface module. Due to the structure of the spectrometer, the two VME crates with 448 channels of readout electronics are placed in two locations; each is 18 m away from the TOF detector. In order to increase the capability of noise immunization and achieve better signal-to-noise ratio, each channel has a preamplifier equipped to PMT base. The preamplifier drives the differential signals over 18 m of shielded twisted-pair cables to the VME crates. The readout system achieved a timing resolution of 25 ps after time-walk and INL corrections. The amplitude-dependent time-walk correction is accomplished by measuring corresponding pulse height. Achieved pulse-height resolution is better than 10 mV in a dynamic range 180-5000 mV. The whole TOF readout system was installed in the spectrometer and has been in operation steadily since then, with the performance well meeting the requirements of the physics experiments.

BES III TOF Trigger Sub-System

The time-of-flight (TOF) is one of the Beijing Spectrometer III (BES III) subsystems. Its trigger logic is implemented on a 9U VME board equipped with a rear side plug-in transition module. This trigger logic makes prompt decisions to select interesting physics events. These decisions are made by evaluating the relevant hit information from the 272 scintillators that compose the TOF detector. The accessory VME transition modules, which are plugged into the rear of the TOF readout electronics VME crate, transfer the 272 hit signals into the trigger module. A total of 34 fiber optical links are allocated to bring the raw hit information from the TOF readout electronics to the trigger module. In practice, only 28 fiber-optical links have been used. According to the physics requirements, the TOF trigger module generates 7-bit real-time trigger signals that are passed to the global trigger system, where the primary L1 trigger decision is made. In addition, the TOF trigger subsystem also sends 136-bit position-hit data to the Track-Match-Logic modules in the global trigger system for particle track processing in real time. The trigger logic algorithms are implemented in nine Altera EP1K30 and one EP1S40 field-programmable gate array (FPGAs) in the TOF trigger module. Compared to the previous Beijing Spectrometer TOF trigger subsystem, the employment of high-capacity FPGAs in the trigger module has the advantage of reconfiguration if the physics requirements evolve in the future. During the tests carried out after the assembly of the entire BES III system, the TOF trigger subsystem performed reliably and efficiently.

Cathode strip chamber (CSC) raw data unpacking and packing using bit field classes

Unprecedented data rates that are expected at the LHC put high demand on the speed of the detector data acquisition system. The CSC subdetector located in the Muon Endcaps of the CMS detector has a data readout system equivalent in size to that of a whole Tevatron detector (60 VME crates in the CSC DAQ equal to the whole D0 DAQ size). As a part of the High Level Trigger, the CSC data unpacking runs online and it needs to be able to cope with high data rates online. Early versions of the unpacking code used bit shifts and masks to unpack binary data. To reduce the unpacking time we decided to switch to bit field based data unpacking. The switch allowed us to gain an order of magnitude in speed. In this paper we explain how bit field data unpacking works and why it is dramatically faster compared to conventional bit shift and mask methods.

Track extrapolation and distribution for the CDF-II trigger system

The CDF-II experiment is a multipurpose detector designed to study a wide range of processes observed in the high energy proton–antiproton collisions produced by the Fermilab Tevatron. With event rates greater than 1 MHz, the CDF-II trigger system is crucial for selecting interesting events for subsequent analysis. This document provides an overview of the Track Extrapolation System (XTRP), a component of the CDF-II trigger system. The XTRP is a fully digital system that is utilized in the track-based selection of high momentum lepton and heavy flavor signatures. The design of the XTRP system includes five different custom boards utilizing discrete and FPGA technology residing in a single VME crate. We describe the design, construction, commissioning and operation of this system.

Cam-driven monochromator for QEXAFS

We have developed a cam-drive for quickly tuning the energy of an X-ray monochromator through an X-ray absorption edge for quick extended X-ray absorption spectroscopy (QEXAFS). The data are collected using a 4-channel, 12-bit multiplexed VME analog to digital converter and a VME angle encoder. The VME crate controller runs a real-time operating system. This system is capable of collecting 2 EXAFS-scans in 1 s with an energy stability of better than 1 eV. Additional improvements to increase the speed and the energy stability are under way.

Development of the readout system for the K2K SciBar detector

Readout electronics for the scintillation bar tracking detector (SciBar) in the K2K neutrino oscillation experiment has been developed. SciBar has 14 336 scintillator bars in total. The deposited energy and timing of particles from neutrino interactions in the scintillator bars are measured by 64-channel multianode photo-multiplier tubes (MAPMTs). Compact custom-designed electronics to record the MAPMT signals were developed, consisting of front-end circuit boards attached to each MAPMT and back-end electronics modules sitting in a VME crate. The front-end circuit board multiplexes pulse-height information from all 64 anodes and generates a fast triggering signal. Two sets of ASICs (IDEAS VA32HDR11 and TA32CG) are employed for these functions. The bias voltages and relay of control signals are also handled on the board. The back-end electronics module controls the front-end board by providing the control, timing, and low-voltage signals. The board also digitizes the multiplexed signal from the front-end. The electronics achieves low noise of less than 0.3 photo-electrons and good linearity up to 300 (150) photo-electrons for MAPMTs at the gain of 5/spl times/10/sup 5/ (10/sup 6/).

Real-time motional Stark effect in jet

The increasing importance of real-time measurements and control systems in JET experiments, regarding e.g. Internal Transport Barrier (ITB) and q-profile control, has motivated the development of a real-time motional Stark effect (MSE) system. The MSE diagnostic allows the measurement of local magnetic fields in different locations along the neutral beam path providing, therefore, local measurement of the current and q-profiles. Recently in JET, an upgrade of the MSE diagnostic has been implemented, incorporating a totally new system which allows the use of this diagnostic as a real-time control tool as well as an extended data source for off-line analysis. This paper will briefly describe the technical features of the real-time diagnostic with main focus on the system architecture, which consists of a VME crate hosting three PowerPC processor boards and a fast ADC, all connected via Front Panel Data Port (FPDP). The DSP algorithm implements a lockin-amplifier required to demodulate the JET MSE signals. Some applications for the system will be covered such as: feeding the real-time equilibrium reconstruction code (EQUINOX) and allowing the full coverage analysis of the Neutral Beam time window. A brief comparison between the real-time MSE analysis and the off-line analysis will also be presented.

The DZERO level 3 data acquisition system

The DZERO experiment began RunII datataking operation at Fermilab in spring 2001. The physics program of the experiment requires the Level 3 data acquisition (DAQ) system system to handle average event sizes of 250 kilobytes at a rate of 1 kHz. The system routes and transfers event fragments of approximately 1-20 kilobytes from 63 VME crate sources to any of approximately 100 processing nodes. It is built upon a Cisco 6509 Ethernet switch, standard PCs, and commodity VME single board computers (SBCs). The system has been in full operation since spring 2002.

Development and test of a prototype regional track-finder for the Level-1 trigger of the cathode strip chamber muon system of CMS

Abstract We report on the development and test of a fast three-dimensional Track-Finder for the Level-1 trigger of the CMS endcap muon system. System tests included four types of custom circuit boards clocked at 40 MHz , optical links, and a high-speed custom auxiliary backplane in a VME crate. The hardware results, calculated using field-programmable gate arrays, programmable logic devices, and memory lookup tables, agreed bit-for-bit with software simulations for both random patterns and realistic tracks.

The DAQ system with a RACEway switch for the PHOBOS experiment at RHIC

The PHOBOS data acquisition system based on a RACEway switching network is described. Occupying a single VME crate, the system utilizes 22 PPC750 CPUs working in parallel to compress data from 135 168 silicon pad detectors and an UltraSPARC VME host for event building and data storage. Lossless Huffman coding is used for compression; this reduces the event size fourfold. The two-host disk array is used to stage data before sending them over Gigabit Ethernet to the Relativistic Heavy Ion Collider (RHIC) central computing facility. All trigger and control logic is formed using universal programmable logic VME modules, which can be programmed in situ, even when the system is running. The event building and run control software is written using the ROOT framework. The slow control and configuration makes use of an Oracle database to store configuration and monitoring parameters. The system has been taking data from the PHOBOS experiment at RHIC since June 2000. The achieved data-taking rate is 280 events/s or 28 MB/s, with additional disk arrays it can potentially reach 80 MB/s.

A high performance real-time plasma control and event detection DSP based VME system

This paper describes the digital signal processors module of a high performance system, specially designed for real-time plasma control and event detection on the next generation fusion experiments with long duration discharges. The system is composed of a commercial CPU board and several on-site developed intelligent modules inserted in the same VME crate.

A high-data-transfer-rate VME system for TCP-IP remote real-time control of the ITER in-vessel vision system

This paper describes the control and data acquisition system (CADAS) for the in-vessel vision system (IVVS) for ITER, which imposes special design requirements due to its reactor characteristics. CADAS provides remote real-time control of the motors (by TCP-IP), fast data acquisition (20 bytes/pixel at 100 kpixels/s), error management and sustainable network data transfer rates (by TCP-IP). It is composed of a supervisory workstation connected by a 100 Mbit/s Ethernet channel to a remote VME crate, housing a board based on a PowerPC604R CPU running LynxOS, and two on-site developed intelligent modules. The architecture of these VME modules as well as the operation software and the application protocol are described. The test results are presented as well as an image obtained during the test procedures performed at the ENEA-Frascati Laboratory.

VME crate interconnection through the SCI network in large data acquisition systems

The possibility of using PCI Mezzanine Cards (PMCs) in VME modules opens new options for interconnecting crates by means of high-speed interconnection networks (Scalable Coherent Interface (SCI), ATM, Fiber Channel, etc.). These new technologies offer major benefits in the construction of large data acquisition networks where the number of VME crates is important. In this paper we explain how to use an SCI to easily connect VME crates and take advantage of the new scalability offered by SCI.

Construction of a compact DAQ-system using DSP-based VME modules

A data-acquisition (DAQ) system based upon the SRARC digital-signal-processor (DSP) has been developed. Either one or two DSPs are mounted onto VME boards, WS2126. VME-to-VME data transfer utilizes SHARC links (33.7 MB/s of measured transfer rate), routed onto a front panel of the WS2126 by I/O piggy pack modules, WS9002. The system has been designed for readout of 81,920 FADC (flash analog-to-digital converter) channels at a maximum trigger rate of 500 Hz. This system could be accomplished in a compact setup of four FADC readout VME crates and one master-control VME crate (for event-building and run-control). The system was tested for readout of the BELLE Si vertex detector (SVD).

The detector control system for BABAR

The hardware/software system for the control of the BABAR detector at PEP-II is described. The contributions of a large and diverse collaboration were channeled into a standard hardware selection (VME crate, embedded CPU, fieldbus, etc.) and a uniform software environment, EPICS (Experimental Physics and Industrial Control System). These choices, with a judicious naming convention in place, allowed a distributed and modular development of the control system, which was then integrated within the Online System in the Interaction Hall. The design of the software to complement the EPICS layer is presented. The coordination of controls activities with the rest of the Online is overseen by Run Control; the interface and functionality that controls supports is described. The archival storage, retrieval and analysis/display of the process control data is described, as well as the interface that allows access to this data with the same API as for our physics data. We also describe our modifications to the online-wide messaging system (cmlog) and our interface to the accelerator, PEP-II. Finally, we present our operational experiences with the control system.

Control system for cancer therapy with a heavy ion beam at GSI

A system of three dimensional raster scanning of carbon beam used for cancer therapy has been designed and built at GSI and is in operation since December, 1997. The following components are presented: 1. The front-end real-time system. It is based on VME boards with DSPs designed and built at GSI. In treatment mode, the beam is steered and controlled automatically according to the patient's treatment plan. 2. The on-line monitoring system. During the treatment all data measured by the front-ends is sent via a separate bus to a memory located in a second VME crate. A data server in this crate sends the data as it arrives via TCP to the therapy on-line monitor program, TOM, which displays the treatment process in nearly real-time. 3. The data acquisition and analysis system. In experimental mode the same system is used for quality control and calibration of the beam before the treatment. The measured beam positions and shapes are sent through the same data channel to an analysis package developed at GSI.