Gigabit Ethernet Link Research Articles

Research and development of the back-end electronics for the two-dimensional improved resistive plate chambers in CMS upgrade

To complement and ensure redundancy in the endcap muon system of the Compact Muon Solenoid (CMS) detector and to extend the Resistive Plate Chamber (RPC) system coverage, improved RPCs (iRPCs) with either orthogonal layer strips with one-end electronics or single layer strips with two-end electronics providing more precise time measurement will be installed in the very forward pseudorapidity region of $$|\eta |<2.4$$ . The iRPC readout system needs to support two-dimensional (2D) or two-end readout. In addition, it must combine detector data with Timing, Trigger and fast Control (TTC) and Slow Control (SC) into one data stream over a bi-directional optical link with a line rate of 4.8 Gb/s between the Front-End Electronics (FEE) and the Back-End Electronics (BEE). To fulfill these requirements, a prototype BEE for the iRPC 2D chamber has been researched and designed. A Micro-Telecommunication and Computing Architecture ( $$\mu $$ TCA)-based processing card was designed in this study to establish a prototype system together with a $$\mu $$ TCA crate. The Giga-Bit Transceiver (GBT) protocol is integrated to provide bi-directional communication between the FEE and BEE. A server is connected with the BEE by a Gigabit Ethernet (GbE) link for SC and a 10-GbE link for Data AcQuisition (DAQ). The Bit Error Rate (BER) test of the back-end board and a joint test with the iRPC 2D prototype chamber were performed. A BER of less than $$1.331\times {10^{-16}}$$ was obtained. The time measurement with a resolution of 3.05 ns was successfully realized, and detector efficiencies of 97.7% for longitudinal strips and 96.0% for orthogonal strips were measured. Test results demonstrate the correctness and reliability of the prototype BEE. The BEE prototype satisfies the requirements for the iRPC 2D chamber, and it worked stably and reliably during a long-term joint test run.

10 Gigabit White Rabbit: Sub-Nanosecond Timing and Data Distribution

Time synchronization is a critical feature for many scientific facilities and industrial infrastructures. The required performance is progressively increasing everyday, for instance, few tens of nanoseconds for Fifth Generation (5G) networks or sub-nanosecond accuracy on next family of particle accelerators and astrophysics telescopes. Due to this exigent accuracy, many applications require specific timing dedicated networks, increasing the system cost and complexity. Under this context, the new IEEE 1588-2019 High Accuracy (HA) default profile is intensively based on White Rabbit (WR) which can provide sub-nanosecond accurate synchronization for Ethernet networks. However, current WR solutions have not been designed to work properly with high data bandwidth delivery services even in 1 Gigabit Ethernet (GbE) links. On this contribution, the authors propose a new architecture design that enables WR and, consequently, the IEEE 1588-2019 HA profile to be deployed over 10 GbE links solving the already identified data bandwidth problem. Furthermore, this work addresses different experiments needed to characterize the system performance in terms of time synchronization and data transfer. As final result, this contribution presents for the first time in the literature a new WR system which allows high bandwidth data exchange in 10 GbE networks while providing sub-nanosecond accuracy synchronization. The proposed solution maintains the time synchronization performance of existing WR 1 GbE devices with significant advantages in terms of latency and data bandwidth, enabling its deployment in applications that integrate data and synchronization information in the same network.

Development and application of a modular test system for the HV-CMOS pixel sensor R&amp;D of the ATLAS HL-LHC upgrade

High-voltage CMOS is a promising technology for the pixel sensor of tracking detectors in the collider experiments. Extensive studies are being carried out by the ATLAS Collaboration to investigate the possibility of using the HV-CMOS technology in the HL-LHC upgrade of the ATLAS inner tracker detector. The CaRIBOu (Control and Readout Itk BOards) is a modular test system developed to test the HV-CMOS pixel sensor prototypes and demonstrators. This test system consists of pixel sensor specific front-end chip boards, a control and readout board (CaR board), a central interface board and the back-end DAQ system. Currently, two DAQ solutions are available for the CaRIBOu system, one is based on the FELIX (front-end link eXchange) readout system and the other is based on the Gigabit Ethernet link. Various testbeam experiments have been carried out with the CaRIBOu system since 2015 for the pixel sensors fabricated by the AMS HV-CMOS 180 nm and 350 nm technologies. The test results show that this test system is very flexible and could be adapted to the test of different pixel sensors with minimum effort, and the performance meets the testbeam requirements.

Optimized framegrabber for the Cherenkov telescope array

Our contribution presents a high bandwidth platform that implements traffic aggregation and switching capabilities for the Cherenkov telescope array (CTA) cameras. Our proposed system integrates two different data flows: a unidirectional one from the cameras to an external server and a second one, fully configurable dedicated to configuration and control traffic for the camera management. The former requires high bandwidth mechanisms to be able to aggregate several 1 gigabit Ethernet links into one high speed 10 gigabit Ethernet port. The latter is responsible for providing routing components to allow a control and management path for all the elements of the cameras. Hence, a simple, efficient, and flexible routing mechanism has been implemented avoiding complex circuitry that impacts in the system performance. As a consequence, an asymmetric network topology allows high bandwidth communication and, at the same time, a flexible and cost-effective implementation. In our contribution, we analyze the camera requirements and present the proposed architecture. Moreover, we have designed several evaluation tests to demonstrate that our solution fulfills the CTA project needs. Finally, we illustrate the general possibilities of the proposed solution for other data acquisition applications and the most promising futures lines of research are discussed.

Gigabit Security Switches Deployment - A Review

With the increasing users and services over the Internet, people demand faster data fetching and more reliable network services. However, the increasing network services and equipment causes the network management much more difficult. This is largely due to the increased size of networks and the requirement for increasingly faster and more efficient networks. In this paper, we discuss high available (HA) architecture for cost effective security switches. Security switch contains of a security switch engine (SSE) and a traditional layer-2 switch. They examine the packet content. These two components connected through a Gigabit Ethernet link. In this paper, we also discuss the types of switches. However, the majority of network administrators currently deploying gigabit networks will find that they are seeing far less than a full 100 Mbps of traffic moving over the wire. Even a standard PC with an off-the-shelf gigabit network card will be able to cope quite easily.

40 Gbps data acquisition system for NectarCAM

The Cherenkov Telescope Array (CTA) will be the next generation ground-based gamma-ray observatory. It will be made up of approximately 100 telescopes of three different sizes, from 4 to 23 meters in diameter. The previously presented prototype of a high speed data acquisition (DAQ) system for CTA (CHEP 2012, [6]) has become concrete within the NectarCAM project, one of the most challenging camera projects with very demanding needs for bandwidth of data handling. We designed a Linux-PC system able to concentrate and process without packet loss the 40 Gb/s average data rate coming from the 265 Front End Boards (FEB) through Gigabit Ethernet links, and to reduce data to fit the two ten-Gigabit Ethernet downstream links by external trigger decisions as well as custom tailored compression algorithms. Within the given constraints, we implemented de-randomisation of the event fragments received as relatively small UDP packets emitted by the FEB, using off-the-shelf equipment as required by the project and for an operation period of at least 30 years. We tested out-of-the-box interfaces and used original techniques to cope with these requirements, and set up a test bench with hundreds of synchronous Gigabit links in order to validate and tune the acquisition chain including downstream data logging based on zeroMQ and Google ProtocolBuffers [8].

SPIDR, a general-purpose readout system for pixel ASICs

The SPIDR (Speedy PIxel Detector Readout) system is a flexible general-purpose readout platform that can be easily adapted to test and characterize new and existing detector readout ASICs. It is originally designed for the readout of pixel ASICs from the Medipix/Timepix family, but other types of ASICs or front-end circuits can be read out as well. The SPIDR system consists of an FPGA board with memory and various communication interfaces, FPGA firmware, CPU subsystem and an API library on the PC . The FPGA firmware can be adapted to read out other ASICs by re-using IP blocks. The available IP blocks include a UDP packet builder, 1 and 10 Gigabit Ethernet MAC's and a “soft core” CPU . Currently the firmware is targeted at the Xilinx VC707 development board and at a custom board called Compact-SPIDR . The firmware can easily be ported to other Xilinx 7 series and ultra scale FPGAs. The gap between an ASIC and the data acquisition back-end is bridged by the SPIDR system. Using the high pin count VITA 57 FPGA Mezzanine Card (FMC) connector only a simple chip carrier PCB is required. A 1 and a 10 Gigabit Ethernet interface handle the connection to the back-end. These can be used simultaneously for high-speed data and configuration over separate channels. In addition to the FMC connector, configurable inputs and outputs are available for synchronization with other detectors. A high resolution (≈ 27 ps bin size) Time to Digital converter is provided for time stamping events in the detector. The SPIDR system is frequently used as readout for the Medipix3 and Timepix3 ASICs. Using the 10 Gigabit Ethernet interface it is possible to read out a single chip at full bandwidth or up to 12 chips at a reduced rate. Another recent application is the test-bed for the VeloPix ASIC, which is developed for the Vertex Detector of the LHCb experiment. In this case the SPIDR system processes the 20 Gbps scrambled data stream from the VeloPix and distributes it over four 10 Gigabit Ethernet links, and in addition provides the slow and fast control for the chip.

Embedded Real-Time H264/AVC High Definition Video Encoder on TI’s KeyStone Multicore DSP

To overcome high computational complexity of advanced video encoders for emerging applications that require real-time processing, multicore technology can be one of the promising solutions to meet this constraint. In this context, this paper presents a parallel implementation of the H264/AVC high definition (HD) video encoder exploiting the power processing of eight-core digital signal processor (DSP) TMS320C6678. GOP Level Parallelism approach is used to improve the encoding speed and meet the real-time encoding compliant. A master core is reserved to handle data transfer between the DSP and the camera interface via a Gigabit Ethernet link. Multithreading algorithm and ping-pong buffers technique are used to enhance the classic GOP level parallelism approach and hide communication overhead. Experimental results on seven slave DSP cores, running each at 1 GHz, show that our implementation allows performing a real-time HD (1280ź?ź720) video encoding. The achieved encoding speed is up to 28 f/s. The proposed parallel implementation accelerates the encoding process by a factor of 6.7 without inducing quality degradation in terms of PSNR or bit-rate increase compared to single core implementation. Experiments show that our proposed scheduling technique for hiding communication overhead saves up to 36 % of the fully encoding chain time which includes frames capturing, frames encoding and bitstream saving in a file.

400 Gigabit Ethernet using advanced modulation formats: Performance, complexity, and power dissipation

We review possible architectures for 400 Gigabit Ethernet links based on advanced modulation formats for the first time. Their optical link power budget, digital complexity, and power dissipation are compared via simulations. The challenges of implementing the physical layer are discussed.

Performance of a Micro-Strip Gas Chamber for event wise, high rate thermal neutron detection with accurate 2D position determination

A two-dimensional (2D) position sensitive detector for neutron scattering applications based on low-pressure gas amplification and micro-strip technology was built and tested with an innovative readout electronics and data acquisition system. This detector contains a thin solid neutron converter and was developed for time- and thus wavelength-resolved neutron detection in single-event counting mode, which improves the image contrast in comparison with integrating detectors. The prototype detector of a Micro-Strip Gas Chamber (MSGC) was built with a solid natGd/CsI thermal neutron converter for spatial resolutions of about 100 μm and counting rates up to 107 neutrons/s. For attaining very high spatial resolutions and counting rates via micro-strip readout with centre-of-gravity evaluation of the signal amplitude distributions, a fast, channel-wise, self-triggering ASIC was developed. The front-end chips (MSGCROCs), which are very first signal processing components, are read out into powerful ADC-FPGA boards for on-line data processing and thereafter via Gigabit Ethernet link into the data receiving PC. The workstation PC is controlled by a modular, high performance dedicated software suite. Such a fast and accurate system is crucial for efficient radiography/tomography, diffraction or imaging applications based on high flux thermal neutron beam.In this paper a brief description of the detector concept with its operation principles, readout electronics requirements and design together with the signals processing stages performed in hardware and software are presented. In more detail the neutron test beam conditions and measurement results are reported. The focus of this paper is on the system integration, two dimensional spatial resolution, the time resolution of the readout system and the imaging capabilities of the overall setup. The detection efficiency of the detector prototype is estimated as well.

High-speed readout of high-Z pixel detectors with the LAMBDA detector

High-frame-rate X-ray pixel detectors make it possible to perform time-resolved experiments at synchrotron beamlines, and to make better use of these sources by shortening experiment times. LAMBDA is a photon-counting hybrid pixel detector based on the Medipix3 chip, designed to combine a small pixel size of 55 μm, a large tileable module design, high speed, and compatibility with ``high-Z'' sensors for hard X-ray detection. This technical paper focuses on LAMBDA's high-speed-readout functionality, which allows a frame rate of 2000 frames per second with no deadtime between successive images. This takes advantage of the Medipix3 chip's ``continuous read-write'' function and highly parallelised readout. The readout electronics serialise this data and send it back to a server PC over two 10 Gigabit Ethernet links. The server PC controls the detector and receives, processes and stores the data using software designed for the Tango control system. As a demonstration of high-speed readout of a high-Z sensor, a GaAs LAMBDA detector was used to make a high-speed X-ray video of a computer fan.

Design and implementation of a nanosecond time-stamping readout system-on-chip for photo-detectors

A readout system suitable for a large number of synchronized photo-detection units has been designed. Each unit embeds a specifically designed fully integrated communicating system based on Xilinx FPGA SoC technology. It runs the VxWorks real-time OS and a custom data acquisition software designed within the Ice middleware framework, resulting in a highly flexible, controllable and scalable distributed application. Clock distribution and delay calibration over customized fixed latency gigabit Ethernet links enable synchronous time-stamping of events with nanosecond precision. The implementation of this readout system on several data-collecting units as well as its performances are described.

Focus on semiconductors and software

Focus on semiconductors and software

Computer Simulation of Chaotic Phenomena in High-Speed Communication Networks

현대 과학과 기술의 다양한 영역에 있어서 ‘카오스’와 ‘자기유사성’은 여러 가지 문제를 내포하고 있으므로, 네트워크 시스템도 이 범주를 벗어날 수 없다. 다양한 네트워크 프로토콜과 어플리케이션 그리고 서비스의 증가는 네트워크 트래픽을 더 복잡하고 예측 불가능하게 만들고 있다. 이것은 ‘자기유사성’ 현상이 TCP 프로토콜 행태에 의한 네트워크 트래픽의 특성에 기인한다는 것을 보여주며, 이 모든 특성은 고속 데이터 전송의 개발과 함께 점점 더 중요해지고 있다. 근본적으로 이러한 성질은 네트워크에서의 폭주를 유발하고 그 결과로써 패킷 손실을 유발하고 있으나 폭주를 제어하는 최신의 구조라 할지라도 어떤 상황 에서는 매우 안정적이지 못하다. 예를 들어 W. Feng이 보여준 바에 의하면 매우 심각한 폭주 상태의 네트워크에서 TCP reno 손실율이 5%를 넘으면 기가비트 이더넷 상에서 어떤 링크 손실률은 50Mb/s 이상에서의 손실로 변환하여 쉽게 계산되지만 이 정도의 손실률 레벨은 수용하기 어려운 값이다. 그러나 정보시스템의 성질을 충분히 이해하고, 모든 고전적 방법이 카오스 이론에 적용 되도록 관련 연구자들에게 가능성을 제공할 필요가 있으며, 아직 특정한 비선형 동적 시스템을 해석하는 모델은 제시되지 않았다. 따라서 본 논문에서는 TCP 프로토콜 행태의 패킷스위칭 네트워크 해석에 대한 새로운 접근을 하였으며, 본 네트워크는 어떤 파라미터 값에서 카오스적인 성질을 보여주는 비선형 동적 시스템으로 해석하였다.

A readout system-on-chip for a cubic kilometer submarine neutrino telescope

This contribution reports on the read-out system of the future KM3NeT undersea network of several thousands of synchronized optical detecting nodes. Each node embeds a specifically designed fully integrated communicating system based on Xilinx FPGA SoC technology. It runs the VxWorks real-time OS and DAQ software designed within the Ice middleware framework resulting in a highly flexible, controllable and scalable distributed application. Clock distribution and delay calibration over customized fixed latency gigabit Ethernet links enable synchronous time stamping of events with nanosecond precision.

A Scalable DAQ System Based on the DRS4 Waveform Digitizing Chip

Many current and future experiments require the highest temporal resolution together with large numbers of channels in the data acquisition system (DAQ) for a minimum cost. The DRS4 waveform digitizing chip allows data sampling at up to 5 Giga-samples per second (GSPS) with high amplitude resolution. The domino wave sampling method offers a significant cost and power reduction compared to a traditional flash analog-to-digital converter (ADC). This work presents a new DAQ system based on the DRS4 chip that allows continuous digitization of analog signals at 120 Mega-samples per second (MSPS) with the possibility to sample a region of interest up to a rate of 5 GSPS, thereby allowing a long event record with small dead-time in the read-out. The signal-to-noise ratio (SNR) of the system is measured to be 9.3 bit for the 120 MSPS signal and 9.6 bit for the DRS4 readout signal. Arbitrarily complex trigger logic can be built entirely in the digital domain in the read-out field-programmable gate array (FPGA). A Gigabit Ethernet link provides high-speed connectivity from the DAQ board to the backend system. Built-in board-to-board communication and the modular design of the system offer great scalability and flexibility with respect to the number of supported data channels.

The T2K Near Detector Data Acquisition Systems

T2K is a second generation long-baseline experiment in Japan, designed to perform precision studies of neutrino oscillations. A neutrino beam originating at the J-PARC facility is sampled by two near detectors, approximately 280 m distant from the source and directed towards the Kamioka underground facility, 295 km distant, where the Super-Kamiokande water Cerenkov detector is situated. Both the on-axis (INGRID) and off-axis (ND280) near detectors employ the MIDAS framework for their data acquisition systems. Events are selected via a flexible hardware trigger, which allows mixed operating modes incorporating beam spill, cosmic and calibration triggers. Data are acquired from the front-end electronics over optical Gigabit Ethernet links by custom MIDAS front-end applications distributed over a cluster of Linux computers. Event fragments from all sub-detectors are merged and logged on a central backend processor and then copied to remote storage for long-term archival. The system is also designed to allow flexible partitioning of sub-detectors for standalone diagnostics and calibration. We present the design of the DAQ system and report on experience gathered during early operation of the near detectors.

Link model simulation and power penalty specification of the versatile link systems

This paper presents simulation and experimental studies of optical power penalties on the Versatile Link, a common R&D project on high-speed optical link for SLHC experiments. The 10 Gigabit Ethernet (10GbE) link model is examined and conservative link power penalties are predicted. We conduct parameter sensitivity analyses and find that the transmitter characteristics affect the link power penalties most. Power penalty differences of multi-mode and single-mode commerical transceiver modules over different fiber lengths are tested to be within the simulation limits. The optical power budgets are then proposed for different Versatile Link variants.

Architecture of a silicon strip beam position monitor

A collaboration between Fermilab and the Institute for High Energy Physics (IHEP), Beijing, has developed a beam position monitor for the IHEP test beam facility. This telescope is based on 5 stations of silicon strip detectors having a pitch of 60 microns. The total active area of each layer of the detector is about 12x10 cm2. Readout of the strips is provided through the use of VA1` ASICs mounted on custom hybrid printed circuit boards and interfaced to Adapter Cards via copper-over-kapton flexible circuits. The Adapter Cards amplify and level-shift the signal for input to the Fermilab CAPTAN data acquisition nodes for data readout and channel configuration. These nodes deliver readout of triggered events and temperature data to an analysis computer over gigabit Ethernet links.

Performance of 10 Gigabit Ethernet Using Commodity Hardware

In the prospect of employing 10 Gigabit Ethernet as networking technology for online systems and offline data analysis centers of High Energy Physics experiments, we performed a series of measurements on the performance of 10 Gigabit Ethernet, using the network interface cards mounted on the PCI-Express bus of commodity PCs both as transmitters and receivers. In real operating conditions, the achievable maximum transfer rate through a network link is not only limited by the capacity of the link itself, but also by that of the memory and peripheral buses and by the ability of the CPUs and of the Operating System to handle packet processing and interrupts raised by the network interface cards in due time. Besides the TCP and UDP maximum data transfer throughputs, we also measured the CPU loads of the sender/receiver processes and of the interrupt and soft-interrupt handlers as a function of the packet size, either using standard or ?jumbo? Ethernet frames. In addition, we also performed the same measurements by simultaneously reading data from Fibre Channel links and forwarding them through a 10 Gigabit Ethernet link, hence emulating the behavior of a disk server in a Storage Area Network exporting data to client machines via 10 Gigabit Ethernet.