Gigabit Ethernet Link Research Articles

A generic readout environment for prototype pixel detectors

Pixel detectors for experimental particle physics research have been implemented with a variety of readout formats and potentially generate massive amounts of data. Examples include the PSI46 device for the Compact Muon Solenoid (CMS) experiment which implements an analog readout, the Fermilab FPIX2.1 device with a digital readout, and the Fermilab Vertically Integrated Pixel device. The Electronic Systems Engineering Department of the Computing Division at the Fermi National Accelerator Laboratory has developed a data acquisition system flexible and powerful enough to meet the various needs of these devices to support laboratory test bench as well as test beam applications. The system is called CAPTAN (Compact And Programmable daTa Acquisition Node) and is characterized by its flexibility, versatility and scalability by virtue of several key architectural features. These include a vertical bus that permits the user to stack multiple boards, a gigabit Ethernet link that permits high speed communications to the system and a core group of boards that provide specific processing and readout capabilities for the system. System software based on distributed computing techniques supports an expandable network of CAPTANs. In this paper, we describe the system architecture and give an overview of its capabilities.

A terabit capacity passive polymer optical backplane based on a novel meshed waveguide architecture

An optical backplane based on a meshed polymer waveguide architecture enabling high-speed board-to-board optical interconnection is presented. This planar array of multimode polymer waveguides can provide passive strictly non-blocking links between server line cards fitted with optical transmitter and receiver arrays. This architecture offers a scalable and low-cost solution to the bandwidth limitations faced by electrical backplanes and is suitable for PCB integration. The reported backplane demonstrator uses a matrix of 100 waveguides each capable of 10 Gb/s operation to interconnect 10 cards for a total capacity of a terabit per second aggregate data rate in multicast mode. Characterisation of the backplane demonstrator reveals low link losses of 2 to 8 dB for a multimode fibre input and crosstalk values below −35 dB. Error free data transmission at 10 Gb/s is achieved with a power penalty of only 0.2 dB at a bit-error-rate of 10−9. Additionally, lossless operation of a Gigabit Ethernet link over the backplane is achieved even when using the worst-case highest loss links.

ATLAS DataFlow: the read-out subsystem, results from trigger and data-acquisition system testbed studies and from modeling

In the ATLAS experiment at the LHC, the output of read-out hardware specific to each subdetector will be transmitted to buffers, located on custom made PCI cards (ROBINs). The data consist of fragments of events accepted by the first-level trigger at a maximum rate of 100 kHz. Groups of four ROBINs will be hosted in about 150 Read-Out Subsystem (ROS) PCs. Event data are forwarded on request via Gigabit Ethernet links and switches to the second-level trigger or to the Event builder. In this paper a discussion of the functionality and real-time properties of the ROS is combined with a presentation of measurement and modelling results for a testbed with a size of about 20% of the final DAQ system. Experimental results on strategies for optimizing the system performance, such as utilization of different network architectures and network transfer protocols, are presented for the testbed, together with extrapolations to the full system.

An FPGA-based network intrusion detection system with on-chip network interfaces

Network intrusion detection systems (NIDS) are critical network security tools that help protect computer installations from malicious users. Traditional software-based NIDS architectures are becoming strained as network data rates increase and attacks intensify in volume and complexity. In recent years, researchers have proposed using FPGAs to perform the computationally-intensive components of intrusion detection analysis. In this work, we present a new NIDS architecture that integrates the network interface hardware and packet analysis hardware into a single FPGA chip. This integration enables a higher performance and more flexible NIDS platform. To demonstrate the benefits of this technique, we have implemented a complete and functional NIDS in a Xilinx Virtex II Pro FPGA that performs in-line packet analysis and filtering on multiple Gigabit Ethernet links using rules from the open-source Snort attack database.

Superhigh‐definition digital cinema distribution system with 8‐million‐pixel resolution

AbstractWe developed a prototype superhigh‐definition (SHD) digital cinema system that can store, transmit, and display extrahigh‐quality movies with a resolution of 8 million pixels, using JPEG2000 coding algorithm. The spatial resolution on image quality is four times better than HDTV, and surpasses that of 35‐mm cinefilm. The system is comprised of three principal apparatus designed for SHD movies: a video server, a real‐time decoder, and a projector. The movie data digitized by scanning the cinefilm is compressed using JPEG2000 coding algorithm and stored in the video server in advance. Using Gigabit Ethernet (GbE) links and TCP/IP protocols, the server transmits the compressed digital movie data stream of 300 to 450 Mbps to a real‐time decoder. The decoder receives the compressed data stream, and outputs a raw, uncompressed, 10‐bit R/G/B digital video signal using a parallel‐processor mechanism in real time. The projector reproduces the SHD movie using three reflective LCD panels (D‐ILA), one for each 10‐bit R/G/B color, while receiving the raw digital video signal through the special video interfaces. This system enables us to distribute digital cinema of original 35‐mm cinefilm quality via the GbE network. © 2006 Wiley Periodicals, Inc. Syst Comp Jpn, 37(4): 35–44, 2006; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/scj.20465

Optical network packet error rate due to physical layer coding

A physical layer coding scheme is designed to make optimal use of the available physical link, providing functionality to higher components in the network stack. This paper presents results of an exploration of the errors observed when an optical gigabit Ethernet link is subject to attenuation. The results show that some data symbols suffer from a far higher probability of error than others. This effect is caused by an interaction between the physical layer and the 8B/10B block coding scheme. The authors illustrate how the application of a scrambler, performing data whitening, restores content-independent uniformity of packet loss. They also note the implications of their work for other (N, K) block-coded systems and discuss how this effect will manifest itself in a scrambler-based system. A conjecture is made that there is a need to build converged systems with the combinations of physical, data link, and network layers optimized to interact correctly. In the meantime, what will become increasingly necessary is both an identification of the potential for failure and the need to plan around it.

Performance of 1 and 10 Gigabit Ethernet cards with server quality motherboards

System administrators often assume that just by plugging in a Gigabit Ethernet Interface, the system will deliver line rate performance; sadly this is not often the case. The behaviour of various 1 and 10 Gigabit Ethernet Network Interface cards (NICs) has been investigated using server quality motherboards. The latency, throughput, and the activity on the PCI buses and Gigabit Ethernet links were chosen as performance indicators. The tests were performed using two PCs connected back-to-back and sending UDP/IP frames from one to the other. This paper shows the importance of having a good combination of memory and peripheral bus chipset, Ethernet Interface, CPU power, good driver, and operating system designs and proper configuration to achieve and sustain gigabit transfer rates. With these considerations taken into account, and suitably designed hardware, transfers can operate at gigabit and multi-gigabit speeds. Some recommendations are given for high performance data servers.

Field Trial of Full-Mesh WDM Network (AWG-STAR) in Metropolitan/Local Area

We describe a field trial of a wavelength routing full-mesh wavelength-division-multiplexing (WDM) (AWG-STAR) network for citywide local administration in Japan. An AWG router and five WDM nodes were connected to the existing fiber in the downtown area of the city of Chitose, Hokkaido. A 5 /spl times/ 5 full-mesh network was overlaid on the Chitose intranet. A video-on-demand system based on Gigabit Ethernet links, high-definition serial digital interface signals, and synchronous digital hierarchy signals of STM-16 links are transmitted and routed simultaneously on the wavelength paths of the network. These systems are used for administration and education information sharing for city residents. The network is managed and controlled remotely from the NTT R&D Center in Atsugi, Japan, which is 50 km southwest of Tokyo and approximately 1000 km from Chitose. AWG-STAR exhibits promising potential as a large capacity network system for both simple full-mesh systems and highly secure private networks.

All-fiber wavelength- and mode-selective coupler for optical interconnections

We experimentally demonstrate a new wavelength- and mode-selective coupler utilizing intermodal coupling between a standard single-mode fiber (SMF) and a hollow optical fiber (HOF). The fabricated device routes optical signals such that a 1.3-microm signal is directed to the HOF port, adiabatically converting the LP(01) mode into a ring-shaped mode, which can be further connected to a gigabit ethernet link, reducing the differential mode-dispersion penalty. Optical signals near 1.5microm , meanwhile, stay in the LP(01) mode of the SMF arm for further connection to conventional wavelength-division multiplexing links based on erbium-doped fiber amplifiers. The performance of the device is characterized in terms of insertion loss, channel isolation, and mode-conversion efficiency.

Deep view: high-resolution reality

We have designed the Deep View visualization system, which consists of a Linux cluster that performs computations to produce 3D geometry, renders the geometry to produce 2D pixels, and then transfers the pixels to be displayed on the T221 display (or video wall). We accelerate the pixel transfer operations using IBM's Scalable Graphics Engine (SGE). We drive high-resolution displays at interactive frame rates using our cluster and the SGE. The SGE is a network-attached frame buffer capable of double buffering up to 16 million pixels. It routes incoming pixels from multiple sources to the appropriate locations in its frame buffer and then transfers the composited result to the T221 display using digital video interface (DVI) output. In total, the SGE can accept up to 16 input links and can drive as many as eight synchronized DVI outputs. In addition, it can time interleave image pairs from its frame buffer to effect time-division stereo. In the current Deep View configuration, the rendered pixels are sent by each node in the cluster to the SGE over a Gigabit Ethernet link, and we use four of the synchronized DVI outputs to drive the T221 at full resolution.

Testing and modeling Ethernet switches and networks for use in ATLAS high-level triggers

The ATLAS second-level trigger will use a multilayered local-area network to transfer 5-Gbyte/s detector data from /spl sim/1500 buffers to a few hundred processors. A model of the network has been constructed to evaluate its performance. A key component of the network model is a model of an individual switch, reproducing the behavior measured in real devices. A small number of measurable parameters are used to model a variety of commercial Ethernet switches. Using parameters measured on real devices. The impact on the overall network performance is modeled. In the ATLAS context, both 100-Mbit and gigabit Ethernet links are required. A system is described that is capable of characterizing the behavior of commercial switches with the required number of nodes under traffic conditions resembling those to be encountered in the ATLAS experiment. Fast Ethernet traffic is provided by a high-density custom-built tester based on field-programmable gate arrays, programmed in Handel-C and VHDL, while the gigabit Ethernet traffic is generated using Alteon NICs with custom firmware. The system is currently being deployed with 32 100-Mbit ports and 16 gigabit ports, and will be expanded to /spl sim/256 nodes of 100 Mbit and /spl sim/50 GBE nodes.

The NA48 on-line PC farm

The NA48 experiment at the CERN SPS measures the direct CP violation parameter Re( ε′/ ε) in the neutral kaon system with an intended precision of 2×10 −4. Thus, the data acquisition system has to handle peak rates up to 150 MByte/s during a 2.5 s beam burst and flat rates up to 20 MByte/s during on-line processing. The NA48 on-line PC farm includes 24 PentiumII-266MHz PCs connected via switched fast ethernet network. This farm receives data from the detector systems and performs event building. Afterwards the data are transferred via an optical gigabit ethernet link to the NA48 central data recording system at CERN computer center. In addition, trigger and checking processes can be applied. The software on the farm is implemented as client/server applications and represents a highly flexible, fail safe, and scalable system. During data taking, in 1999, 100 TByte were collected, processed, and stored by the acquisition system with an overall efficiency of 99%.

A statistical analysis of conditioned launch for gigabit ethernet links using multimode fiber

This paper reports, what is believed to be the first statistical analysis of restricted launch schemes. This is achieved by a detailed theoretical study of a large range of perturbed multimode optical fibers (MMF's) that are believed to be representative of those currently in use. It is shown that radial "offset" launches allow a robust and reliable solution to bandwidth collapse that occurs when standard laser sources are used with multimode fibers. Using such an approach, it is shown that gigabit per second link lengths using the offset launch technique can be increased to 550 m at an operating wavelength of 1300 nm using 62.5-/spl mu/m diameter MMF. These link lengths are greater than those achievable with conventional techniques. The approach is also used to indicate the tolerance on alignment precision required for successful operation and to suggest a simple measurement for the assessment of link performance. Finally, the use of angled launching into MMF is shown to allow enhanced operating bandwidths.