Abstract
The majority of contemporary high-energy physics experiments study rare phenomena, which necessitates real-time high-throughput data processing to reduce the raw detector data rate of several Tbyte/s to a rate which is feasible for storage and detailed analysis. Unique trigger systems select the physical events relevant to the experiment. Typically, data fragments corresponding to the same event and originating from multiple detector data sources need to be assembled in a specific location before being processed further. The resulting communication model can lead to congestions and to inefficient system utilization if data are transferred without supervision since numerous sources are attempting to use common interconnect and computing recourses concurrently. This thesis deals with the measures taken to ensure a congestion-free, load-balanced operation of a real-time trigger farm processing data packets as small as several kbytes at a megahertz rate. The input data are initially split among multiple data feeds and need to be assembled and processed within a few milliseconds. The processing farm is built around commodity PCs which are interconnected with a commercial high-speed low-latency network implementing a torus topology. The thesis presents a system for data traffic management based on a global traffic supervisor and a dedicated control network. The former allocates distributed computing resources dynamically in order to avoid network congestions as well as to balance the load of the system. The latter communicates supervising information to all data feeds in order to initiate a controlled data transfer. A congestion-free system operation is demonstrated in a farm prototype with an integrated hardware-based implementation of the traffic shaping system. Based on parameters measured in the prototype, simulation results of a large-scale processing farm are presented. Both the prototype and the simulation results demonstrate that the system is capable of transferring input data initially split among multiple PCI-based feeding nodes, each one transmitting sub-fragments of 128 bytes, to a specific remote shared memory location at a rate beyond 2 MHz. The obtained results demonstrate the applicability of multicomputer systems based on commodity components for high-rate, low-latency trigger processing if certain care is taken in organizing the actual data transfers. This organization has to ensure efficient event building and appropriate allocation of the available processing resources.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.