Data Carousel Research Articles

Updates to the ATLAS Data Carousel Project

Updates to the ATLAS Data Carousel Project

The ATLAS Data Carousel Project Status

The High Luminosity upgrade to the LHC, which aims for a tenfold increase in the luminosity of proton-proton collisions at an energy of 14 TeV, is expected to start operation in 2028/29 and will deliver an unprecedented volume of scientific data at the multi-exabyte scale. This amount of data has to be stored, and the corresponding storage system must ensure fast and reliable data delivery for processing by scientific groups distributed all over the world. The present LHC computing and data management model will not be able to provide the required infrastructure growth, even taking into account the expected hardware technology evolution. To address this challenge, the Data Carousel R&D project was launched by the ATLAS experiment in the fall of 2018. State-of-the-art data and workflow management technologies are under active development, and their current status is presented here.

ATLAS Data Carousel

The ATLAS experiment at CERN’s LHC stores detector and simulation data in raw and derived data formats across more than 150 Grid sites world-wide, currently in total about 200PB on disk and 250PB on tape. Data have different access characteristics due to various computational workflows, and can be accessed from different media, such as remote I/O, disk cache on hard disk drives or SSDs. Also, larger data centers provide the majority of offline storage capability via tape systems. For the HighLuminosity LHC (HL-LHC), the estimated data storage requirements are several factors bigger than the present forecast of available resources, based on a flat budget assumption. On the computing side, ATLAS Distributed Computing was very successful in the last years with high performance and high throughput computing integration and in using opportunistic computing resources for the Monte Carlo simulation. On the other hand, equivalent opportunistic storage does not exist. ATLAS started the Data Carousel project to increase the usage of less expensive storage, i.e. tapes or even commercial storage, so it is not limited to tape technologies exclusively. Data Carousel orchestrates data processing between workload management, data management, and storage services with the bulk data resident on offline storage. The processing is executed by staging and promptly processing a sliding window of inputs onto faster buffer storage, such that only a small percentage of input data are available at any one time. With this project, we aim to demonstrate that this is the natural way to dramatically reduce our storage cost. The first phase of the project was started in the fall of 2018 and was related to I/O tests of the sites archiving systems. Phase II now requires a tight integration of the workload and data management systems. Additionally, the Data Carousel studies the feasibility to run multiple computing workflows from tape. The project is progressing very well and the results presented in this document will be used before the LHC Run 3.

Enabling Data Intensive Science on Supercomputers for High Energy Physics R&D Projects in HL-LHC Era

The ATLAS experiment at CERN’s Large Hadron Collider uses theWorldwide LHC Computing Grid, the WLCG, for its distributed computing infrastructure. Through the workload management system PanDA and the distributed data management system Rucio, ATLAS provides seamless access to hundreds of WLCG grid and cloud based resources that are distributed worldwide, to thousands of physicists. PanDA annually processes more than an exabyte of data using an average of 350,000 distributed batch slots, to enable hundreds of new scientific results from ATLAS. However, the resources available to the experiment have been insufficient to meet ATLAS simulation needs over the past few years as the volume of data from the LHC has grown. The problem will be even more severe for the next LHC phases. High Luminosity LHC will be a multiexabyte challenge where the envisaged Storage and Compute needs are a factor 10 to 100 above the expected technology evolution. The High Energy Physics (HEP) community needs to evolve current computing and data organization models in order to introduce changes in the way it uses and manages the infrastructure, focused on optimizations to bring performance and efficiency not forgetting simplification of operations. In this paper we highlight recent R&D projects in HEP related to data lake prototype, federated data storage and data carousel.

Efficient Access to Massive Amounts of Tape-Resident Data

Randomly restoring files from tapes degrades the read performance primarily due to frequent tape mounts. The high latency and time-consuming tape mount and dismount is a major issue when accessing massive amounts of data from tape storage. BNL’s mass storage system currently holds more than 80 PB of data on tapes, managed by HPSS. To restore files from HPSS, we make use of a scheduler software, called ERADAT. This scheduler system was originally based on code from Oak Ridge National Lab, developed in the early 2000s. After some major modifications and enhancements, ERADAT now provides advanced HPSS resource management, priority queuing, resource sharing, web-browser visibility of real-time staging activities and advanced real-time statistics and graphs. ERADAT is also integrated with ACSLS and HPSS for near real-time mount statistics and resource control in HPSS. ERADAT is also the interface between HPSS and other applications such as the locally developed Data Carousel, providing fair resource-sharing policies and related capabilities. ERADAT has demonstrated great performance at BNL.

Multicast Wi-Fi Raptor-enabled data carousel design: simulation and practical implementation

Multicast is an efficient way of transmitting the same set of data to multiple interested users. Unlike the 3rd Generation Partnership Project (3GPP) cellular standards, for Wi-Fi, there is no standardised solution for reliable multicast data transmission. Multicast packets are delivered to multiple users as a broadcast service without support for automatic repeat request. Hence, multicast transmission often results in high packet loss. In order to improve the reliability of multicast delivery, a fixed low-speed (robust) transmission mode can be used. However, this results in the inefficient use of scarce and valuable radio bandwidth. This paper presents a reliable and efficient Wi-Fi multicast delivery solution for use in challenging outdoor environments. An application layer forward error correction (AL-FEC)-enabled data carousel is proposed to enhance reliability. For multicast transmission, we demonstrate that limitations in the Wi-Fi clients are a major source of packet loss, even in ideal channel conditions. Client limitations (particularly data rate limitations) were found to vary as a function of modulation and coding mode, Raptor code parameters and multicast server rate. Our initial Raptor-enabled carousel designs are based on computer simulations and lab-based trials. Analysis is then extended to field trials using a practical implementation of the recommended design. These trials were performed in central Bristol with parameters such as received signal level, packet loss traces and file download times recorded at the clients. Finally, we compare our site-specific simulated results against real-world measurements.

Robust Multicast Delivery for WLANs: Transforming Theory into Practice

Abstract Multicast is an efficient way of transmitting the same set of data to multiple interested users. WLANs do not provide any standardized solution for reliable data multicast. Multicast packets are delivered to multiple users as a simple broadcast service without support for Automatic Repeat Request. Hence, multicast transmission results in higher packet error rates. In order to improve the reliability of multicast delivery, a fixed low speed (robust) transmission mode is generally required. However, this results in the inefficient use of scarce and valuable bandwidth. This paper presents a reliable and efficient Wi-Fi multicast delivery solution for challenging outdoor applications. An Application Layer Forward Error Correction enabled data carousel is proposed for reliable multicast transmission over standard IEEE 802.11 WLANs. Field trials have been conducted to demonstrate and quantify system performance. Results show that the proposed system enhances user quality of experience while time efficiently utilising available bandwidth. The proposed interleaved carousel that is Raptor code enabled, attains 74.5% less download time than the traditional carousel, because of randomisation of errors and the protection of Raptor AL-FEC. 90% of users are able to download successfully a video file of 2.8MB within 12 sec, with multicasting using the Modulation and Coding Scheme (MCS) at 26Mb/s. Higher MCS modes become viable by exploiting the rateless properties of RaptorQ codes to radically enhance the reliably of multicast transmissions.

P.6.d.004 Transient inactivation of nucleus accumbens reduces morphine place preference in rats

Multicast is an efficient way of transmitting the same set of data to multiple interested users. WLANs do not provide any standardized solution for reliable data multicast. Multicast packets are delivered to multiple users as a simple broadcast service without support for Automatic Repeat Request. Hence, multicast transmission results in higher packet error rates. In order to improve the reliability of multicast delivery, a fixed low speed (robust) transmission mode is generally required. However, this results in the inefficient use of scarce and valuable bandwidth. This paper presents a reliable and efficient Wi-Fi multicast delivery solution for challenging outdoor applications. An Application Layer Forward Error Correction enabled data carousel is proposed for reliable multicast transmission over standard IEEE 802.11 WLANs. Field trials have been conducted to demonstrate and quantify system performance. Results show that the proposed system enhances user quality of experience while time efficiently utilising available bandwidth. The proposed interleaved carousel that is Raptor code enabled, attains 74.5% less download time than the traditional carousel, because of randomisation of errors and the protection of Raptor AL-FEC. 90% of users are able to download successfully a video file of 2.8MB within 12 sec, with multicasting using the Modulation and Coding Scheme (MCS) at 26Mb/s. Higher MCS modes become viable by exploiting the rateless properties of RaptorQ codes to radically enhance the reliably of multicast transmissions.

Applying data carousel to improve search efficiency in P2P system

In recent years, advanced data broadcasting services, which provide viewers with multimedia data on digital television, have drawn growing interest. At the same time, there is file-sharing peer-to-peer service on digital television field. However, few studies pay attention to their convergence that is data broadcasting can help peer-to-peer service. This paper studies file-sharing peer-to-peer application among set-top boxes in bidirectional HFC networks. The study proposes the new architecture that using data carousel completes search work in peer-to-peer system. Finally, the experiment demonstrates advantages of the proposed strategy in hybrid peer-to-peer system.

Scheduling Data Broadcast Based on Multi-Frequency in Mobile Interactive Broadcasting

Considering digital multimedia broadcasting (DMB) with reverse channel through cellular, PCS, and WLAM, we propose Multi-Frequency Scheduling (MFS) algorithm for the data carousel. At the start of each carousel, the server/station chooses timely items and sequences them to configure a data carousel, regarding the statistics of client requests. Partitioning whole items into two sets, hot and cold, according to the request ratio, items have different broadcast frequency and preference when making a carousel. Simulations show that our proposed MFS features that the server answers as many client's requests as possible, while reducing clients' expected waiting time

An Interactive Service Platform Solution Based On Enhanced Data Carousel Scheme

The interactive service platform (ISP) provides a whole solution for interactive digital TV (DTV) from front-end to back-end. The ISP mainly consists of a data broadcast system (DBS) in front-end and an application execution system (AES) in back-end. In interactive DTV domain, interactivity is obtained through interactive applications that can be transmitted in broadcast networks by means of a specified carousel scheme. Since additional DSM-CC messages are used to encapsulate applications, the carousel scheme introduces redundant information that leads to bandwidth redundancy in ISP. This paper presents a whole solution for interactive service platform in interactive DTV domain. The architectures of the DBS and AES and their functional elements are introduced respectively. As the basis of interactive application transmission, the enhanced data carousel(EDC) scheme derived from DVB data carousel is described in detail. Compared to DVB object carousel scheme, the EDC scheme introduces less redundant information, shows a better performance in bandwidth redundancy and presents to be more compact. Optimization strategies applying to broadcast application execution environment based on EDC scheme in AES are also presented. The proposed platform has been implemented and the experimental result shows a good performance.

Integrating packet‐level FEC with data carousels for reliable content delivery in satellite broadcast/multicast systems

AbstractOne key aspect of digital multimedia broadcasting is the reliable point‐to‐multipoint distribution of content. Since the capacity and energy constraints in wireless environments do not favour the provision of a return channel for user feedback, the use of partial reliability techniques is often the only realistic option for the reliable transport layer design. In this paper, we focus on the two main reliable transport mechanisms in unidirectional, point‐to‐multipoint systems, namely packet‐level forward error correction (FEC) and data carousels. We approach them as building components of an integrated scheme and investigate its performance via analytical means. Our analysis demonstrates that the network responsiveness, expressed by the average content access time, is optimized for certain packet‐level FEC redundancy values. This is clearly different from setting FEC without considering the data carousel dimension, where the FEC redundancy is determined from the probability of recovering the whole file versus FEC overhead trade‐off curves. We describe design alternatives for both scheme components, such as different FEC code types, rules for assigning FEC redundancy per carousel item, and ways to retrieve items from the data carousel, and evaluate their impact on the performance of the scheme. Our results suggest that the superposition of FEC on data carousels mitigates the otherwise significant impact of the data item retrieval technique on performance, at least for close‐to‐optimal FEC settings. On the contrary, the careful selection of FEC code and FEC redundancy assignment rule for data carousel items results in performance gains of up to 11 and 18% for the average content access time and FEC overhead, respectively, depending on the item demand and length distributions. Copyright © 2006 John Wiley & Sons, Ltd.

Cross‐layer protocols for satellite communication networks: part II

Cross‐layer protocols for satellite communication networks: part II

Performance analysis of a file delivery system based on the FLUTE protocol

AbstractThis paper presents the results of performance tests carried out for a file delivery system based on the file delivery over unidirectional transport (FLUTE) protocol. FLUTE is a file transport protocol used to deliver files over IP networks, including the Internet and unidirectional systems, from a sender to one or more receivers. FLUTE uses UDP, an unreliable transport protocol, and so reliable delivery must be guaranteed by other means. This paper shows how FLUTE manages to recover from packet losses using forward path redundancy (forward error correction (FEC) and repeat transmissions in a data carousel), and a simple HTTP‐based point‐to‐point file repair scheme which is specified in 3GPP and DVB standards. The results presented in this paper show that careful optimization of FEC overhead, and the number of repeat transmissions, gives the best system performance in most cases. Based on the simplified error reception and distribution model depicted in this study, it is illustrated that the simple client‐server point‐to‐point file repair is optimal only for small groups. Several options to improve the configuration of FLUTE senders are provided, to deliver reception guarantees with optimal data expense from the system point of view. Copyright © 2006 John Wiley & Sons, Ltd.

Itinerant Delivery of Popular Data via WIDE Hot Spots

Wireless Information Delivery Environment (WIDE) is a distributed data dissemination system, which uses IEEE 802.11b technology. WIDE aims to deliver popular information services to registered mobile clients in WLAN hot spots. Data delivery is based on broadcasting and multicasting to provide scalability and efficient use of the wireless channel. Reliability is assured with a combination of Forward Error Correction (FEC), data carousel, and ARQ techniques. This paper presents the proposed system architecture with the details of reliable and secure data dissemination mechanisms. Functional evaluation of the proposed system and mechanisms on the implemented prototype are also included in this paper.

オブジェクト連動データ放送システムのメタデータの伝送手法とその評価実験

We propose a new up-to-the-minute data broadcasting service that uses the latest digital technologies. We produced prototype and carried out experiments to evaluate the performance of the system. The proposed service—the object-linked broadcasting service—provides various information regarding an object chosen by a viewer from a TV picture. We encountered some technical issues, such as observing broadcasting regulations, making dynamic descriptions for the objects, and working with one way transmission, in perfecting the system. To resolve these issues, we employed BML for browsing, ECMA Script for the dynamic description, Binary Table for the data structure, and DSM-CC for the transmission method. The experimental results show the system performance is acceptable. We found that an 80-second data carousel can carry related data for up to 20 objects.

Design and implementation of broadcast file system based on DSM-CC data carousel protocol

This paper presents a solution for the broadcast file system based on digital storage media command and control (DSM-CC) data carousel protocol. Given the complexity of the object carousel protocol, we innovatively modify the data carousel protocol so that the hierarchical directory structure can be transmitted. The corresponding software has been implemented in set-top boxes. With the help of the virtual filesystem switch (VFS) mechanism, the broadcast file system can be accessed through unified interfaces. Considering the differences between the broadcast file system and the traditional file system, this paper also proposes the optimized strategies while receiving and caching data from the network in order to show better performance.