Abstract
Inherited from the flexible architecture of Xrootd, Xcache allows a wide range of customization through configurations and plugin modules. This paper describes several completed and ongoing R&D efforts of using Xcache in the LHC ATLAS distributed computing environment, in particular, using Xcache with the ATLAS data management system Rucio for easy-to-use and to improve cache hit rate, to replace Squid and improve distribution of large files in CVMFS, to adapt the HPC environment and the data lake model for efficient data distribution and access for the HPCs.
Highlights
Xcache is a Squid-like cache, but it primarily uses the “xroot” (a.k.a. “root”) protocol [1], HTTP protocol being added on
The global Logical File Name (gLFN) is a concept initially developed by the ATLAS FAX project [10], and represents a location independent file path for users to access the data file via the FAX system and its redirection network (“/atlas/Rucio” identifies that this is a Rucio managed file for the ATLAS experiment; “scope:file” is the Rucio Data Identifier (DID) [11], which identifies the file in Rucio system)
The Rucio plugin for Xcache targets individual users or user groups so that they can just access their data without having to find out the ATLAS data placement and follow the change of the data placement, and without having to manage the storage space
Summary
Xcache is a Squid-like cache, but it primarily uses the “xroot” (a.k.a. “root”) protocol [1], HTTP protocol being added on. “root”) protocol [1], HTTP protocol being added on. It is a multi-threaded file caching application that can asynchronously fetch and cache file segments or whole files. Its primary design use case is caching static scientific data files of any format, large or small. A single Xcache can be deployed via container or CVMFS [3] for a user or a small user group, while a cluster of Xcache can be built for large or heavy use cases. Xcache works with Rucio [5] to improve cache hit rate and provide a location independent data access via the global logical file name. Xcache can use HTTP protocol with clients, and this capability is explored to replace Squid cache in the CVMFS data distribution chain. This paper will discuss work being done, as well as plans on using Xcache with HPCs and the data lake model [7] to efficiently distribute data and access data on HPCs
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
