Abstract
Following the success of the XRootd-based US CMS data federation, the AAA project investigated extensions of the federation architecture by developing two sample implementations of an XRootd, disk-based, caching proxy. The first one simply starts fetching a whole file as soon as a file open request is received and is suitable when completely random file access is expected or it is already known that a whole file be read. The second implementation supports on-demand downloading of partial files. Extensions to the Hadoop Distributed File System have been developed to allow for an immediate fallback to network access when local HDFS storage fails to provide the requested block. Both cache implementations are in pre-production testing at UCSD.
Highlights
In February 2013 the CMS experiment [1] at the CERN LHC finished its first data taking period, called “Run 1,” and entered into the “Long Shutdown 1” period expected to last until spring 2015
The physics analyses of the harvested data are still ongoing, as are the detector simulations and related computing activities required for an efficient commencement of the upcoming “Run 2.”
The 20 PB of data from the experiment in various formats is distributed among participating Tier 0, Tier 1 and Tier 2 computing sites with the goal of optimizing the usage of available computing resources as well as to provide sufficient processing power to all physicists that require access to the data
Summary
In February 2013 the CMS experiment [1] at the CERN LHC finished its first data taking period, called “Run 1,” and entered into the “Long Shutdown 1” period expected to last until spring 2015. This paper presents two implementations of an XRootd, disk-based caching proxy developed in the context of the AAA project. 2. Two implementations of a disk-based caching proxy Since CMS data federation already relies completely on XRootd to provide remote file access, the decision to base caching proxies on XRootd was an obvious one.
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
