Abstract
The projected Storage and Compute needs for the HL-LHC will be a factor up to 10 above what can be achieved by the evolution of current technology within a flat budget. The WLCG community is studying possible technical solutions to evolve the current computing in order to cope with the requirements; one of the main focus is resource optimization, with the ultimate aim of improving performance and efficiency, as well as simplifying and reducing operation costs. As of today the storage consolidation based on a Data Lake model is considered a good candidate for addressing HL-LHC data access challenges. The Data Lake model under evaluation can be seen as a logical system that hosts a distributed working set of analysis data. Compute power can be “close” to the lake, but also remote and thus completely external. In this context we expect data caching to play a central role as a technical solution to reduce the impact of latency and reduce network load. A geographically distributed caching layer will be functional to many satellite computing centers that might appear and disappear dynamically. In this talk we propose a system of caches, distributed at national level, describing both deployment and results of the studies made to measure the impact on the CPU efficiency. In this contribution, we also present the early results on novel caching strategy beyond the standard XRootD approach whose results will be a baseline for an AI-based smart caching system.
Highlights
With the upcoming High Luminosity LHC (HL-LHC) [1] program at CERN all HEP experiments will face a new challenge, the exabyte era of computing [2]
The projected Storage and Compute needs for the HL-LHC will be a factor up to 10 above what can be achieved by the evolution of current technology within a flat budget
The WLCG community is studying possible technical solutions to evolve the current computing in order to cope with the requirements; one of the main focus is resource optimization, with the ultimate aim of improving performance and efficiency, as well as simplifying and reducing operation costs
Summary
With the upcoming High Luminosity LHC (HL-LHC) [1] program at CERN all HEP experiments will face a new challenge, the exabyte era of computing [2]. In order to cope with this, a series of R&D programs have been established with the purpose of finding viable solutions for the optimization of the computing models In this context the activity presented in this work focuses on the storage, looking for solutions in order to minimize the hardware usage and to increase performances, e.g. to improve CPU efficiency by reducing I/O latencies and, to introduce handles to simplify operations, which represent an important cost to the collaboration. In Sec. we report on the deployment done to integrate an INFN federation of distributed caches within the “Anydata, Anytime, Anywhere (AAA)” federation [7] of CMS This includes a summary of the studies made to measure the effect of data caches on CPU job efficiency. We conclude with the plan for the extension toward an ML-based mechanism for caching management
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
