Abstract
EOS is an open source distributed disk storage system in production since 2011 at CERN. Development focus has been on low-latency analysis use cases for LHC1 and non- LHC experiments and life-cycle management using JBOD2 hardware for multi PB storage installations. The EOS design implies a split of hot and cold storage and introduced a change of the traditional HSM3 functionality based workflows at CERN.The 2015 deployment brings storage at CERN to a new scale and foresees to breach 100 PB of disk storage in a distributed environment using tens of thousands of (heterogeneous) hard drives. EOS has brought to CERN major improvements compared to past storage solutions by allowing quick changes in the quality of service of the storage pools. This allows the data centre to quickly meet the changing performance and reliability requirements of the LHC experiments with minimal data movements and dynamic reconfiguration. For example, the software stack has met the specific needs of the dual computing centre set-up required by CERN and allowed the fast design of new workflows accommodating the separation of long-term tape archive and disk storage required for the LHC Run II.This paper will give a high-level state of the art overview of EOS with respect to Run II, introduce new tools and use cases and set the roadmap for the next storage solutions to come.
Highlights
EOS[1] is a multi-protocol disk-only storage system developed at CERN since 2010 to store physics analysis data physics experiments
The total storage space today is segmented into six independent failure domains for the four LHC experiments Alice, Atlas, CMS and Lhcb, a shared experiment instance for smaller experiments PUBLIC and a generic user instance USER for all CERN users
The current deployment with 140 PB of storage space and 44.000 hard disks demonstrated an unprecedented scalability for a low-cost storage system in high energy physics
Summary
EOS[1] is a multi-protocol disk-only storage system developed at CERN since 2010 to store physics analysis data physics experiments (including the LHC experiments). 1. Introduction EOS[1] is a multi-protocol disk-only storage system developed at CERN since 2010 to store physics analysis data physics experiments (including the LHC experiments). Capacity Server Hard Disks Files Directories Replicas theor. Disk BW Internal Messaging State Machine Users storing data Quota rules The state and configuration modifications are exchanged between storage and metadata servers using a message queue service MQ.
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
