Abstract
Over the last two decades, ROOT TTree has been used for storing over one exabyte of High-Energy Physics (HEP) events. The TTree columnar on-disk layout has been proved to be ideal for analyses of HEP data that typically require access to many events, but only a subset of the information stored for each of them. Future colliders, and particularly HL-LHC, will bring an increase of at least one order of magnitude in the volume of generated data. Therefore, the use of modern storage hardware, such as low-latency high-bandwidth NVMe devices and distributed object stores, becomes more important. However, TTree was not designed to optimally exploit modern hardware and may become a bottleneck for data retrieval. The ROOT RNTuple I/O system aims at overcoming TTree’s limitations and at providing improved effciency for modern storage systems. In this paper, we extend RNTuple with a backend that uses Intel DAOS as the underlying storage, demonstrating that the RNTuple architecture can accommodate high-performance object stores. From the user perspective, data can be accessed with minimal changes to the code, that is by replacing a filesystem path by a DAOS URI. Our performance evaluation shows that the new backend can be used for realistic analyses, while outperforming the compatibility solution provided by the DAOS project.
Highlights
1 Introduction In High-Energy Physics (HEP), an event is encoded as a record that may contain a number of variable-length collections or properties
We present the results of the experimental evaluation of the RNTuple DAOS backend we carried out
We present an extension to RNTuple which allows users to store HEP data in a DAOS container, closing the gap between the HEP community and the nextgeneration HPC centers
Summary
In High-Energy Physics (HEP), an event is encoded as a record that may contain a number of variable-length collections or properties. Most analysis of HEP data require access to many events, but only require a subset of the properties stored for each instance In this scenario, using conventional row-oriented storage systems is suboptimal, as they incur in an overhead due to reading a high volume of unneeded data. ROOT TTree [1] avoids this overhead by using a columnar layout, i.e., consecutively storing values of the same property for a range of rows This encoding avoids the aforementioned overhead, and contributes to improve data compression as similar values are stored together. The layered design of RNTuple permits the separation of encoding and storage of pages (groups of values belonging to the same data column) This separation is crucial for supporting different storage systems, such as POSIX files or object stores. For the sake of space, we will only summarize the high-level architecture
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
