Abstract
With the advent of HPC (High-Performance Computing) Cloud, storage systems based on parallel file systems such as Lustre have become one of important components to support various types of workloads from many users having different storage access patterns. One of the solutions to adapt to such various I/O access patterns is Progressive File Layout (PFL) which can adjust the chunk size and stripe pattern dynamically according to the I/O traffics. However, PFL still has a performance limitation to the small file I/O due to the HDD specific hardware characteristics. In this paper, we propose an optimized Progressive File Layout (PFL) method which can effectively leverage a combination of SSDs and HDDs to support mixed sizes of I/O workloads. By dynamically directing relatively small file I/O operations into the SSD based storage devices, our framework can significantly improve the performance. We have performed a comparative analysis of our PFL on SSDs in Lustre file system with PFL on HDDs and vanilla Lustre configuration with HDDs. Experimental results from various workloads with mixed sizes of I/O threads on our testbed show that the proposed PFL on SSDs can achieve enhanced total I/O throughput by more than six times compared to conventional solutions.
Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
