Abstract
HPC system usually uses a shared filesystem like Lustre as temporary file storage like scratch directory. It needs automated purge process to remove unused files for maintaining optimal performance of the shared filesystem. However, the purge process in large capacity file system takes much time to search and remove target files. In this paper, accelerated purge processes using MPI are proposed. First, master/slave parallel purge (MSPP) process is the method that master node distributes purge tasks among slave nodes. Second, evenly distributed purge (EDPP) process is the method that all node are involved in purge process that improves load balancing. Experimental results show that the purging time of proposed EDPP method for 1958 GB scratch data has been reduced by 4.04 and 2.47 times, respectively, when it is compared with the results of single node purge (SNP) and MSPP methods.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
