Abstract
Running HPC in the cloud has gained more and more practice. As a new cloud paradigm, serverless is highly attractive for HPC service providers due to its distinctive benefits such as scalability. However, it is difficult for serverless to meet the demands of MPI programming and running, resulting in that MPI programs cannot scale with serverless functions. We introduce a serverless parallel function model to solve the problems. It divides parallelism at function and worker levels to bridge gaps in programming and running between serverless and MPI. Then we present the SMPI framework atop the model. For programming, SMPI redefines the function generation pipeline for parallel functions to prepare metadata for MPI parallel functions. For running, SMPI employs the parallel function gateway and scheduler to realize parallel function invocation and instantiation for MPI parallel functions. It is implemented and evaluated with OpenFaaS. Experiments show that SMPI supports MPI programming and running in a complete serverless manner. Compared to server-centric methods, it reduces efforts on cluster maintenance, provides scalable serverless MPI computing with competitive performance (0.559–1.048s slower of start-up time, and 0.145s–0.945s slower of computing time than best-behaved baseline), and is potential to scale on multiple clusters for higher scalability.
Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
