Building an HPC-as-a-Service Toolkit for User-Interactive HPC Services in the Cloud

Abstract

It is a challenging task to provide High Performance Computing (HPC) services in the cloud. The estimation of the performance is extremely difficult, and it is affected by the capacity of system (e.g., the number of CPU cores, the access speed of network storage, and the communications throughput among multiple processors) as well as the feature of applications. In this paper, we propose to build a user-interactive HPC-as-a-Service environment, HPC-as-a-Service Toolkit (HPCaaST), to provide HPC services in the cloud. We introduce an adaptive HPC service method which includes pre-process configuration, adaptive estimations, and early data evaluation, to improve the user interactivities of HPC services. Our proposed adaptive method computes the completion times and prices of on-going HPC jobs in real-time, which enables users to start same jobs with different configurations simultaneously and select the best one at an early stage. We conduct experiments with Fire Dynamics Simulator and results show that with the adaptive estimation, users can save much money by selecting the best strategy from four configurations when 10%-20% of the simulation is completed.