An AutoML Based Algorithm for Performance Prediction in HPC Systems

Abstract

Neural networks are extensively utilized for building performance prediction models for high-performance computing systems. It is challenging to construct the neural network architecture that provides accurate performance (runtime) predictions by exploring the number of layers and neurons in each layer. Automated machine learning (AutoML) using neural architecture search (NAS) frameworks such as Auto-Keras have been proposed to tune neural networks automatically. Researchers have utilized AutoML frameworks for many application domains to build neural networks automatically, providing accurate predictions. However, AutoML has not been explored for performance prediction domain based on our literature survey. Hence, our goal is to show the feasibility of AutoML-based algorithm for performance prediction of HPC systems. In this paper, we propose a novel AutoML based algorithm that builds a neural network model layer-by-layer for performance prediction. Our algorithm takes decisions based on prediction accuracy metric values to add a new layer to tune the network. We have performed extensive experiments utilizing applications from SPEC CPU 2006, SPEC CPU 2017 benchmark suites. Performance prediction accuracy results of RMSE (Root Mean Squared Error), MedAE (Median Absolute Error) and MedAPE (Median Absolute Percentage Error) from our experiments demonstrate superior performance of our algorithm compared to the state-of-the-art Auto-Keras framework.