Adaptive Resource Allocation with Job Runtime Uncertainty

Abstract

In this paper, we address the problem of dynamic resource allocation in presence of job runtime uncertainty. We develop an execution delay model for runtime prediction, and design an adaptive stochastic allocation strategy, named Pareto Fractal Flow Predictor (PFFP). We conduct a comprehensive performance evaluation study of the PFFP strategy on real production traces, and compare it with other well-known non-clairvoyant strategies over two metrics. In order to choose the best strategy, we perform bi-objective analysis according to a degradation methodology. To analyze possible biasing results and negative effects of allowing a small portion of the problem instances with large deviation to dominate the conclusions, we present performance profiles of the strategies. We show that PFFP performs well in different scenarios with a variety of workloads and distributed resources.