PVBSSS: parallel validation-based shared-state scheduler

Abstract

Shared-state schedulers have been proposed for resolving the scalability limitation of monolithic schedulers in large-scale server clusters. Recent studies have explored how to apply optimistic concurrency control (OCC) strategies on scheduling on large-scale clusters. However, the naive OCC can only detect some simple conflicts, and is not efficient on placing long decision time jobs. In this paper, we present the parallel validation-based shared-state scheduler (i.e., PVBSSS). It is an extended shared-state scheduler that aims for mixed short and long jobs scheduling in large-scale shared clusters. In PVBSSS, parallel validation is implemented as the OCC algorithm to ensure strong serialisability for concurrent scheduling transactions. Furthermore, admission control and a scheduling strategy are implemented in PVBSSS to alleviate scheduling conflicts between schedulers. We experimentally evaluated our approach using the Omega scheduler's public simulator and a cluster comprising 30 work nodes. We conjecture that our experimental study not only demonstrates the feasibility and efficiency of PVBSSS, but also will help developers to make more informed decisions on both designing and managing scheduling on large-scale cloud platform.