A distributed evolutionary method to design scheduling policies for volunteer computing

Abstract

Volunteer Computing (VC) is a paradigm that uses idle cycles from computing resources donated by volunteers and connected through the Internet to compute large-scale, loosely-coupled simulations. A big challenge in VC projects is the scheduling of work-units across heterogeneous, volatile, and error-prone computers. The design of effective scheduling policies for VC projects involves subjective and time-demanding tuning that is driven by the knowledge of the project designer. VC projects are in need of a faster and project-independent method to automate the scheduling design.To automatically generate a scheduling policy, we must explore the extremely large space of syntactically valid policies. Given the size of this search space, exhaustive search is not feasible. Thus in this paper we propose to solve the problem using an evolutionary method to automatically generate a set of scheduling policies that are project-independent, minimize errors, and maximize throughput in VC projects. Our method includes a genetic algorithm where the representation of individuals, the fitness function, and the genetic operators are specifically tailored to get effective policies in a short time. The effectiveness of our method is evaluated with SimBA, a Simulator of BOINC Applications. Contrary to manually-designed scheduling policies that often perform well only for the specific project they were designed for and require months of tuning, our resulting scheduling policies provide better overall throughput across the different VC projects considered in this work and were generated by our method in a time window of one week.