Utilizing idle CPU cycles in a distributed computing system

Abstract

Distributed computing systems contain frequently large numbers of idle workstations. Batch job scheduling systems exist that assign jobs to idle workstations and control the job execution in a way that interactive users are impacted as little as possible. However, when workstations are used for interactive as well as batch loads, compromises have to be made. Either interactive users are favored resulting in a reduction of the available CPU cycles for batch job processing, or batch jobs are prioritized which may disturb interactive users. By analyzing the usage records of an interactively used workstation cluster of more than 100 machines, we could quantify the loss of available CPU cycles for batch job processing against the collisions with interactive users. We observed many unused resources even during working-hours. Due to the fact that in this environment the resources available for batch job processing are changing dynamically and cannot be predicted exactly, current scheduling algorithms often deliver bad job turnaround times. We have developed a novel batch job scheduling algorithm which considers the usage history of individual workstations and delivers better job turnaround times. A newly built tool allows to evaluate the performance of different scheduling algorithms under the same environment conditions.