Energy Utilization Task Scheduling for MapReduce in Heterogeneous Clusters

Abstract

Nowadays, energy costs are the most important factor in cloud computing. Therefore, the implementation of energy-aware task scheduling methods is of utmost importance. A task scheduling framework considering deadlines, data locality and resource utilization is proposed to save on energy costs in heterogeneous clusters. The framework consists of task list construction, task scheduling and slot list updating. In terms of deadline constraints, number of job slots allocated and possible processing times of jobs, a new job sequence is proposed to construct an reasonable task list. Tasks are scheduled to promising slots from their rack-local servers, cluster-local servers and remote servers in the produced task scheduling, which greatly improves data locality. After the assignment among tasks and slots, an update of available slots in clusters is proposed not only to find available slots but also to improve server resource utilization using fuzzy logic with the available number of slots according to current CPU, memory and bandwidth utilization. Experimental results show that the proposed heuristic results in lower energy consumption than the adapted existing algorithms with a variable total number of slots.