Interdatacenter Job Routing and Scheduling With Variable Costs and Deadlines

Abstract

To reduce their operational costs, datacenter (DC) operators can schedule large jobs at DCs in different geographical locations with time- and location-varying electricity and bandwidth prices. We introduce a framework and algorithms to do so that minimize electricity and bandwidth cost subject to job indivisibility, deadlines, priorities, and DC resource constraints. In doing so, we provide a way for DC operators to predict their operational costs for different DC placements and capacities, and thus make informed decisions about how to expand their DC network. Our distributed algorithm uses estimated job arrivals and day-ahead electricity prices to optimize over sliding time windows. We demonstrate its effectiveness on a Google DC trace and investigate the effects of different cost and performance criteria. The algorithm leverages heterogeneous job resource requirements and routing and scheduling flexibility: even deadline and indivisibility constraints yield little cost increase, though they significantly improve job completion times and localization at only one DC, respectively. We show that our algorithm reduces the cost much more than optimizing only electricity, only bandwidth, or a combination of resource costs and job completion times.