EETS: An energy-efficient task scheduler in cloud computing based on improved DQN algorithm

Abstract

The huge energy consumption of data centers in cloud computing leads to increased operating costs and high carbon emissions to the environment. Deep Reinforcement Learning (DRL) technology combines of deep learning and reinforcement learning, which has an obvious advantage in solving complex task scheduling problems. Deep Q Network(DQN)-based task scheduling has been employed for objective optimization. However, training the DQN algorithm may result in value overestimation, which can negatively impact the learning effectiveness. The replay buffer technique, while increasing sample utilization, does not distinguish between sample importance, resulting in limited utilization of valuable samples. This study proposes an enhanced task scheduling algorithm based on the DQN framework, which utilizes a more optimized Dueling-network architecture as well as Double DQN strategy to alleviate the overestimation bias and address the shortcomings of DQN. It also incorporates a prioritized experience replay technique to achieve importance sampling of experience data, which overcomes the problem of low utilization due to uniform sampling from replay memory. Based on these improved techniques, we developed an energy-efficient task scheduling algorithm called EETS (Energy-Efficient Task Scheduling). This algorithm automatically learns the optimal scheduling policy from historical data while interacting with the environment. Experimental results demonstrate that EETS exhibits faster convergence rates and higher rewards compared to both DQN and DDQN. In scheduling performance, EETS outperforms other baseline algorithms in key metrics, including energy consumption, average task response time, and average machine working time. Particularly, it has a significant advantage when handling large batches of tasks.