Energy Cooperation in Ultradense Network Powered by Renewable Energy Based on Cluster and Learning Strategy

Abstract

A new method about renewable energy cooperation among small base stations (SBSs) is proposed, which is for maximizing the energy efficiency in ultradense network (UDN). In UDN each SBS is equipped with energy harvesting (EH) unit, and the energy arrival times are modeled as a Poisson counting process. Firstly, SBSs of large traffic demands are selected as the clustering centers, and then all SBSs are clustered using dynamic k-means algorithm. Secondly, SBSs coordinate their renewable energy within each formed cluster. The process of energy cooperation among SBSs is considered as Markov decision process. Q-learning algorithm is utilized to optimize energy cooperation. In the algorithm there are four different actions and their corresponding reward functions. Q-learning explores the action as much as possible and predicts better action by calculating reward. In addition, ε greedy policy is used to ensure the algorithm convergence. Finally, simulation results show that the new method reduces data dimension and improves calculation speed, which furthermore improves the utilization of renewable energy and promotes the performance of UDN. Through online optimization, the proposed method can significantly improve the energy utilization rate and data transmission rate.