Maximizing network throughput by cooperative reinforcement learning in clustered solar-powered wireless sensor networks

Abstract

Power management in wireless sensor networks is very important due to the limited energy of batteries. Sensor nodes with harvesters can extract energy from environmental sources as supplemental energy to break this limitation. In a clustered solar-powered sensor network where nodes in the network are grouped into clusters, data collected by cluster members are sent to their cluster head and finally transmitted to the base station. The goal of the whole network is to maintain an energy neutrality state and to maximize the effective data throughput of the network. This article proposes an adaptive power manager based on cooperative reinforcement learning methods for the solar-powered wireless sensor networks to keep harvested energy more balanced among the whole clustered network. The cooperative strategy of Q-learning and SARSA( λ) is applied in this multi-agent environment based on the node residual energy, the predicted harvested energy for the next time slot, and cluster head energy information. The node takes action accordingly to adjust its operating duty cycle. Experiments show that cooperative reinforcement learning methods can achieve the overall goal of maximizing network throughput and cooperative approaches outperform tuned static and non-cooperative approaches in clustered wireless sensor network applications. Experiments also show that the approach is effective in response to changes in the environment, changes in its parameters, and application-level quality of service requirements.