Improving Energy Efficiency and QoS of LPWANs for IoT Using Q-Learning Based Data Routing

Abstract

Recent proliferation of Internet of Things (IoT) demands large scale connectivity among smart IoT devices over a vast geographical area. However, limited radio range and lack of scalability of conventional wireless sensor networks do not allow a wide area connectivity among IoT devices. To address these challenges, Low-Power Wide-Area Networks (LPWANs) are emerging to provide long-range communication capability with low-power consumption of the end devices. Nevertheless, given the demand in delivering an increasingly large volume of data generated by IoT devices, the direct data transmission model is not suitable due to its poor network lifetime. Therefore, in this work, a multi-hop data routing method is proposed for LPWANs. Since multi-hop data transmission faces several challenges such as increased data latency, higher interference, and reduced data throughput (i.e., poor bandwidth utilization), we propose a reinforcement learning method to address those challenges. The proposed method updates the Q-matrix of the network at varying discrete time instants and selects relay devices in such a way that maximizes the cumulative reward value between selected device-gateway pairs. The applicability and effectiveness of the proposed method are illustrated over both simulated LPWAN testbed and real field data sets. The obtained results clearly demonstrate the improved network performance in terms of energy efficiency and QoS of the proposed method as compared to various existing methods.