Remote Proximity Sensing With a Novel Q-Learning in Bluetooth Low Energy Network

Abstract

This paper presents a novel Q-Learning method in forwarding the proximity sensing information to the remote server through low-power mesh network overlays on Bluetooth Low Energy (BLE) technology. Even though proximity sensing information can be easily monitored with pervasive smartphones, it is almost impossible to remotely monitor this information in a harsh location where it is not easy to access the Internet. With our overlay mesh network, each node should decide to either forward the packet or continue with their own activity when receiving the packet forwarding request, so as to minimize the end-to-end packet delivery latency but maximize the utilization of underlying infrastructures. Reinforcement learning (RL) is employed to train each node to make the above decision. Despite extensive upfront training, there is a high possibility that each node might still encounter an unseen state owing to the network dynamics. However, our novel Q-learning is able to deal with above challenges by constructing a Q-table during online learning, and then use the Q-table as input data for offline training. The experimental results indicate the substantial performance gain of our proposed approach in comparison to the existing Q-learning methods.