Using deep reinforcement learning to train and periodically re-train a data-collecting drone based on real-life measurements

Abstract

In a smart city, many different kinds of smart meters (i.e., IoT devices) are used to help govern the city and their data need to be periodically collected for billing or analytics purposes. In our previous work (Wang et al., 2021), we designed and implemented a drone-based wake-up and data collection system for smart meters by using wake-up radios (WuR) and WiFi. Using this system, in this work we use the deep reinforcement learning (DRL) method to train and periodically re-train the drone agent so that it can optimally collect the data of smart meters deployed on a field. For a specific neighborhood, our method uses the results of many real-life wireless transmission measurements in the neighborhood as the initial input to the DRL method. As a result, the trained agent can direct the drone to finish the data collection task of the neighborhood more efficiently.We used the actions recommended by the trained agent to direct the drone to collect the data of ten smart meters deployed on our campus. Experimental results show that the trained agent enabled the drone to finish the data collection task with a much shorter flying distance, as compared with other methods that do not consider real-life neighborhood-specific geographic effects on wireless communication. For example, compared with a greedy method, our method without re-training the drone agent can reduce 19.7% of the flying distance. In each real-life data collection task, our method also records the actual wake-up and data collection success/failure results of all smart meters and periodically uses a batch of these data to re-train the drone agent. Experimental results show that such a periodic re-training design can continue to improve the performance of the drone agent for a specific neighborhood. For example, compared with our method without re-training the drone agent, our method with re-training the drone agent can further reduce 14.6% of the flying distance.