Abstract
Considerable attention has been given to leverage a variety of smart city applications using unmanned aerial vehicles (UAVs). The rapid advances in artificial intelligence can empower UAVs with autonomous capabilities allowing them to learn from their surrounding environment and act accordingly without human intervention. In this paper, we propose a spatiotemporal scheduling framework for autonomous UAVs using reinforcement learning. The framework enables UAVs to autonomously determine their schedules to cover the maximum of pre-scheduled events spatially and temporally distributed in a given geographical area and over a pre-determined time horizon. The designed framework has the ability to update the planned schedules in case of unexpected emergency events. The UAVs are trained using the Q-learning (QL) algorithm to find effective scheduling plan. A customized reward function is developed to consider several constraints especially the limited battery capacity of the flying units, the time windows of events, and the delays caused by the UAV navigation between events. Numerical simulations show the behavior of the autonomous UAVs for various scenarios and corroborate the ability of QL to handle complex vehicle routing problems with several constraints. A comparison with an optimal deterministic solution is also provided to validate the performance of the learning-based solution.
Highlights
As the world becomes increasingly interconnected and technology-dependent, a new wave of smart applications is changing how we handle and confront everyday activities
In this paper, we have developed a generic framework for autonomous unmanned aerial vehicles (UAVs) while taking into account their limited battery capacity
We modeled a reward function that jointly considers the limited battery capacity of the flying units, the time windows of events, and the delays caused by their navigation between events to feed the adopted Q-learning method
Summary
As the world becomes increasingly interconnected and technology-dependent, a new wave of smart applications is changing how we handle and confront everyday activities. With the use of the uprising Internet-of-things (IoT) technologies, UAVs are involved in a wide range of applications in smart cities, such as capturing live events, monitoring traffic network, delivery of goods [3], disaster management, to name a few [4], [5]. Despite the benefits brought from the use of UAVs, several challenges should be addressed to ensure effective operation of the fleet and satisfying applications outputs. Such challenges revolve around the technical specifications of the UAVs and mainly their limited battery capacities [6], [7]. Optimized scheduling must be designed to effectively determine an activity plan that ensure safe execution of the UAV missions. In several applications, such as real-time operations and emergency situations, the ability
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
