Evaluation of Reinforcement and Deep Learning Algorithms in Controlling Unmanned Aerial Vehicles

Yalew Zelalem Jembre,Syed Hassan Ahmed Shah,Rajib Paul,Yuniarto Wimbo Nugroho,Beomjoon Kim,Muhammad Attique,Muhammad Toaha Raza Khan

doi:10.3390/app11167240

Yalew Zelalem Jembre, Syed Hassan Ahmed Shah + Show 5 more

Open Access

https://doi.org/10.3390/app11167240

Copy DOI

Abstract

Unmanned Aerial Vehicles (UAVs) are abundantly becoming a part of society, which is a trend that is expected to grow even further. The quadrotor is one of the drone technologies that is applicable in many sectors and in both military and civilian activities, with some applications requiring autonomous flight. However, stability, path planning, and control remain significant challenges in autonomous quadrotor flights. Traditional control algorithms, such as proportional-integral-derivative (PID), have deficiencies, especially in tuning. Recently, machine learning has received great attention in flying UAVs to desired positions autonomously. In this work, we configure the quadrotor to fly autonomously by using agents (the machine learning schemes being used to fly the quadrotor autonomously) to learn about the virtual physical environment. The quadrotor will fly from an initial to a desired position. When the agent brings the quadrotor closer to the desired position, it is rewarded; otherwise, it is punished. Two reinforcement learning models, Q-learning and SARSA, and a deep learning deep Q-network network are used as agents. The simulation is conducted by integrating the robot operating system (ROS) and Gazebo, which allowed for the implementation of the learning algorithms and the physical environment, respectively. The result has shown that the Deep Q-network network with Adadelta optimizer is the best setting to fly the quadrotor from the initial to desired position.

Highlights

In recent times, drone or unmanned aerial vehicle (UAV) technology has advanced significantly, and it can be applied in the military sector and in civilian areas, such as in search and rescue (SAR) and package shipment, due to its high mobility and large overload maneuver [1]
A quadrotor or quadcopter is a type of UAV with four rotors designed in a cross configuration with two pairs of opposite rotors rotating in the clockwise direction, whereas the other rotor pair rotates in a counter-clockwise direction to balance the torque [5]
Evaluation we first explain the parameters used for the simulation and follow with the result obtained from the experiment

Summary

Introduction

Drone or unmanned aerial vehicle (UAV) technology has advanced significantly, and it can be applied in the military sector and in civilian areas, such as in search and rescue (SAR) and package shipment, due to its high mobility and large overload maneuver [1]. Many researchers worldwide are working to address issues related to UAVs. we focus on the application, performance, and implementation of machine learning algorithms for controlling UAVs. Even though there are several types of UAVs, such as fixed wings, quadrotors, blimps, helicopters, and ducted fan [2], due to its small size, low inertia, maneuverability, and cheap price, the quadrotor had become an industry favorite [3]. A quadrotor or quadcopter (the terms UAV, drone, quadcopter, and quadrotor are used interchangeably) is a type of UAV with four rotors designed in a cross configuration with two pairs of opposite rotors rotating in the clockwise direction, whereas the other rotor pair rotates in a counter-clockwise direction to balance the torque [5].

Objectives

Methods

Results

Discussion

Conclusion