Uncertainty-Aware Autonomous Mobile Robot Navigation with Deep Reinforcement Learning

Abstract

Autonomous robots are capable of making decisions based on the information they can obtain from their environments, as opposed to simply following a program. Reinforcement learning is a machine learning paradigm that is widely used in autonomous robotics, since their principles are very similar. Reinforcement learning is based on how humans and animals perceive, reason and act on their environments by trial-and-error, learning which actions are better and which are worse depending on the reward they produce. Likewise, robots interact with their environments using their sensors to perceive and their actuators, motors and end effectors to actuate. The main drawback of reinforcement learning is the steep computational cost in processes that involve large state and action spaces. However, this problem has been mitigated by the use of deep learning, which has proved to be an outstanding solution by automatically discovering relevant features and representations in raw and high-dimensional data. This combination results in a new paradigm known as deep reinforcement learning, that has been successfully employed in robotic tasks such as navigation and manipulation. Developments in robotics have enabled the presence of robots in increasingly dynamic, unpredictable, unstructured and partially observable environments, wherein uncertainty is a key element that stems from the lack of information. Thus, the ability to cope with uncertainty is crucial for autonomous robots. However, this scarcity of information has been widely ignored in most robotics works for the sake of simplicity, leading to inaccurate, approximate and unsafe solutions. In order to improve the navigation as well as other robotic tasks and, therefore, the safety of the robot itself and all the objects and beings that surround it, more research is needed on the different approaches and methods for dealing with uncertain information. This chapter aims to provide an overview of deep reinforcement learning algorithms and strategies for mobile robot navigation with handling of uncertainty.