Safe deployment of a reinforcement learning robot using self stabilization

Abstract

• In RL applications, there is no guarantee that a robot will operate within the same state space in which it was trained. • This paper focuses on ensuring the safe deployment of a robot. • This work defines a condition on the state and action spaces, that if satisfied, guarantees safe recovery. • We also propose a strategy and design that facilitate this recovery within a finite number of steps after perturbation. In toy environments like video games, a reinforcement learning agent is deployed and operates within the same state space in which it was trained. However, in robotics applications such as industrial systems or autonomous vehicles, this cannot be guaranteed. A robot can be pushed out of its training space by some unforeseen perturbation, which may cause it to go into an unknown state from which it has not been trained to move towards its goal. While most prior work in the area of RL safety focuses on ensuring safety in the training phase, this paper focuses on ensuring the safe deployment of a robot that has already been trained to operate within a safe space. This work defines a condition on the state and action spaces, that if satisfied, guarantees the robot’s recovery to safety independently. We also propose a strategy and design that facilitate this recovery within a finite number of steps after perturbation. This is implemented and tested against a standard RL model, and the results indicate a significant improvement in performance.