Efficient Safe Control via Deep Reinforcement Learning and Supervisory Control – Case Study on Multi-Robot Warehouse Automation

Abstract

Safe control has recently attracted much attention due to its applications in safety-critical cyber-physical systems. Supervisory control theory (SCT) is a formal control method that provides correct-by-construction safety certificates, but is computationally inefficient when the number of system components is large. On the other hand, deep reinforcement learning (DRL) provides a toolbox of efficient algorithms to compute control decisions even for very large state space, but does not always guarantee safety. In this paper, we propose to synergize SCT and DRL into a new efficient safe control approach. Specifically, we first employ DRL algorithms to efficiently compute sub-optimal solutions which may be unsafe; then we convert the obtained solutions into a standard supervisory control problem with an automaton (plant model) and a set of unsafe states (safety specification); finally we use SCT to synthesize a supervisor with a safety certificate. A case study of multi-robot warehouse logistic automation is conducted to demonstrate the efficiency of this proposed approach.