A Hybrid Dynamical Systems Perspective on Reinforcement Learning for Cyber-Physical Systems: Vistas, Open Problems, and Challenges

Abstract

The next generation of autonomous cyber-physical systems will integrate a variety of heterogeneous computation, communication, and control algorithms. This integration will lead to closed-loop systems with highly intertwined interactions between the digital world and the physical world. For these systems, designing robust and optimal data-driven control algorithms necessitates fundamental breakthroughs at the intersection of different areas such as adaptive and learning-based control, optimal and robust control, hybrid dynamical systems theory, and network control, to name just a few. Motivated by this necessity, control techniques inspired by ideas of reinforcement learning have emerged as a promising paradigm that could potentially integrate most of the key desirable features. However, while significant results in reinforcement learning have been developed during the last decades, the literature is still missing a systematic framework for the design and analysis of reinforcement learning-based controllers that can safely and systematically integrate the intrinsic continuous-time and discrete-time dynamics that emerge in cyber-physical systems. Motivated by this limitation, and by recent theoretical frameworks developed for the analysis of hybrid systems, in this chapter we explore some vistas and open problems that could potentially be addressed by merging tools from reinforcement learning and hybrid dynamical systems theory, and which could have significant implications for the development of the next generation of autonomous cyber-physical systems.