Model-Free design of control systems over wireless fading channels

Abstract

Wireless control systems rely on wireless networks to exchange information between spatially distributed actuators, plants and sensors. The noise in wireless channels renders traditional control policies suboptimal, and their performance is moreover directly dependent on the allocation of wireless resources between control loops sharing the wireless medium, making the design of control and resource allocation policies critical to achieve reliable performance of the wireless control system. The co-design of control-aware resource allocation policies and communication-aware controllers, however, is a challenging problem due to its infinite dimensionality, existence of constraints on the use of communication resources or performance of individual plants, and need for explicit knowledge of the plants and wireless network models. To overcome those challenges, we propose a constrained reinforcement learning approach to design model-free control-aware communication policies and communication-aware control policies for wireless control systems. We demonstrate the near optimality of control system performance and stability using near-universal policy parametrizations and present a practical model-free algorithm to learn the co-design policy. Numerical experiments show the strong performance of learned policies over baseline solutions.