Mean-field controllability and decentralized stabilization of Markov chains

Abstract

In this paper, we present several novel results on controllability and stabilizability properties of the Kolmogorov forward equation of a continuous time Markov chain (CTMC) evolving on a finite state space, using the transition rates as the control parameters. First, we characterize all the stationary distributions that are stabilizable using time-independent control parameters. We then present a result on small-time local and global controllability of the system from and to strictly positive equilibrium distributions when the underlying graph is strongly connected. Additionally, we show that any target distribution can be reached asymptotically using time-varying control parameters. For bidirected graphs, we construct rational and polynomial density feedback laws that stabilize strictly positive stationary distributions while satisfying the additional constraint that that the feedback law takes zero value at equilibrium. This last result enables the construction of decentralized density feedback controllers, using tools from linear systems theory and sum-of-squares based polynomial optimization, that stabilize a swarm of agents modeled as a CTMC to a target state distribution with no state-switching at equilibrium. We validate the effectiveness of the constructed feedback laws with stochastic simulations of the CTMC for finite numbers of agents and numerical solutions of the corresponding mean-field models.