Chapter Ten - Model-Free Learning of Nash Games With Applications to Network Security

Abstract

We propose novel mathematical frameworks for model-free learning algorithms for games on complex systems with application to network security. We will use cyber-physical systems with sparse communication that can however yield global mission and provide formal optimality and robustness guarantees. Given the presence of modeling uncertainties, the unavailability of the model, the possibility of cooperative/noncooperative goals, and malicious attacks compromising the security of networked teams, there is a need for completely model-free plug-and-play approaches that respond to situations not programmed or anticipated in the design, so as to guarantee mission completion. We will be inspired by the prefrontal cortex and basal ganglia of the human brain and will combine interdisciplinary ideas from different fields (ie, computational intelligence, game theory, control theory, and information theory) to develop new self-configuring algorithms for decision and control given the unavailability of a model, the presence of enemy components, and the possibility of actuation and jamming network attacks. Because of the adaptive nature of the algorithms, the systems will be capable of breaking or splitting into parts that are themselves autonomous and resilient. The algorithms will be characterized by strong abilities of self-learning and adaptivity and will be implemented locally, and not globally, which implies that fewer sensors will be used for measurements. The approaches will be based on machine learning and especially in distributed actor-critic neural network structures, where the actor neural networks will approximate the optimal policies and the critic neural networks will approximate the optimal cost for each agent in the network.