Graph Patterns, Reinforcement Learning and Models of Reputation for Improving Coalition Formation in Collaborative Multi-agent Systems

Abstract

We outline a new model of multi-agent coalition formation which focuses on how collaborative agents can improve their coalition formation skills over time, learning from their prior interactions. The proposed research direction builds on our prior work on distributed coalition formation in collaborative multi-agent systems (MAS), centered at partitioning the underlying network of agents into non-overlapping cliques. At the core of that prior research is the MCDCF algorithm which provides a semantically simple, fully decentralized, local and (for sufficiently sparse networks) scalable mechanism for multi-agent coalition formation [1, 2, 3]. Our goal is to extend the MCDCF-based coalition formation along several new dimensions. First, we want to consider candidate coalitions that (i) no longer have to be cliques but can be more general types of (connected) subgraphs, and (ii) that also satisfy additional, more complex “compatibility” properties stemming from individual agents’ capabilities and preferences. Second, we begin exploration of semantically more rich and versatile ways of capturing this inter-agent compatibility than what’s found in the existing literature. In particular, we propose applying graph pattern techniques to capture a variety of qualitative “compatibility relationships” among agents. Next, we revisit approaches to and benefits of reinforcement learning (RL) in the context of autonomous agents repeatedly engaging in coalition formation. Last but not least, we discuss benefits of each agent maintaining other agents’ reputations that quantify those agents’ coalition formation effectiveness in the past. With these extensions, we argue that the resulting modeling framework adequately captures core aspects of a much richer class of multi-agent coalition formation scenarios, as well as, more broadly, of a variety of distributed consensus reaching problems in collaborative MAS.