Cooperation learning in Multi-Agent Systems with annotation and reward

Abstract

This paper proposes a novel approach for enabling agents to learn to cooperate each other based on annotation and reward in Multi-Agent Systems (MAS). We propose two methods toward cooperation learning in MAS: 1) a cooperation method, and 2) a learning method. As for 1), our method enables each agent to interact with other agents by sending its proposal and receiving the counterproposals for the proposal from the agents. The counterproposals are constructed by modifying the communicated proposal, and are utilized for clarifying the difference in opinions among agents. Conflict resolution is conducted to reduce the difference to facilitate cooperation. Furthermore, annotation, which acts as a kind of design rationale, is added onto the communicated proposal and counterproposals to facilitate conflict resolution. As for 2), we propose an extension of reinforcement learning method so that agents can learn the appropriate behavior based on the reward which is given through the interaction among agents. We define two kinds of reward for each agent: the reward for the proposal constructed by the agent, and the reward for coherence among agents. Comparative simulation studies for micro satellite design, which fits for cooperative problem solving by MAS, were conducted to evaluate the proposed approach. The results are encouraging and show that it is worth following this path. Especially, the results indicate that the appropriate balance between exploration and exploitation, which is important in cooperative problem solving in general, can be learned with the proposed approach.