Multi-Objective Multiagent Credit Assignment in NSGA-II Using Difference Evaluations

Abstract

Determining the contribution of an agent to a system-level objective function (credit assignment) is a key area of research in cooperative multiagent systems. Multi-objective optimization is a growing area of research, though mostly focused on single agent settings. Many real-world problems are multiagent and multi-objective, (e.g., air traffic management, scheduling observations across multiple exploration robots) yet there is little work on their intersection.In this work, we leverage recent advances in single-objective multiagent learning to address multi-objective domains. We focus on the impact of difference evaluation functions (which extracts an agent's contribution to the team objective) on the Non-dominated Sorting Genetic Algorithm-II (NSGA-II), a state-of-the-art multi-objective evolutionary algorithm. We derive multiple methods for incorporating difference evaluations into the NSGA-II framework, and test each in a multiagent rover exploration domain, which is a good surrogate for a wide variety of distributed scheduling and resource gathering problems. We show that how and where difference evaluations are incorporated in the NSGA-II algorithm is critical, and can either provide significant benefits or destroy system performance, depending on how it is used. Median performance of the correctly used difference evaluations dominates best-case performance of NSGA-II in a multiagent multi-objective problem.