Autonomous Learning Agents: Layered Learning and Ad Hoc Teamwork

Abstract

In order to achieve long-term autonomy in the real world, fully autonomous agents need to be able to learn, both to improve their behaviors in a complex, dynamically changing world, and to enable interaction with previously unfamiliar agents. This talk begins by presenting layered learning, a hierarchical machine learning paradigm that enables learning of complex behaviors by incrementally learning a series of sub-behaviors. Layered learning was the key deciding factor in UT Austin Villa's recent RoboCup 3D simulation league championship. The talk then introduces teamwork as an emerging multiagent learning challenge. Ad teamwork is based on the premise that as autonomous agents become capable of long-term autonomy, they will increasingly need to band together for cooperative activities with previously unfamiliar teammates. In such ad hoc team settings, team strategies cannot be developed a priori. Rather, an agent must learn to cooperate with new teammates on the fly. This talk reports on both theoretical and empirical teamwork results, including from recent pick up RoboCup robot soccer competitions.