Team Modeling with Deep Behavioral Cloning for the RoboCup 2D Soccer Simulation League

Abstract

Soccer is still considered an open problem by the AI community due to its complex stochastic real-time multiagent nature. The RoboCup Soccer Simulation 2D League has been used as a testbed for new ideas and techniques for many subjects, including team modeling. However, even though team modeling has been an indispensable part of the best league participants to date, in practice it typically consists of ad-hoc heuristics encoded as rules. This requires time-consuming manual work, does not scale to multiple teams, and does not work well at unaccounted scenarios. This paper presents a data-driven method for modeling teams by training Deep Neural Networks with large amounts of data with an Imitation Learning formulation. We demonstrate the approach by training a deep model of the Japanese team Helios using 57,578,668 state-action pairs of players. The resulting model achieved 84.5% accuracy on action selection and a small error on regression of the action parameters. The network is shown to be an effective movement predictor of Helios field players and have negligible degradation when Helios is evaluated against adversaries not seen at training time.