XSS: A Soccer Server extension for automated learning of high-level robotic soccer strategies

Abstract

A solution for an optimal robotic soccer strategy is yet to be found. Multiple agents interacting in an environment with continuous state and action spaces is a recipe for machine learning lethargy. Fortunately, due to the increasing hardware performance and new algorithms, the area of reinforcement learning is growing considerably. However, simulating the environment for strategy purposes is not following this trend. Several simulators are available, including those used in major soccer competitions (e.g. RoboCup). However, no option combines a good repository of teams with a simple command set that abstracts low-level actions. To clarify this problem, we surveyed the most promising simulators and proposed an extension for the well-established Soccer Server. The objective was to simplify the process of learning strategy-related behaviors through automated optimization algorithms. The results have shown a clear advantage in using the extension to improve the agent's performance. This work contributes to the development of future strategies related with RoboCup or other soccer competitions. Despite the good results, there is space for improvement in computational efficiency and behavior diversity.