3D Simulation League Research Articles

Team Modeling with Deep Behavioral Cloning for the RoboCup 2D Soccer Simulation League

Soccer is still considered an open problem by the AI community due to its complex stochastic real-time multiagent nature. The RoboCup Soccer Simulation 2D League has been used as a testbed for new ideas and techniques for many subjects, including team modeling. However, even though team modeling has been an indispensable part of the best league participants to date, in practice it typically consists of ad-hoc heuristics encoded as rules. This requires time-consuming manual work, does not scale to multiple teams, and does not work well at unaccounted scenarios. This paper presents a data-driven method for modeling teams by training Deep Neural Networks with large amounts of data with an Imitation Learning formulation. We demonstrate the approach by training a deep model of the Japanese team Helios using 57,578,668 state-action pairs of players. The resulting model achieved 84.5% accuracy on action selection and a small error on regression of the action parameters. The network is shown to be an effective movement predictor of Helios field players and have negligible degradation when Helios is evaluated against adversaries not seen at training time.

Evaluation-function modeling with multi-layered perceptron for RoboCup soccer 2D simulation

In the RoboCup soccer simulation 2D league, players make a decision at each cycle in real time. The performance of a team highly depends on the agents’ decision-making process, which is composed of a action planning method and an evaluation function of the soccer field. In this work, a cooperative action planning based on the tree search is employed. Each action is evaluated by an evaluation function. We employ a multi-layered perceptron to construct an evaluation function. We examine the performance of the soccer agents when various sets of features are used as the input of the neural network. A feature vector is made of kick sequences executed by an expert team extracted from log files. To investigate the efficiency of our approach, we compare the performance of a team using an evaluation function modeled by neural networks against a team using a hand-tuned evaluation function.

Quantifying the impact of communication on performance in multi-agent teams

In this work, we relate the extent and quality of inter-agent communication and the overall performance in teams of multiple agents. Specifically, we examine the RoboCup Soccer Simulation 2D League, and carry out multiple simulation experiments against two evenly matched teams. For each simulated run (a 2D soccer simulation game), we generate the communication efficiencies (i.e., communications sent/communications received) for each agent pair. Applying linear regression and principal component analyses, we then correlate these efficiencies with measures of performance (i.e., goals scored and goals conceded), enabling the construction of inter-agent communication networks. Analysis of these networks highlights the microscopic player-to-player and macroscopic role-to-role communications correlated with performance. The approach determines the salient pathways within inter-agent communications which globally affect the coordination and the overall performance in multi-agent teams.

Multimodal sensory fusion for soccer robot self-localization based on long short-term memory recurrent neural network

Self-localization is a fundamental requirement for autonomous mobile robots. With the rapid development in sensor technology, the sensor suites of robot provide multimodal information that naturally ensures perception robustness, multimodal sensory fusion are able to provide a better solution for enhance the capability of self-localization. This paper proposes a multimodal sensory fusion method based on Long Short-Term Memory (LSTM) Recurrent Neural Network (RNN) for RoboCup 3D Simulation league. This approach fuses Inertia Navigation System (INS) and vision perceptor information from different sensors at feature level instead of raw data. The experiment results demonstrate that the proposed approach makes an improvement in predictive accuracy and efficiency compared with the standard Extended Kalman Filter (EKF) and the static Particle Filter (PF) methods.

Optimizing player’s formations for corner-kick situations in RoboCup soccer 2D simulation

In the domain of the Soccer simulation 2D league of the RoboCup, appropriate player positioning against the opponent team formation is an important factor of soccer team performance. In this work, we propose to use a meta-heuristic algorithm called the firefly algorithm to optimize player positioning. We used sequential Bayesian estimation as well as parallelization to reduce the necessary number of time-consuming simulated soccer matches. As a first trial of our system, we optimized the corner-kick formation. Preliminary results in optimizing the corner-kick formation are not advantageous over the previous handmade formation due to the difficulty in tuning the meta-heuristic algorithm parameters. However, it is also shown that the proposed system is effective in handling a high load of simulations over the span of weeks and therefore is promising to be usable to optimize player positioning.

Quantifying Long-Range Interactions and Coherent Structure in Multi-Agent Dynamics.

We develop and apply several novel methods quantifying dynamic multi-agent team interactions. These interactions are detected information-theoretically and captured in two ways: via (i) directed networks (interaction diagrams) representing significant coupled dynamics between pairs of agents, and (ii) state-space plots (coherence diagrams) showing coherent structures in Shannon information dynamics. This model-free analysis relates, on the one hand, the information transfer to responsiveness of the agents and the team, and, on the other hand, the information storage within the team to the team's rigidity and lack of tactical flexibility. The resultant interaction and coherence diagrams reveal implicit interactions, across teams, that may be spatially long-range. The analysis was verified with a statistically significant number of experiments (using simulated football games, produced during RoboCup 2D Simulation League matches), identifying the zones of the most intense competition, the extent and types of interactions, and the correlation between the strength of specific interactions and the results of the matches.

SCRAM: Scalable Collision-avoiding Role Assignment with Minimal-Makespan for Formational Positioning

Teams of mobile robots often need to divide up subtasks efficiently. In spatial domains, a key criterion for doing so may depend on distances between robots and the subtasks' locations. This paper considers a specific such criterion, namely how to assign interchangeable robots, represented as point masses, to a set of target goal locations within an open two dimensional space such that the makespan (time for all robots to reach their target locations) is minimized while also preventing collisions among robots. We present scaleable (computable in polynomial time) role assignment algorithms that we classify as being SCRAM (Scalable Collision-avoiding Role Assignment with Minimal-makespan). SCRAM role assignment algorithms use a graph theoretic approach to map agents to target goal locations such that our objectives for both minimizing the makespan and avoiding agent collisions are met. A system using SCRAM role assignment was originally designed to allow for decentralized coordination among physically realistic simulated humanoid soccer playing robots in the partially observable, non-deterministic, noisy, dynamic, and limited communication setting of the RoboCup 3D simulation league. In its current form, SCRAM role assignment generalizes well to many realistic and real-world multiagent systems, and scales to thousands of agents.

RoboCupサッカーシミュレーションリーグ2Dにおける リターンパス：chain action上の状態予測と局面評価関数

RoboCupサッカーシミュレーションリーグ2Dにおける リターンパス：chain action上の状態予測と局面評価関数

Using model-based collaborative filtering techniques to recommend the expected best strategy to defeat a simulated soccer opponent

How to improve the performance of a simulated soccer team using final game statistics? This is the question this research aims to answer using model-based collaborative techniques and a robotic team – FC Portugal – as a case study. After developing a framework capable of automatically calculating the final game statistics through the RoboCup log files, a feature selection algorithm was used to select the variables that most influence the final game result. In the next stage, given the statistics of the current game, we rank the strategies that obtained the maximum average of goal difference in similar past games. This is done by splitting offline past games into different k-clusters. Then, for each cluster, the expected best strategy was assigned. The online phase consists in the selection of the expected best strategy for the cluster in which the current game best fits. Regarding the final results, our approach proved that it is possible to improve the performance of a robotic team by more than 35%, even in a competitive environment such as the RoboCup 2D simulation league.

Improving a simulated soccer team's performance through a Memory-Based Collaborative Filtering approach

Collaborative filtering techniques have been used for some years, almost exclusively in Internet environments, helping users find items they are expected to like by using the user's past purchases to provide such recommendations. With this concept in mind, this research uses a collaborative filtering technique to automatically improve the performance of a simulated soccer team. Many studies have attempted to address this problem over the last years but none has shown meaningful improvements in the performance of the soccer team. Using a collaborative filtering technique based on nearest neighbors and the FC Portugal team as the test subject (in the context of the RoboCup 2D Simulation League), several simulations were run for matches against different teams with much better, better and worse performance than FC Portugal. The strategy used by FC Portugal was to combine 8 set-plays and 2 team formations. The simulation results revealed an improvement in performance between 32% and 384%. In the future, there are plans to expand this approach to other contexts, such as the 3D Simulation League.