The Fittest Wins: A Multistage Framework Achieving New SOTA in ViZDoom Competition

Abstract

This paper offers an integrated solution for First-Person Shooter (FPS) games to train agents with adaptive strategies. Solving such complex decision tasks requires generalization ability and adaptive strategies. We develop a framework using a novel Adaptive Strategic Control (ASC) algorithm combined with advanced techniques like the hindsight experience replay (HER), multi-agent reinforcement learning (MARL), and league training. The approach adopts a multi-stage learning scheme, consisting of learning a goal-conditioned navigation policy, then transferring to learn sophisticated shooting skills by playing against a league of players, and finally learning adaptive strategies. Our agent achieves the SOTA result in past ViZDoom AI Competitions, surpassing previous top-ranked agents (never seen during training) by a large margin. We provide comprehensive analysis and experiments to elaborate the effect of each component in affecting the agent performance and demonstrate that the proposed and adopted techniques are essential to achieve superior performance in ViZDoom Competition and potentially valuable for general end-to-end FPS games.