Playing real-time strategy games by imitating human players’ micromanagement skills based on spatial analysis

Abstract

Unlike the situation with board games, artificial intelligence (AI) for real-time strategy (RTS) games usually suffers from numerous possible future outcomes because the state of the game is continuously changing in real time. Furthermore, AI is also required to be able to handle the increased complexity within a small amount of time. This constraint makes it difficult to build AI for RTS games with current state-of-the art intelligence techniques. A human player, on the other hand, is proficient in dealing with this level of complexity, making him a better game player than AI bots. Human players are especially good at controlling many units at the same time. It is hard to explain the micro-level control skills needed with only a few rules programmed into the bots. The design of micromanagement skills is one of the most difficult parts in the StarCraft AI design because it must be able to handle different combinations of units, army size, and unit placement. The unit control skills can have a big effect on the final outcome of a full game in professional player matches. For StarCraft AI competitions, they employed a relatively simple scripted AI to implement the unit control strategy. However, it is difficult to generate cooperative behavior using the simple AI strategies. Although there has been a few research done on micromanagement skills, it is still a challenging problem to design human-like high-level control skills. In this paper, we proposed the use of imitation learning based on human replays and influence map representation. In this approach, we extracted huge numbers of cases from the replays of experts and used them to determine the actions of units in the current game case. This was done without using any hand-coded rules. Because this approach is data-driven, it was essential to minimize the case search times. To support fast and accurate matching, we chose to use influence maps and data hashing. They allowed the imitation system to respond within a small amount time (one frame, 0.042s). With a very large number of cases (up to 500,000 cases), we showed that it is possible to respond competitively in real-time, with a high winning percentage in micromanagement scenarios.