Real-time Strategy Game StarCraft Research Articles

Solving large-scale multi-agent tasks via transfer learning with dynamic state representation

Many research results have emerged in the past decade regarding multi-agent reinforcement learning. These include the successful application of asynchronous advantage actor-critic, double deep Q-network and other algorithms in multi-agent environments, and the more representative multi-agent training method based on the classical centralized training distributed execution algorithm QMIX. However, in a large-scale multi-agent environment, training becomes a major challenge due to the exponential growth of the state-action space. In this article, we design a training scheme from small-scale multi-agent training to large-scale multi-agent training. We use the transfer learning method to enable the training of large-scale agents to use the knowledge accumulated by training small-scale agents. We achieve policy transfer between tasks with different numbers of agents by designing a new dynamic state representation network, which uses a self-attention mechanism to capture and represent the local observations of agents. The dynamic state representation network makes it possible to expand the policy model from a few agents (4 agents, 10 agents) task to large-scale agents (16 agents, 50 agents) task. Furthermore, we conducted experiments in the famous real-time strategy game Starcraft II and the multi-agent research platform MAgent. And also set unmanned aerial vehicles trajectory planning simulations. Experimental results show that our approach not only reduces the time consumption of a large number of agent training tasks but also improves the final training performance.

Learning from Demonstration for Goal-Driven Autonomy

Goal-driven autonomy (GDA) is a conceptual model for creating an autonomous agent that monitors a set of expectations during plan execution, detects when discrepancies occur, builds explanations for the cause of failures, and formulates new goals to pursue when planning failures arise. While this framework enables the development of agents that can operate in complex and dynamic environments, implementing the logic for each of the subtasks in the model requires substantial domain engineering. We present a method using case-based reasoning and intent recognition in order to build GDA agents that learn from demonstrations. Our approach reduces the amount of domain engineering necessary to implement GDA agents and learns expectations, explanations, and goals from expert demonstrations. We have applied this approach to build an agent for the real-time strategy game StarCraft. Our results show that integrating the GDA conceptual model into the agent greatly improves its win rate.

Kiting in RTS Games Using Influence Maps

Influence Maps have been successfully used in controlling the navigation of multiple units. In this paper, we apply the idea to the problem of simulating a kiting behavior (also known as ¨attack and flee'¨) in the context of real-time strategy (RTS) games. We present our approach and evaluate it in the popular RTS game StarCraft, where we analyze the benefits that our approach brings to a StarCraft playing bot.

AIIDE 2012 StarCraft Competition

In 2012, AIIDE will host the Third Annual StarCraft AI Competition. Participants are given the task of building the best performing AI system for the popu- lar real-time strategy game StarCraft Brood War, which was released by Blizzard Entertainment in 1998. The goals of the competition are to provide a testbed for real-time AI systems and to promote game AI research by exhibiting AI techniques such as scripting, planning, optimization, spatial reasoning, and opponent modeling in a fast-paced popular video game.

AIIDE 2013 StarCraft Competition

In 2013, AIIDE will host the Fourth Annual Star-Craft AI Competition. Participants are given the task of building the best performing AI system for the popular real-time strategy game StarCraft Brood War (Blizzard Entertainment). Thee goals of the competition are to provide a testbed for real-time AI systems and to promote game AI research by exhibiting AI techniques such as scripting, planning, optimization, spatial reasoning, and opponent modeling in a fast-paced popular video game.

Learning Micro-Management Skills in RTS Games by Imitating Experts

We investigate the problem of learning the control of small groups of units in combat situations in Real Time Strategy (RTS) games. AI systems may acquire such skills by observing and learning from expert players, or other AI systems performing those tasks. However, access to training data may be limited, and representations based on metric information -- position, velocity, orientation etc. -- may be brittle, difficult for learning mechanisms to work with, and generalise poorly to new situations. In this work we apply \textit{qualitative spatial relations} to compress such continuous, metric state-spaces into symbolic states, and show that this makes the learning problem easier, and allows for more general models of behaviour. Models learnt from this representation are used to control situated agents, and imitate the observed behaviour of both synthetic (pre-programmed) agents, as well as the behaviour of human-controlled agents on a number of canonical micro-management tasks. We show how a Monte-Carlo method can be used to decompress qualitative data back in to quantitative data for practical use in our control system. We present our work applied to the popular RTS game Starcraft.

Walling in Strategy Games via Constraint Optimization

This paper presents a constraint optimization approach to walling in real-time strategy (RTS) games. Walling is a specific type of spatial reasoning, typically employed by human expert players and not currently fully exploited in RTS game AI, consisting on finding configurations of buildings to completely or partially block paths. Our approach is based on local search, and is specifically designed for the real-time nature of RTS games. We present experiments in the context of the RTS game StarCraft showing promising results.

AIIDE 2014 StarCraft Competition

In 2014, AIIDE will host the Fifth Annual StarCraft AI Competition. Participants are given the task of building the best performing AI system for the popular real-time strategy game StarCraft Brood War (Blizzard Entertainment). The goals of the competition are to provide a testbed for real-time AI systems and to promote game AI research by ex- hibiting AI techniques such as scripting, planning, optimization, spatial reasoning, and opponent modeling in a fast-paced popular video game.

Rock, Paper, StarCraft: Strategy Selection in Real-Time Strategy Games

The correct choice of strategy is crucial for a successful real-time strategy (RTS) game player. Generally speaking, a strategy determines the sequence of actions the player will take in order to defeat his/her opponents. In this paper we present a systematic study of strategy selection in the popular RTS game StarCraft. We treat the choice of strategy as a game itself and test several strategy selection techniques, including Nash Equilibrium and safe opponent exploitation. We adopt a subset of AIIDE 2015 StarCraft AI tournament bots as the available strategies and our results suggest that it is useful to deviate from Nash Equilibrium to exploit sub-optimal opponents on strategy selection, confirming insights from computer rock-paper-scissors tournaments.

A Benchmark for StarCraft Intelligent Agents

The problem of comparing the performance of different Real-Time Strategy (RTS) Intelligent Agents (IA) is non-trivial. And often different research groups employ different testing methodologies designed to test specific aspects of the agents. However, the lack of a standard process to evaluate and compare different methods in the same context makes progress assessment difficult. In order to address this problem, this paper presents a set of benchmark scenarios and metrics aimed at evaluating the performance of different techniques or agents for the RTS game StarCraft. We used these scenarios to compare the performance of a collection of bots participating in recent StarCraft AI (Artificial Intelligence) competitions to illustrate the usefulness of our proposed benchmarks.

The Development Tendency of Artificial Intelligence in Command and Control: A Brief Survey

Google’s DeepMind’s AlphaStar has achieved great success in the real-time strategy game StarCraft 2, and it has also made the combination of artificial intelligence and command and control, attracted widespread attention. Artificial intelligence theory and its engineering practice accumulated over many years have injected new development momentum into traditional fields, such as command and control. We download more than 900 high-level papers from Web of Science (WoS) that combine artificial intelligence and command and control as the research object, and employ bibliometric, CiteSpace, VOSviewer, and other bibliometric analysis software. From the number of documents, co-citing, co-occurrence of keywords and research hotspots, the four aspects are studied and visualized analysis, which clarifies the research status and key documents of artificial intelligence in command and control, and reveals the frontier hotspots and research directions in this field. Finally, a summary of the four visual analysis aspects points out important references for choosing research directions, exploring the frontiers of disciplines, and assisting scientific decision-making in this field.

A strategic decision-making architecture toward hybrid teams for dynamic competitive problems

Advances in artificial intelligence create new opportunities for computers to support humans as peers in hybrid teams in several complex problem-solving situations. This paper proposes a decision-making architecture for adaptively informing decisions in human-computer collaboration for large-scale competitive problems under dynamic environments. The proposed architecture integrates methods from sequence learning, model predictive control, and game theory. Computers in this architecture learn objectives and strategies from experimental data to support humans with strategic decisions while operational decisions are made by humans. The paper also presents data-driven methods for partitioning tasks among a team of computers in this architecture. The generalized methodology is illustrated on the real-time strategy game Starcraft II. The results from this application show that low-performing players can benefit from the game-theoretic decision support whereas this support can be overly conservative for high-performing players. The proposed approach provides safe though suboptimal suggestions particularly against an opponent with an unknown level of expertise. The results further show that problem solution with a team of computers based on non-intuitive task partitioning significantly improves the quality of decisions compared to an all-in-one solution with a single computer.

Lenticular nucleus volume predicts performance in real-time strategy game: cross-sectional and training approach using voxel-based morphometry.

It is unclear why some people learn faster than others. We performed two independent studies in which we investigated the neural basis of real‐time strategy (RTS) gaming and neural predictors of RTS game skill acquisition. In the first (cross‐sectional) study, we found that experts in the RTS game StarCraft® II (SC2) had a larger lenticular nucleus volume (LNV) than non‐RTS players. We followed a cross‐validation procedure where we used the volume of regions identified in the first study to predict the quality of learning a new, complex skill (SC2) in a sample of individuals who were naive to RTS games (a second (training) study). Our findings provide new insights into how the LNV, which is associated with motor as well as cognitive functions, can be utilized to predict successful skill learning and be applied to a much broader context than just video games, such as contributing to optimizing cognitive training interventions.

Algorithm Selection in Adversarial Settings: From Experiments to Tournaments in StarCraft

Algorithm selection—the mapping of problem instances to algorithms—has been successfully applied to a variety of complex theoretical and practical problems, including computer games. In this paper, we extend the traditional framework, which considers a single decision maker, to adversarial settings, by modeling algorithm selection as a normal-form game. In this “game of algorithm selection,” agents select algorithms to play a computer game on their behalf. The game's payoff matrix stores the relative performance among algorithms. We also consider nonstationary scenarios, where algorithms can learn from previous matches. We apply this approach to real-time strategy game StarCraft , using bots developed for the game as our algorithms. Our experiments suggest that minimax-Q is a suitable method for algorithm selection in both stationary and nonstationary conditions. We proceed by implementing our approach in MegaBot, a fully capable StarCraft bot. MegaBot showed robustness to nonstationarity, competing in the difficult scenario of StarCraft AI tournaments. MegaBot successfully learns how to select algorithms, exhibiting increasing win rates with tournament progress. In 2016, its debut year, it left 60% of opponents behind in two out of three tournaments. In 2017, however, MegaBot faced difficulties as its algorithm portfolio got outdated compared to newer entries.

<sc>ghost</sc>: A Combinatorial Optimization Framework for Real-Time Problems

This paper presents GHOST , a combinatorial optimization framework that a real-time strategy (RTS) AI developer can use to model and solve any problem encoded as a constraint satisfaction/optimization problem (CSP/COP). We show a way to model three different problems as a CSP/COP, using instances from the RTS game StarCraft as test beds. Each problem belongs to a specific level of abstraction (the target selection as reactive control problem, the wall-in as a tactics problem, and the build order planning as a strategy problem). In our experiments, GHOST shows good results computed within some tens of milliseconds. We also show that GHOST outperforms state-of-the-art constraint solvers, matching them on the resources allocation problem, a common combinatorial optimization problem.

Evolving Effective Microbehaviors in Real-Time Strategy Games

We investigate heuristic search algorithms to generate high-quality micromanagement in combat scenarios for real-time strategy (RTS) games. Macro- and micromanagement are two key aspects of RTS games. While good macro helps a player collect more resources and build more units, good micro helps a player win skirmishes and battles against equal numbers and types of opponent units or win even when outnumbered. In this paper, we use influence maps and potential fields as a basis representation to evolve short-term positioning and movement tactics. Unit microbehaviors in combat are compactly encoded into 14 parameters. A genetic algorithm evolves good microbehaviors by manipulating these 14 parameters. We compared the performance of our evolved ECSLBot with two other state-of-the-art bots, UAlbertaBot and Nova , on several skirmish scenarios in a popular RTS game StarCraft . The results show that the ECSLBot tuned by genetic algorithms outperforms UAlbertaBot and Nova in kiting efficiency, target selection, and fleeing. Further experiments show that the parameter values evolved in one scenario work well in other scenarios and that we can switch between preevolved parameter sets to perform well in unseen scenarios containing more than one type of opponent unit. We believe our representation and approach applied to each unit type of interest can result in effective microperformance against melee and ranged opponents and provides a viable approach toward complete RTS bots.

A Multi-objective Genetic Algorithm for Build Order Optimization in StarCraft II

This article presents a modified version of the multi-objective genetic algorithm NSGA II in order to find optimal opening strategies in the real-time strategy game StarCraft II. Based on an event-driven simulator capable of performing an accurate estimate of in-game build-times the quality of different build lists can be judged. These build lists are used as chromosomes within the genetic algorithm. Procedural constraints e.g. given by the Tech-Tree or other game mechanisms, are implicitly encoded into them. Typical goals are to find the build list producing most units of one or more certain types up to a certain time (Rush) or to produce one unit as early as possible (Tech-Push). Here, the number of entries in a build list varies and the objective values have in contrast to the search space a very small diversity. We introduce our game simulator including its graphical user interface, the modifications necessary to fit the genetic algorithm to our problem, test our algorithm on different Tech-Pushes and Rushes for all three races, and validate it with empirical data of expert StarCraft II players.

Controllable procedural map generation via multiobjective evolution

This paper shows how multiobjective evolutionary algorithms can be used to procedurally generate complete and playable maps for real-time strategy (RTS) games. We devise heuristic objective functions that measure properties of maps that impact important aspects of gameplay experience. To show the generality of our approach, we design two different evolvable map representations, one for an imaginary generic strategy game based on heightmaps, and one for the classic RTS game StarCraft. The effect of combining tuples or triples of the objective functions are investigated in systematic experiments, in particular which of the objectives are partially conflicting. A selection of generated maps are visually evaluated by a population of skilled StarCraft players, confirming that most of our objectives correspond to perceived gameplay qualities. Our method could be used to completely automate in-game controlled map generation, enabling player-adaptive games, or as a design support tool for human designers.