Drafting in Collectible Card Games via Reinforcement Learning

Abstract

Collectible card games (CCGs), such as Magic: the Gathering and Hearthstone, are played by tens of millions of players worldwide and are challenging for humans and artificial intelligence (AI) agents alike. To win, players must be proficient in two interdependent tasks: deck-building and battling. We present three deep reinforcement learning approaches for deck-building in the arena mode that differ in considering past information when making new choices. We formulate the problem in a game-agnostic manner and perform experiments on Legends of Code and Magic, a CCG designed for AI research. Results show that our trained draft agents outperform the currently best draft agents of the game and do so by building very different decks. Moreover, a Strategy Card Game AI competition participant improves from tenth to fourth place when using our best draft agent to build decks. This work is a step towards strong and fast game-playing AI for CCGs, one of the current academic AI milestones that would enable thorough playtesting of new cards before they are released – a long-standing problem in the CCG industry.