Self-supervised Contrastive Learning for Predicting Game Strategies

Abstract

Many games enjoyed by players primarily consist of a matching system that allows the player to cooperate or compete with other players with similar scores. However, the method of matching only the play score can easily lose interest because it does not consider the opponent’s playstyle or strategy. In this study, we propose a self-supervised contrastive learning framework that can enhance the understanding of game replay data to create a more sophisticated matching system. We use actor-critic-based reinforcement learning agents to collect many replay data. We define a positive pair and negative examples to perform contrastive learning. Positive pair is defined by sampling from the frames of the same replay data, otherwise negatives. To evaluate the performance of the proposed framework, we use Facebook ELF, a real-time strategy game, to collect replay data and extract data features from pre-trained neural networks. Furthermore, we apply k-means clustering with the extracted features to visually demonstrate that different play patterns and proficiencies can be clustered appropriately. We present our clustering results on replay data and show that the proposed framework understands the nature of the data with consecutive frames.