Machine Learning Classifications of Player Behavior

Abstract

Computer games often incorporate asymmetric game characters and classes, with the objective of providing a varied experience. Characters and classes are given different attributes, skills, and bonuses allowing players to employ different techniques and strategies. However, little research has been done on analyzing players’ behavior to determine if differences in characters and classes affect player behavior. In the field of game user research (GUR), the most common method of analysis is qualitative in nature. Previous quantitative work in the field focuses on drawing distinctions between the players themselves, but not distinctions about behavior caused by a game class independent of the player. The exploration of game class attributes and their effectiveness in directing player behavior is vitally important information for developers. Many games are regularly “rebalanced”, which is the practice of adjusting the capabilities and attributes of different classes to achieve a desired impact on gameplay, whether it be increased fairness or a more diverse gameplay experience. We strive to address the question of whether the varied attributes assigned to different gameplay classes do in fact elicit different in-game behavior depending on the class being used. Conclusions in this realm are often anecdotal or qualitative, with developers relying on methods like playtesting, heuristic evaluation, and think-aloud to guide their decisions concerning game updates and rebalancing. Even when analytics are used to inform rebalancing, it is typically limited to data visualization techniques to gain a better general understanding of gameplay. In this paper, we use quantitative methods over a sample of gameplay data to determine if the varied attributes that developers assign to different classes truly result in measurable and persistent behavioral differences. If we can accurately and reliably predict the gameplay class that a player is piloting based solely on their in-game behavior, then we know that the varied attributes assigned to the different classes do in fact have an impact on how players behave. The free-to-play World World II naval combat game World of Warships serves as the basis for this research. World of Warships is a massively multiplayer online game wherein each player controls a single ship, with each battle consisting of 12 ships on one team facing 12 ships of an opposing team. There are a variety of maps and gameplay objectives, but battles are largely focused on eliminating all enemy ships. World of Warship has four classes of ships: Destroyers, Cruisers, Battleships, and Aircraft Carriers. Each of these classes has significantly different capabilities in speed, armaments, armor, and more. Our hypothesis is that different classes of ships will encourage players to use different techniques and strategies and that we can identify the class of ship based solely on the player’s behavior. We use supervised machine learning techniques to automate analysis of player behavior and detect the ship class that the player is piloting. Our work will help with rebalancing the game during updates, providing information about how players change their behavior depending on the attributes assigned to each gameplay class. We employ a variety of machine learning and statistical analysis methods, using gameplay data from hundreds of online battles and thousands of players. This data serves as a quantification of in-game player behavior, reflecting the strategies and tactics that each player exhibited. A random forest model proved to be the most effective in classifying ship class based on player behavior, resulting in a training accuracy of 97% and a testing accuracy of 83%. The next section provides a review of current work in automated behavior analysis. We follow our review of previous work with a detailed discussion of our approach and results. Lastly, we provide a brief conclusion and proposal for future work that will build on the results.