A Statistical Analysis of Amateur Go players to Assist AI-cheating Detection

Abstract

Since the democratization of powerful AI engines for the game of Go, it is not uncommon to see a drastic level increase of some players that must be explained with the help of AI. This is considered cheating and forbidden by most organizations. When looking at online beginners and stronger amateur players, we discovered that they can display playing strength below professional level and still confidently win the game, as opposed to professional players. This makes using only AI-likeness metrics not sufficient to detect such players. We propose a method based on the analysis of a player’s performance considering point loss distribution over several games, taking into account only relevant moves of a game. We still use an AI-likeness metric for analyzing individual games where the use of AI may not be consistent. We evaluated our methods on two European go official online leagues, where cheating detection was already performed (for a total of about 150 unique regular players, with levels ranging from 20 kyu to 5 European dan). We show that our system confirmed 5 cases of players previously banned for cheating (out of 6). Our methods do not set out to categorize players between “cheaters” and “not cheaters,” but rather rank them in order of suspicion, for the sake of assisting referees and providing them a way to effectively investigate suspicious players over time.