Double Q–learning Agent for Othello Board Game

Abstract

This paper presents the first application of the Double Q–learning algorithm to the game of Othello. Reinforcement learning has previously been successfully applied to Othello using the canonical reinforcement learning algorithms, Q–learning and TD–learning. However, the algorithms suffer from considerable drawbacks. Q–learning frequently tends to be overoptimistic during evaluation, while TD–learning can get stuck in local minima. To overcome the disadvantages of the existing work, we propose using a Double Q–learning agent to play Othello and prove that it performs better than the existing learning agents. In addition to developing and implementing the Double Q–learning agent, we implement the Q–learning and TD– learning agents. The agents are trained and tested against two fixed opponents: a random player and a heuristic player. The performance of the Double Q–learning agent is compared with performance of the existing learning agents. The Double Q– learning agent outperforms them, although it takes longer, on average, to make each move. Further, we show that the Double Q–learning agent performs at its best with two hidden layers using the tanh function.