Deep Reinforcement Learning Model Design and Transmission for Network Delay Compensation in 3D Online Shooting Game

Abstract

A long network delay leads to decreased player performance in online games and an unfair game experience between the players. In this study, we propose a novel delay compensation method to reduce the impact of network delay on player performance in online 3D shooting games. The key contributions of the proposed scheme are two-fold. The first contribution utilizes two deep reinforcement learning (DRL) networks, timing and direction DRL, to predict each player's sudden movement under the network delay. Specifically, the timing DRL outputs when the player will change the movement direction while the direction DRL outputs the indenting direction of the player. The second contribution of this study is to send each player's trained model to other players at the beginning of the game match. This research is the first to discuss how to transfer the trained model, as well as the effect of the model transmissions on the delay compensation performance in the band-limited paths. We evaluated the effectiveness of the proposed delay compensation using an online 3D shooting game implemented by Unity 3D. The obtained evaluation results indicate that the proposed delay compensation accurately reproduces the player's position, even under long network delays. In addition, the transmitted model can predict the player's position when the available rate for the model transmission is approximately 60 KB.