Fair Server Selection in Edge Computing With $Q$-Value-Normalized Action-Suppressed Quadruple Q-Learning

Abstract

Edge computing is a promising paradigm that brings servers closer to users, leading to lower latencies and enabling latency-sensitive applications such as cloud gaming, virtual/augmented reality, telepresence, and telecollaboration. Due to the high number of possible edge servers and incoming user requests, the optimum choice of user-server matching has become a difficult challenge, especially in the 5G era where the network can offer very low latencies. In this article, we introduce the problem of fair server selection as not only complying with an application’s latency threshold but also reducing the variance of the latency among users in the same session. Due to the dynamic and rapidly evolving nature of such an environment and the capacity limitation of the servers, we propose as solution a reinforcement learning (RL) method in the form of a quadruple Q-Learning model with action suppression, <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$Q$</tex-math></inline-formula> -value normalization, and a reward function that minimizes the variance of the latency. Our evaluations in the context of a cloud gaming application show that, compared to an existing methods, our proposed method not only better meets the application’s latency threshold but is also more fair with a reduction of up to 35% in the standard deviation of the latencies experienced by users.