Joint QoS Control and Bitrate Selection for Video Streaming based on Multi-agent Reinforcement Learning

Abstract

Limited network resource and explosive growth in video services have made the networks resource scheduling and adaptive video streaming control even more important for the multi-users quality of experience (QoE) within the cell. The adaptive streaming technology enables the media player to dynamically select the most suitable encoding bitrate to satisfy the users QoE. To achieve the highest possible quality of service (QoS), network nodes QoS control, aka downstream bandwidth configuration, can be dynamically optimized to improve the utilization of bandwidth resources. In this paper, we propose a joint QoS control and adaptive bitrate (ABR) algorithm based on multi-agent reinforcement learning with asynchronous advantage actor-critic (MARL-A3C). The proposed method can process video streams from multiple user nodes simultaneously and adapt to different indicators of QoS and QoE. Under the predetermined QoE standard, the algorithm dynamically controls the maximum downlink rate for the video streaming delivery links, and dynamically selects the encoding bitrate of the video in an active manner for multiple users. The MARL algorithm learns the reward value function of each user state through continuous interaction with the environment and then learns the optimal strategy through these reward value functions. We evaluate the proposed MARL-A3C algorithm on a simulation platform of a fog radio access point (F-RAN) system and compare it with the state-of-art ABR algorithms. The experiment results show that the MARL-A3C outperforms the existing methods in the multi-user F-RAN scenario.