Abstract
Very high throughput satellite (VHTS) systems are expected to have a huge increase in traffic demand in the near future. Nevertheless, this increase will not be uniform over the entire service area due to the non-uniform distribution of users and changes in traffic demand during the day. This problem is addressed by using flexible payload architectures, which allow the allocation of payload resources flexibly to meet the traffic demand of each beam, leading to dynamic resource management (DRM) approaches. However, DRM adds significant complexity to VHTS systems, so in this paper we discuss the use of one reinforcement learning (RL) algorithm and two deep reinforcement learning (DRL) algorithms to manage the resources available in flexible payload architectures for DRM. These algorithms are Q-Learning (QL), Deep Q-Learning (DQL) and Double Deep Q-Learning (DDQL) which are compared based on their performance, complexity and added latency. On the other hand, this work demonstrates the superiority a cooperative multiagent (CMA) decentralized distribution has over a single agent (SA).
Highlights
VERY high throughput satellite (VHTS) systems have a fundamental role in the support of future 5G and broadcast terrestrial networks (a), (b)
Liu et al (p) suggested an assignment gamebased dynamic power allocation (AG-DPA) to achieve a low suboptimal complexity in multibeam satellite systems. They compared the results obtained with a proportional power allocation (PPA) algorithm, obtaining a remarkable advantage in terms of power saving; resource management is still insufficient with respect to the required traffic demand since the error obtained between the capacity offered and the traffic demand in some cases is greater than 200 Mbps
The paper is organized as follows: Section II includes a background summary of reinforcement learning, Section III explains the full flexibility system model and problem definition, Section IV presents reformulating the problem as a cooperative multiagent (CMA) RL and CMA deep reinforcement learning (DRL) problem, Section V describes the proposed DRL algorithm using a CMA distribution, Section VI presents the simulation results and the analysis of the case study and, Section VII contains the conclusions of the study
Summary
VERY high throughput satellite (VHTS) systems have a fundamental role in the support of future 5G and broadcast terrestrial networks (a), (b). Liu et al (p) suggested an assignment gamebased dynamic power allocation (AG-DPA) to achieve a low suboptimal complexity in multibeam satellite systems They compared the results obtained with a proportional power allocation (PPA) algorithm, obtaining a remarkable advantage in terms of power saving; resource management is still insufficient with respect to the required traffic demand since the error obtained between the capacity offered and the traffic demand in some cases is greater than 200 Mbps. The main advantage of this methodology is that the management is done with a low computational cost since the neural network training is performed offline This methodology presents several challenges; one of them is the exponential dependence of the number of classes on the number of beams, in addition to the possible variations of power, bandwidth and/or beamwidth, which results in unsolvable problems due to the increase in flexibility. The paper is organized as follows: Section II includes a background summary of reinforcement learning, Section III explains the full flexibility system model and problem definition, Section IV presents reformulating the problem as a CMA RL and CMA DRL problem, Section V describes the proposed DRL algorithm using a CMA distribution, Section VI presents the simulation results and the analysis of the case study and, Section VII contains the conclusions of the study
Sign-in/Register to view highlights & summary 
Published Version (
Free)
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call