Reinforcement Learning Based Resource Allocation Research Articles

Reinforcement Learning-Based Resource Allocation for Multiple Vehicles with Communication-Assisted Sensing Mechanism

Reinforcement Learning-Based Resource Allocation for Multiple Vehicles with Communication-Assisted Sensing Mechanism

Reinforcement Learning-Based Resource Allocation and Energy Efficiency Optimization for a Space–Air–Ground-Integrated Network

Reinforcement Learning-Based Resource Allocation and Energy Efficiency Optimization for a Space–Air–Ground-Integrated Network

Reinforcement learning-based resource allocation for dynamic aggregated WiFi/VLC HetNet

High transmission rates and low power consumption make visible light communication (VLC) a highly promising supplementary technology for the next generation of mobile communication. Taking into account the limited coverage area, VLC can be combined with WiFi as a heterogeneous network (HetNet), thanks to their non-overlapping spectrum. In order to fully utilize the advantages of the WiFi and VLC technologies, a new aggregated WiFi/VLC HetNet consisting of a single WiFi AP and multiple VLC APs, is designed, where the user equipment (UE), for the first time, can access multiple VLC APs and one WiFi AP simultaneously, and be allowed to acquire multiple resource blocks (RB) in each AP at the same time. To optimize the performance of the above designed HetNet, a multi-objective optimization problem (MOOP) is formulated, which aims to maximize system throughput while reducing the handover rate. Since the above MOOP is non-convex and nonlinear, the traditional resource allocation (RA) algorithm has a complex calculation process and poor timeliness performance to deal with this problem. To solve the above MOOP, a reinforcement learning (RL)-based RA algorithm is proposed, and considering the RB handover overhead in the aggregated WiFi/VLC HetNet, a reward function related to the system throughput and the UE handover rate is carefully designed. System throughput, system handover rate, user satisfaction, as well as user fairness performance are analyzed under three typical indoor illumination layouts. Finally, compared with the greedy algorithm and the hypergraph-based carrier aggregation algorithm, numerical results show that the proposed RL-based RA algorithm could improve the system throughput over the former two algorithms by 30.26% and 19.71%, while reducing the system handover rate by 0.15% and 0.02%, respectively.

Deep Reinforcement Learning-Based Resource Allocation for RSMA in LEO Satellite-Terrestrial Networks

Deep Reinforcement Learning-Based Resource Allocation for RSMA in LEO Satellite-Terrestrial Networks

Reinforcement Learning-Based Resource Allocation for Coverage Continuity in High Dynamic UAV Communication Networks

Reinforcement Learning-Based Resource Allocation for Coverage Continuity in High Dynamic UAV Communication Networks

Deep reinforcement learning based resource allocation in edge-cloud gaming

Deep reinforcement learning based resource allocation in edge-cloud gaming

Hierarchical Reinforcement Learning Based Resource Allocation for RAN Slicing

Hierarchical Reinforcement Learning Based Resource Allocation for RAN Slicing

Deep Reinforcement Learning Based Resource Allocation for Fault Detection with Cloud Edge Collaboration in Smart Grid

Deep Reinforcement Learning Based Resource Allocation for Fault Detection with Cloud Edge Collaboration in Smart Grid

Deep Reinforcement Learning-Based Resource Allocation for Multi-UAV-Assisted Full-Duplex Wireless-Powered IoT Networks

Deep Reinforcement Learning-Based Resource Allocation for Multi-UAV-Assisted Full-Duplex Wireless-Powered IoT Networks

Deep Reinforcement Learning Based Resource Allocation and Trajectory Planning in Integrated Sensing and Communications UAV Network

In this paper, multi-UAVs serve as mobile aerial ISAC platforms to sense and communicate with on-ground target users. To optimize the communication and sensing performance, we formulate a joint user association, UAV trajectory planning and power allocation problem to maximize the minimum weighted spectral efficiency among UAVs. This paper exploits the centralized and the decentralized deep reinforcement learning (DRL) solutions to solve the sequential decision-making problem. On one hand, we first introduce the centralized soft actor-critic (SAC) algorithm. Then, we explore the equivalent transformation of the optimization objective based on symmetric group, propose the random and the adaptive data augmentation schemes to design the replay memory buffer of SAC, and accordingly propose SAC algorithms assisted by data augmentation to tackle the transformed problem. On the other hand, the multi-agent soft actor-critic (MASAC), a decentralized solution, is also introduced to solve this sequential decision-making problem. The experiment results reveal the effectiveness of the centralized and the decentralized solutions in considered scenarios. Specifically, the SAC assisted by the adaptive scheme significantly outperforms other centralized solutions in the training speed and the weighted spectral efficiency. Meanwhile, the decentralized MASAC algorithm behaves best in the early training speed.

Federated Reinforcement Learning-Based Resource Allocation in D2D-Enabled 6G

The current 5G and conceived 6G era with ultrahigh density, ultra-high frequency bandwidth, and ultra-low latency can support emerging applications like Extended Reality (XR), Vehicle to Everything (V2X), and massive Internet of Things (IoT). With the rapid growth of transmission rate requirements and link numbers in the wireless communication network, how to allocate resources reasonably and further improve spectrum utilization challenges the traditional approaches. To address these problems, technologies such as device-to-device (D2D) communication and machine learning (ML) are introduced to the traditional cellular communication network to improve network performance. However, due to the interference caused by spectrum reusing, efficient resource allocation for both cellular users and D2D users is necessary. In this article, we consider underlay mode D2D-enabled wireless network to improve the spectrum utilization, and deep reinforcement learning (DRL)- based federated learning (FL)-aided decentralized resource allocation approach to maximize the sum capacity and minimize the overall power consumption while guaranteeing the quality of service (QoS) requirement of both cellular users and D2D users. The performance of the proposed schemes is evaluated through simulations under 5G millimeter-wave (mm-wave) and 6G terahertz (THz) scenarios separately. The simulation results show that the proposal achieves significant network performance compared with the baseline algorithms.

Multi-Agent Deep Reinforcement Learning Based Resource Allocation for Ultra-Reliable Low-Latency Internet of Controllable Things

As a promising technology in the 5G era, the artificial intelligence (AI) enabled Internet of controllable things (IoCT) is expected to be an integral part of heterogeneous networks (HetNets) in the future. However, the realization of ultra-reliable low-latency communications (URLLC) in IoCT communications underlaid HetNet has stringent quality of service (QoS) requirements, resulting in unprecedented challenges for existing wireless resource allocation methods. In this paper, we first describe a cellular HetNet model with uplink IoCT communications, then formulate a dynamic mixed-integer nonlinear programming (MINLP) resource allocation problem for maximizing the long-term average energy efficiency under URLLC requirements including reliability, latency, and transmission rate. To solve the problem, we propose a decentralized MADRL-based resource allocation algorithm with a decentralized partially observable Markov decision process (dec-POMDP) and a mixed-centralized-decentralized (MCD) framework to address the partial observability and the scalability issues, respectively. In addition, we design a reward function featuring the objective decomposition, baseline-guided scaling, and QoS violation penalty so that the agents are coordinated. Extensive experiments demonstrate the convergence, scalability, and robustness of the proposed algorithm. Besides, the proposed algorithm substantially outperforms conventional resource allocation methods and different agent communication mechanisms in terms of maximizing energy efficiency.

Collective reinforcement learning based resource allocation for digital twin service in 6G networks

The 6th generation (6G) mobile communications technology will realize the interconnection of humans, machines, things as well as virtual space. The development of digital twins (DTs) and 6G has accelerated the Internet of Things (IoT) in an unprecedented way. The combination of DTs and edge intelligence (EI) enables powerful digital space synchronized with the real world constructed in the intelligent edge, bringing real-time, and adaptive services delivery of IoT. However, the dynamic features and heterogeneous resources in 6G-enabled IoT make the resource allocation for computation-intensive and delay-sensitive DTs services more challenging. In this paper, we first define the DTs implementation process as a DT service function chain (DTSFC) and address the resource allocation problem of DTs-empowered networks in form of dynamic DTSFCs orchestration. We further propose a novel collective reinforcement learning (CRL) method which is inspired by human collaboration, to realize the effective resource allocation of DTSFCs. Numerical results verify that the proposed CRL algorithm improves the learning efficiency and generalization ability compared with the benchmarks.

Federated Reinforcement Learning-Based Resource Allocation for D2D-Aided Digital Twin Edge Networks in 6G Industrial IoT

The 6G is conceived to address the expected high level of requirements (such as ultra-high data transmission rate, support for the highest moving speed and seamless connection, etc.) in the next decade and beyond. In the context of 6G, a large number of Industrial Internet of Things (IIoT) devices may access the network, and thanks to the rapid development of artificial intelligence make smart manufacturing has the opportunity to be realized. However, a large number of IoT devices, the tremendous volume of data, the heterogeneous nature of devices, and the increasing concerns of privacy challenge the efficient management and quality of services in IIoT. To address these problems, in this article, a device-to-device (D2D) communication-aided digital twin edge network is proposed, where edge computing is introduced to bring computing and storage resources near to the end devices, and digital twin is utilized to fill the gap between physical and virtual space and D2D communication is applied to assist resource limited IoT devices to achieve normal communication. Moreover, digital twin-empowered federated reinforcement learning is leveraged to provide privacy awareness and decentralized resource allocation strategy training on D2D communication links to further improve network performance. The simulation results show that the proposal achieves significant network performance compared with baseline algorithms.

Resource Allocation With Workload-Time Windows for Cloud-Based Software Services: A Deep Reinforcement Learning Approach

As the workloads and service requests in cloud computing environments change constantly, cloud-based software services need to adaptively allocate resources for ensuring the Quality-of-Service (QoS) while reducing resource costs. However, it is very challenging to achieve adaptive resource allocation for cloud-based software services with complex and variable system states. Most of the existing methods only consider the current condition of workloads, and thus cannot well adapt to real-world cloud environments subject to fluctuating workloads. To address this challenge, we propose a novel Deep Reinforcement learning based resource Allocation method with workload-time Windows (DRAW) for cloud-based software services that considers both the current and future workloads in the resource allocation process. Specifically, an original Deep Q-Network (DQN) based prediction model of management operations is trained based on workload-time windows, which can be used to predict appropriate management operations under different system states. Next, a new feedback-control mechanism is designed to construct the objective resource allocation plan under the current system state through iterative execution of management operations. Extensive simulation results demonstrate that the prediction accuracy of management operations generated by the proposed DRAW method can reach 90.69%. Moreover, the DRAW can achieve the optimal/near-optimal performance and outperform other classic methods by 3~13% under different scenarios.

Deep Reinforcement Learning-Based Resource Allocation for UAV-Enabled Federated Edge Learning

Deep Reinforcement Learning-Based Resource Allocation for UAV-Enabled Federated Edge Learning

Energy-Efficient Resource Allocation Based on Deep Q-Network in V2V Communications.

Recently, with the development of autonomous driving technology, vehicle-to-everything (V2X) communication technology that provides a wireless connection between vehicles, pedestrians, and roadside base stations has gained significant attention. Vehicle-to-vehicle (V2V) communication should provide low-latency and highly reliable services through direct communication between vehicles, improving safety. In particular, as the number of vehicles increases, efficient radio resource management becomes more important. In this paper, we propose a deep reinforcement learning (DRL)-based decentralized resource allocation scheme in the V2X communication network in which the radio resources are shared between the V2V and vehicle-to-infrastructure (V2I) networks. Here, a deep Q-network (DQN) is utilized to find the resource blocks and transmit power of vehicles in the V2V network to maximize the sum rate of the V2I and V2V links while reducing the power consumption and latency of V2V links. The DQN also uses the channel state information, the signal-to-interference-plus-noise ratio (SINR) of V2I and V2V links, and the latency constraints of vehicles to find the optimal resource allocation scheme. The proposed DQN-based resource allocation scheme ensures energy-efficient transmissions that satisfy the latency constraints for V2V links while reducing the interference of the V2V network to the V2I network. We evaluate the performance of the proposed scheme in terms of the sum rate of the V2X network, the average power consumption of V2V links, and the average outage probability of V2V links using a case study in Manhattan with nine blocks of 3GPP TR 36.885. The simulation results show that the proposed scheme greatly reduces the transmit power of V2V links when compared to the conventional reinforcement learning-based resource allocation scheme without sacrificing the sum rate of the V2X network or the outage probability of V2V links.

Deep Reinforcement Learning Based Resource Allocation Approach for Wireless Networks Considering Network Slicing Paradigm

In this paper, we present an approach for resource scheduling in wireless networks based on the Network Slicing (NS) paradigm using Double Deep Q-Network (DDQN) Reinforcement Learning (RL) algorithm. More specifically, we propose a joint power and Scheduling Block (SB) allocation algorithm for networks with NS. The reinforcement learning algorithm applied to the resource allocation problem is formulated using state transitions regarding the system dynamics. We also present an algorithm, namely Network Slicing based on Reinforcement Learning (NSRL) that combines the proposed reinforcement learning based resource allocation with an approach based on reservation and sharing of resources among the slices where each RL agent acts in one slice. Simulations are carried out considering User Equipments (UEs) within a small cell coverage area - (Small Cells) with different Modulation and Coding Schemes (MCS) standardized by the 3rd Generation Partnership Project (3GPP) based on a simplified NS scenario with fifth generation wireless network (5G) characteristics. In the simulations, two slices are considered for the UEs: one considering Ultra-reliable and Low Latency Communications (URLLC) and other related to enhanced Mobile Broadband (eMBB) services. Simulation results show that the NSRL algorithm efficiently allocates power and SBs, outperforming other algorithms in the literature.

Deep Reinforcement Learning Based Resource Allocation for Network Slicing With Massive MIMO

Deep Reinforcement Learning Based Resource Allocation for Network Slicing With Massive MIMO

Deep Reinforcement Learning Based Resource Allocation in Multi-UAV-Aided MEC Networks

Resource allocation for mobile edge computing (MEC) in unmanned aerial vehicle (UAV) networks has been a popular research issue. Different from existing works, this paper considers a multi-UAV-aided uplink communication scenario and investigates a resource allocation problem of minimizing the total system latency and the energy consumption, subject to constraints on transmit power of mobile users (MUs), system latency caused by transmission and computation. The problem is confirmed to be a challenging time-series mixed-integer non-convex programming problem, and we propose a joint UAV Movement control, MU Association and MU Power control (UMAP) algorithm to solve it effectively, where three sub-problems are optimized iteratively. Specifically, UAV movement and MU association are optimized utilizing deep reinforcement learning (DRL) to decrease the energy consumption and system latency. Next, a closed-form solution of the MU transmit power is derived. Finally, simulation results show that the UMAP algorithm can significantly decrease the system latency and energy consumption and increase the coverage rate compared with benchmark algorithms.