Learning Based Resource Allocation Scheme Research Articles

A Federated Learning-Based Resource Allocation Scheme for Relaying-Assisted Communications in Multicellular Next Generation Network Topologies

Growing and diverse user needs, along with the need for continuous access with minimal delay in densely populated machine-type networks, have led to a significant overhaul of modern mobile communication systems. Within this realm, the integration of advanced physical layer techniques such as relaying-assisted transmission in beyond fifth-generation (B5G) networks aims to not only enhance network performance but also extend coverage across multicellular orientations. However, in cellular environments, the increased interference levels and the complex channel representations introduce a notable rise in the computational complexity associated with radio resource management (RRM) tasks. Machine and deep learning (ML/DL) have been proposed as an efficient way to support the enhanced user demands in densely populated environments since ML/DL models can relax the traffic load that is associated with RRM tasks. There is, however, in these solutions the need for distributed execution of training tasks to accelerate the decision-making process in RRM tasks. For this purpose, federated learning (FL) schemes are considered a promising field of research for next-generation (NG) networks’ RRM. This paper proposes an FL approach to tackle the joint relay node (RN) selection and resource allocation problem subject to power management constraints when in B5G networks. The optimization objective of this approach is to jointly elevate energy (EE) and spectral efficiency (SE) levels. The performance of the proposed approach is evaluated for various relaying-assisted transmission topologies and through comparison with other state-of-the-art ones (both ML and non-ML). In particular, the total system energy efficiency (EE) and spectral efficiency (SE) can be improved by up to approximately 10–20% compared to a state-of-the-art centralized ML scheme. Moreover, achieved accuracy can be improved by up to 10% compared to state-of-the-art non-ML solutions, while training time is reduced by approximately 50%.

Energy-Efficient Resource Allocation Based on Deep Q-Network in V2V Communications.

Recently, with the development of autonomous driving technology, vehicle-to-everything (V2X) communication technology that provides a wireless connection between vehicles, pedestrians, and roadside base stations has gained significant attention. Vehicle-to-vehicle (V2V) communication should provide low-latency and highly reliable services through direct communication between vehicles, improving safety. In particular, as the number of vehicles increases, efficient radio resource management becomes more important. In this paper, we propose a deep reinforcement learning (DRL)-based decentralized resource allocation scheme in the V2X communication network in which the radio resources are shared between the V2V and vehicle-to-infrastructure (V2I) networks. Here, a deep Q-network (DQN) is utilized to find the resource blocks and transmit power of vehicles in the V2V network to maximize the sum rate of the V2I and V2V links while reducing the power consumption and latency of V2V links. The DQN also uses the channel state information, the signal-to-interference-plus-noise ratio (SINR) of V2I and V2V links, and the latency constraints of vehicles to find the optimal resource allocation scheme. The proposed DQN-based resource allocation scheme ensures energy-efficient transmissions that satisfy the latency constraints for V2V links while reducing the interference of the V2V network to the V2I network. We evaluate the performance of the proposed scheme in terms of the sum rate of the V2X network, the average power consumption of V2V links, and the average outage probability of V2V links using a case study in Manhattan with nine blocks of 3GPP TR 36.885. The simulation results show that the proposed scheme greatly reduces the transmit power of V2V links when compared to the conventional reinforcement learning-based resource allocation scheme without sacrificing the sum rate of the V2X network or the outage probability of V2V links.

Multi-agent Reinforcement Learning-based Resource Allocation Scheme for UAV-assisted Internet of Remote Things Systems

Multi-agent Reinforcement Learning-based Resource Allocation Scheme for UAV-assisted Internet of Remote Things Systems

Deep Learning-Based Resource Allocation Scheme for Heterogeneous NOMA Networks

Deep Learning-Based Resource Allocation Scheme for Heterogeneous NOMA Networks

Privacy-Preserving Intelligent Resource Allocation for Federated Edge Learning in Quantum Internet

Federated edge learning (FEL) is a promising paradigm of distributed machine learning that can preserve data privacy while training the global model collaboratively. However, FEL is still facing model confidentiality issues due to eavesdropping risks of exchanging cryptographic keys through traditional encryption schemes. Therefore, in this paper, we propose a hierarchical architecture for quantum-secured FEL systems with ideal security based on the quantum key distribution (QKD) to facilitate public key and model encryption against eavesdropping attacks. Specifically, we propose a stochastic resource allocation model for efficient QKD to encrypt FEL keys and models. In FEL systems, remote FEL workers are connected to cluster heads via quantum-secured channels to train an aggregated global model collaboratively. However, due to the unpredictable number of workers at each location, the demand for secret-key rates to support secure model transmission to the server is unpredictable. The proposed systems need to efficiently allocate limited QKD resources (i.e., wavelengths) such that the total cost is minimized in the presence of stochastic demand by formulating the optimization problem for the proposed architecture as a stochastic programming model. To this end, we propose a federated reinforcement learning-based resource allocation scheme to solve the proposed model without complete state information. The proposed scheme enables QKD managers and controllers to train a global QKD resource allocation policy while keeping their private experiences local. Numerical results demonstrate that the proposed schemes can successfully achieve the cost-minimizing objective under uncertain demand while improving the training efficiency by about 50\% compared to state-of-the-art schemes.

DRAM: A DRL-based resource allocation scheme for MAR in MEC

Mobile Edge Computing (MEC) and 5G technology allow clients to access computing resources at the network frontier, which paves the way for applying Mobile Augmented Reality (MAR) applications. Under the MEC paradigm, MAR clients can offload complex tasks to the MEC server and enhance the human perception of the world by merging the received virtual information with the real environment. However, the resource allocation problem arises as a critical challenge in circumstances where several MAR clients compete for limited resources at the network frontier. In this paper, we aim to design an online resource allocation scheme on the MEC server that takes both high quality of experience and good fairness performance for MAR clients into consideration. We first formulate this problem as a Markov decision process and tackle the challenge of applying the deep reinforcement learning paradigm. Then, we propose DRAM, a Deep reinforcement learning-based Resource allocation scheme for mobile Augmented reality service in MEC. We also propose a self-adaptive algorithm on the MAR client that is derived based on the analysis of the MAR service to tackle client adaptation problems. The simulation results demonstrated that DRAM can provide high quality of experience and simultaneously achieve good fairness performance by coordinating with clients’ adaptation algorithms.

Deep Scanning—Beam Selection Based on Deep Reinforcement Learning in Massive MIMO Wireless Communication System

In this paper, we investigate a deep learning based resource allocation scheme for massive multiple-input-multiple-output (MIMO) communication systems, where a base station (BS) with a large scale antenna array communicates with a user equipment (UE) using beamforming. In particular, we propose Deep Scanning, in which a near-optimal beamforming vector can be found based on deep Q-learning. Through simulations, we confirm that the optimal beam vector can be found with a high probability. We also show that the complexity required to find the optimum beam vector can be reduced significantly in comparison with conventional beam search schemes.

Imperfect CSI Based Intelligent Dynamic Spectrum Management Using Cooperative Reinforcement Learning Framework in Cognitive Radio Networks

The rapid development of wireless traffic pushed the wireless community to research different solutions towards the efficient utilization of the available radio spectrum. However, a recent study shows that most of the dynamically allocated spectrum bands (radio frequency resources), experience significant underutilization as cognitive radio (CR) technology still lacks intelligence. An intelligence in CRs can be incorporated with machine learning algorithms. Further, the perfect channel state information (CSI) is hardly obtained and CSI imperfections play a crucial role in Dynamic Spectrum Management. Thus, for efficient utilization of available spectrum, a decentralized Multi-Agent Reinforcement Learning based resource allocation scheme has been proposed. A robust resource allocation scheme is proposed which integrates machine learning and CR technology into a sophisticated multi-agent system (MAS). Moreover, assisted with cloud computing which provides a huge amount of storage space, reduces operating expenditures, and provides wider flexibility of cooperation. Hence, to foster the performance of the proposed scheme, a cooperative framework in MAS is introduced which enhances the performance of the proposed scheme in terms of network capacity, outage probability, and convergence speed. Numerical results verify the effectiveness of the proposed scheme and show the non-negligible impact of imperfect CSI, thus highlighting the importance of robust designs that maintains users’ QoS in practical wireless networks.

Learning-Based Proactive Resource Allocation for Delay-Sensitive Packet Transmission

In this article, a learning-based proactive resource sharing scheme is proposed for the next-generation core communication networks, where the available forwarding resources at a switch are proactively allocated to the traffic flows in order to maximize the efficiency of resource utilization with delay satisfaction. The resource sharing scheme consists of two joint modules, estimation of resource demands and allocation of available resources. For service provisioning, resource demand of each traffic flow is estimated based on the predicted packet arrival rate. Considering the distinct features of each traffic flow, a linear regression algorithm is developed for resource demand estimation, utilizing the mapping relation between traffic flow status and required resources, upon which a network switch makes decision on allocating available resources for delay satisfaction and efficient resource utilization. To learn the implicit relation between the allocated resources and delay, a multi-armed bandit learning-based resource allocation scheme is proposed, which enables fast resource allocation adjustment to traffic arrival dynamics. The proposed algorithm is proved to be asymptotically approaching the optimal strategy, with polynomial time complexity. Extensive simulation results are presented to demonstrate the effectiveness of the proposed resource sharing scheme in terms of delay satisfaction, traffic adaptiveness, and resource allocation gain.

Intelligent resource allocation management for vehicles network: An A3C learning approach

With the increasing demand of users for high-speed, low-delay and high-reliability services in connected vehicles network, wireless networks with communication, caching and computing convergence become the trend of network development in the future. To improve the quality of services of vehicles network, we propose a virtualized framework for mobile vehicle services, which using a learning-based resource allocation scheme. The dynamic change processes are modeled as Markov chains without making assumptions about the optimization goal and reducing the complexity of resource allocation computing. A high performance asynchronous advantage actor–critic learning algorithm is proposed to solve the complex dynamic resource allocation problem. Base on software-defined networking and information-centric networking, the method can dynamic orchestration of computing and communication resources to enhance the performance of virtual wireless networks. Simulation results verify that the proposed scheme can converge at a fast speed and improve the network operator’s total rewards.

Random forests for resource allocation in 5G cloud radio access networks based on position information

Next generation 5G cellular networks are envisioned to accommodate an unprecedented massive amount of Internet of things (IoT) and user devices while providing high aggregate multi-user sum rates and low latencies. To this end, cloud radio access networks (CRAN), which operate at short radio frames and coordinate dense sets of spatially distributed radio heads, have been proposed. However, coordination of spatially and temporally denser resources for larger sets of user population implies considerable resource allocation complexity and significant system signalling overhead when associated with channel state information (CSI)-based resource allocation (RA) schemes. In this paper, we propose a novel solution that utilizes random forests as supervised machine learning approach to determine the resource allocation in multi-antenna CRAN systems based primarily on the position information of user terminals. Our simulation studies show that the proposed learning based RA scheme performs comparably to a CSI-based scheme in terms of spectral efficiency and is a promising approach to master the complexity in future cellular networks. When taking the system overhead into account, the proposed learning-based RA scheme, which utilizes position information, outperforms legacy CSI-based scheme by up to 100%. The most important factor influencing the performance of the proposed learning-based RA scheme is antenna orientation randomness and position inaccuracies. While the proposed random forests scheme is robust against position inaccuracies and changes in the propagation scenario, we complement our scheme with three approaches that restore most of the original performance when facing random antenna orientations of the user terminal.

A Reinforcement Learning-Based Resource Allocation Scheme for Cloud Robotics

In recent years, robotic systems combined with cloud computing capability have become an emerging topic of discussion in academic fields. The concept of cloud robotics allows the system to offload computing-intensive tasks from the robots to the cloud. An appropriate resource allocation scheme is necessary for the cloud computing service platform to efficiently allocate its computing resources, when the robots send requests asking for computing service. This paper proposes a resource allocation scheme based on reinforcement learning (RL), which can make the cloud to decide whether a request should be accepted and how many resources are supposed to be allocated. The scheme realizes an autonomous management of computing resources through online learning, reduces human participation in scheme planning, and improves the overall utility of the system in the long run. Numerical results demonstrate that the proposed RL-based computing resource allocation scheme has better performances than the greedy allocation scheme.