Radio Resource Management Research Articles

A novel method for spectrum sensing in cognitive radio networks using fractional GWO[sbnd]CS optimization

The co-channel interference and radio resources in a wireless communication system are often systematically managed by employing Radio Resource Management (RRM) strategies. The parameters of radio resources such as transmit power, data rate, error coding, and user allocation can be controlled via the RRM techniques whose main aim is to better exploit the limited radio frequency spectrum resources. The unallocated spectrum resources become spectrum holes and the effective utilization of the spectrum holes is made possible through cognitive radio by improving the network throughput. To enhance the effective energy utilization capacity of the spectrum holes by considering different sensing states, this paper integrates the grey wolf optimization (GWO) with the Cuckoo search (CS) algorithm to form the Fractional GWOCS Optimization model. The Fractional Optimization Model (FOM) allows the Secondary User (SU) for performing periodic sensing and data transfer. The main aim of the Fractional GWOCS Optimization model is to optimize different parameters like power spectral density (PSD), transmission power, and sensing bandwidth (SB). When computing energy efficiency concerning detection probability, SB, PSD, mutual optimization of PSD and SB, and convergence, the proposed methodology offers improved outcomes when compared to the existing techniques.

Toward Safe and Accelerated Deep Reinforcement Learning for Next-Generation Wireless Networks

Deep reinforcement learning (DRL) algorithms have recently gained wide attention in the wireless networks domain. They are considered promising approaches for solving dynamic radio resource management (RRM) problems in next-generation networks. Given their capabilities to build an approximate and continuously updated model of the wireless network environments, DRL algorithms can deal with the multifaceted complexity of such environments. Nevertheless, several challenges hinder the practical adoption of DRL in commercial networks. In this article, we first discuss two key practical challenges that are faced but rarely tackled when developing DRL-based RRM solutions. We argue that it is inevitable to address these DRL-related challenges for DRL to find its way to RRM commercial solutions. In particular, we discuss the need to have safe and accelerated DRL-based RRM solutions that mitigate the slow convergence and performance instability exhibited by DRL algorithms. We then review and categorize the main approaches used in the RRM domain to develop safe and accelerated DRL-based solutions. Finally, a case study is conducted to demonstrate the importance of having safe and accelerated DRL-based RRM solutions. We employ multiple variants of transfer learning (TL) techniques to accelerate the convergence of intelligent radio access network (RAN) slicing DRL-based controllers. We also propose a hybrid TL-based approach and sigmoid function-based rewards as examples of safe exploration in DRL-based RAN slicing.

Multi-Agent Reinforcement Learning for Dynamic Resource Management in 6G in-X Subnetworks

The 6G network enables a subnetwork-wide evolution, resulting in a “network of subnetworks”. However, due to the dynamic mobility of wireless subnetworks, the data transmission of intra-subnetwork and inter-subnetwork will inevitably interfere with each other, which poses a great challenge to radio resource management. Moreover, most existing approaches require the instantaneous channel gain between subnetworks, which are usually difficult to be collected. To tackle these issues, in this paper we propose a novel effective intelligent radio resource management method using multi-agent deep reinforcement learning (MARL), which only needs the sum of received power, named received signal strength indicator (RSSI), on each channel instead of channel gains. However, to directly separate individual interference from RSSI is an almost impossible thing. To this end, we further propose a novel MARL architecture, named GA-Net, which integrates a hard attention layer to model the importance distribution of inter-subnetwork relationships based on RSSI and excludes the impact of unrelated subnetworks, and employs a graph attention network with a multi-head attention layer to exact the features and calculate their weights that will impact individual throughput. Experimental results prove that our proposed framework significantly outperforms both traditional and MARL-based methods in various aspects.

Multiaccess Point Coordination for Next-Gen Wi-Fi Networks Aided by Deep Reinforcement Learning

Wi-Fi in the enterprise - characterized by overlapping Wi-Fi cells - constitutes the design challenge for next-generation networks. Standardization for recently started IEEE 802.11be (Wi-Fi 7) Working Groups has focused on significant medium access control layer changes that emphasize the role of the access point (AP) in radio resource management (RRM) for coordinating channel access due to the high collision probability with the distributed coordination function (DCF), especially in dense overlapping Wi-Fi networks. This paper proposes a novel multi-AP coordination system architecture aided by a centralized AP controller (APC). Meanwhile, a deep reinforcement learning channel access (DLCA) protocol is developed to replace the binary exponential backoff mechanism in DCF to enhance the network throughput by enabling the coordination of APs. First-Order Model-Agnostic Meta-Learning further enhances the network throughput. Subsequently, we also put forward a new greedy algorithm to maintain proportional fairness (PF) among multiple APs. Via the simulation, the performance of DLCA protocol in dense overlapping Wi-Fi networks is verified to have strong stability and outperform baselines such as Shared Transmission Opportunity (SH-TXOP) and Request-to-Send/Clear-to-Send (RTS/CTS) in terms of the network throughput by 10% and 3% as well as the network utility considering proportional fairness by 28.3% and 13.8%, respectively.

Longitudinal Control-Oriented Spectrum Sharing Based on C-V2X for Vehicle Platoons

Combined with the development of wireless communication, the vehicle platoon will play a key role in the field of intelligent transportation system, due to its significant advantages in improving traffic safety, increasing road capacity, decreasing energy consumption, etc. In a vehicle platoon, the driving-related information is collected through vehicle-to-vehicle (V2V) communication to guarantee safety and improve control performance. However, it has been proved that the delay and reliability of V2V communication greatly affect the platoon stability. In this article, we investigate the spectrum-sharing problem for a vehicle platoon satisfying longitudinal control requirements. Particularly, we derive the communication delay constraints that guarantee plant stability and string stability. Then, using the obtained delay constraint, the wireless resource management, including the optimal power allocation and selection of spectrum sharing of the involved vehicles, is carried out in consideration of intracell interference and heterogeneous communication requirements. Finally, simulation results demonstrated the feasibility and effectiveness of the proposed scheme in vehicle-following control and wireless resource allocation.

Approaching AI-native RANs through generalization and scalability of learning

A holistic vision of an AI-native radio-access network (RAN) would be a system designed for artificial intelligence (AI) algorithms, in which a single AI algorithm could learn and govern most networking operations, ranging from the physical layer to Radio Resource Management (RRM).

Dynamic Power Allocation in High Throughput Satellite Communications: A Two-Stage Advanced Heuristic Learning Approach

The dynamic radio resource management technology is an essential technology in high throughput satellite (HTS) communications. Aiming at the problem that the traditional static radio resource allocation is difficult to meet the dynamic traffic, the dynamic radio resource allocation based on the meta-heuristic algorithm has been extensively studied. Since wireless channel conditions, differentiated services have stochastic properties, and the environment's dynamics are unknown in HTS, the dynamic radio resource allocation based on model-free deep reinforcement learning (DRL) method was used to learn the optimal policy through interactions with the situation. However, the generalization ability of the existing DRL cannot fully meet the great dynamic change in HTS. To address this issue, we explore an advanced heuristic learning approach that combines DRL with one heuristic algorithm. In the proposed approach, we apply the DRL to accelerate the convergence of the heuristic algorithm and verify the proposed method in the dynamic power allocation (DPA) in HTS. Compared with the traditional methods, the simulation results show that the proposed approach can accelerate the convergence of the heuristic method for the allocation of power in HTS and has a low time complexity of the decision. Compared with the traditional methods, our proposed method can reduce the computational overhead by 6.2-94.8%.

A QoE-Aware Nonlinear Fuzzy Radio Resource Management Approach for Revenue Enhancement

In this article, the problem of radio resource management is addressed by adjusting the tradeoff between two key quality of experience (QoE) influencing factors, namely transmission rate, and service price. To this end, a piecewise utility function is employed to assess the user QoE in three different quality classes. It comprises the combination of two utility functions corresponding to the transmission rate and the service price. A low-complexity fuzzy-based approach is then introduced to reallocate additional resources to dissatisfied users to increase the number of satisfied users. The simulation results indicate the efficiency and applicability of the novel approach in terms of increasing the service provider revenue while providing approximately the same overall QoE and power consumption. The results also show proposed approach achieves an improvement of at least 15% in total service provider revenue on average compared to other methods.

Multi-Agent Deep Reinforcement Learning for Enhancement of Distributed Resource Allocation in Vehicular Network

To solve a decentralized radio resource management problem in a 5G vehicular network, we propose a novel resource allocation algorithm based on a multiagent deep reinforcement learning (MARL). We let each vehicle act as an individual agent that can select a unique combination of transport block (TB) and transmission power to broadcast periodic packets. Agent explores the environment and collects observations that later will be used to find the best combination of TB and transmission power. We apply an actor-critic reinforcement learning technique to choose optimal TB for each agent. To eliminate a nonstationarity in a multiagent setting, we utilize a centralized training that allows all agents to share their observations over critic networks. The information shared through critic network can assist each agent to learn the policies of other agents. In a decentralized execution, each agent may only use its actor network and local observation to find the most appropriate TB in the given level of transmission power. While training, the actions taken by actor are evaluated by corresponding critic that maps Q-value for all feasible actions in the given state. Our method results in 18% higher packet reception ratio than a spectrum allocation scheme based on a double DQN. The proposed method achieves 33% higher reward than the previous state-of-the-art that is also based on MARL.

Gaussian mixture model based radio environment mapping for an underlay Device‐to‐Device network

AbstractEfficient management of the scant radio resources in densely deployed Device‐to‐Device (D2D) and IoT networks in 5G+ and 6G wireless era would require continuous and reliable information feedback regarding the actual on‐ground resource utilization. Toward that goal, radio environment maps (REM) are becoming increasingly important as they provide critical information relevant for network management. REM essentially entails the estimation of the radio environment at unsampled locations using the values measured by a few dedicated REM sensors by interpolation. Hence, the accuracy of REM is dependent upon two major factors (1) the sensor deployment and (2) the interpolation scheme. So far, the ordinary Kriging (OK) with randomly deployed sensors is the most prevalent method of REM creation. However, the underlying assumption in OK is that the radio environment conforms to a Gaussian spatial random field (GSRF). In reality, the spatial random field generated by a D2D network displays non‐Gaussian characteristics. Hence, a variational Bayesian based Gaussian mixture model enabled Kriging (VK) method has been proposed in this work which achieves better interpolation accuracy than OK with a lower run‐time computational complexity. A comparison of various sensor deployment schemes on the basis of REM accuracy in fading channel conditions has also been conducted. Additionally, a spatial‐correlation based method has been proposed to estimate the minimum number of required REM sensors to ensure reliable REM performance. The simulation results indicate a superiority of hexagonal deployment of sensors over other schemes and establish VK as a more accurate interpolation algorithm than OK.

Maritime Communications: A Survey on Enabling Technologies, Opportunities, and Challenges

Water covers 71% of the Earth’s surface, where the steady increase in oceanic activities has promoted the need for reliable maritime communication technologies. The existing maritime communication systems involve terrestrial, aerial, and space networks. This article presents a holistic overview of the different forms of maritime communications and provides the latest advances in various marine technologies. This article first introduces the different techniques used for maritime communications over the radio frequency (RF) and optical bands. Then, we present the channel models for RF and optical bands, modulation and coding schemes, coverage and capacity, and radio resource management in maritime communications. After that, this article presents some emerging use cases of maritime networks, such as the Internet of Ships and the ship-to-underwater Internet of Things. Finally, we highlight a few exciting open challenges and identify a set of future research directions for maritime communication, including bringing broadband connectivity to the deep sea, using terahertz and visible light signals for on-board applications, and data-driven modeling for radio and optical marine propagation.

Reinforcement Learning for Radio Resource Management in RAN Slicing: A Survey

Dynamic radio resource allocation to network slices in mobile networks is challenging due to the diverse requirements of RAN slices and the dynamic environment of wireless networks. Reinforcement learning (RL) has been successfully applied to solve different network resource allocation problems where an agent learns how to choose the best action from the interactions with the environment. This survey studies the state-of-the-art RL approaches that address radio resource management in radio access network (RAN) slicing. To this end, we first categorize different problem definitions based on the network environment. Then, we explain how each environment can be modeled as a Markov decision process (MDP) and what RL algorithms can be used to solve them. In addition, we discuss the challenges present in existing works and suggest strategies to address them.

Joint Management of Compute and Radio Resources in Mobile Edge Computing: A Market Equilibrium Approach

Edge computing has been recently introduced to bring computational capabilities closer to end-users of modern network-based services, supporting existing and future delay-sensitive applications by effectively addressing the high propagation delay issue that affects cloud computing. However, the problem of efficiently and fairly managing the system resources presents particular challenges due to the limited capacity of both edge nodes and wireless access networks and the heterogeneity of resources and services’ requirements. To this end, we propose a techno-economic market where service providers act as buyers, securing both radio and computing resources to execute their associated end-users’ jobs while being constrained by a budget limit. We design an allocation mechanism that employs convex programming to find the unique market equilibrium point that maximizes fairness while ensuring that all buyers receive their preferred resource bundle. Additionally, we derive theoretical properties that confirm how the market equilibrium approach strikes a balance between fairness and efficiency. We also propose alternative allocation mechanisms and give a comparison with the market-based mechanism. Finally, we conduct simulations to numerically analyze and compare the performance of the mechanisms and confirm the market model's theoretical properties.

A Joint Learning and Game-Theoretic Approach to Multi-Dimensional Resource Management in Fog Radio Access Networks

Fog radio access networks (F-RANs) have been regarded as a promising paradigm to support latency-sensitive and computation-intensive services by leveraging computing and caching capabilities of fog access points (F-APs). To execute offloaded computation tasks, it is essential to pre-store necessary programs and databases at F-APs, referred as service caching. However, due to system dynamics and the coupling of decisions, cache, radio, and computation resource management at F-RANs is a challenging problem. In this paper, a joint multi-dimensional resource optimization problem is formulated, aiming at minimizing the weighted sum of expected latency cost and caching cost, which is featured by a two-timescale structure. For radio and computation resource allocation on a small timescale, a coalitional game based approach is proposed under given caching decisions. On a larger timescale, services are cached at each F-AP using a multi-agent reinforcement learning algorithm. The advantage of the proposed algorithms is that they can implicitly take the impact of small timescale resource allocation into account while adapting to the long-term statistics of channel coefficients and user service requests. We analyze the proposed algorithms with respect to their convergence, optimality, and complexity, and validate their performance with extensive simulations, where superior performance is observed over several baseline schemes.

From Design to Deployment of Zero Touch Deep Reinforcement Learning WLANs

Machine learning is increasingly used to automate networking tasks, in a paradigm known as zero touch network and service management (ZSM). In particular, deep reinforcement learning (DRL) techniques have recently gained much attention for their ability to learn taking complex decisions in different fields. In the ZSM context, DRL is an appealing candidate for tasks such as dynamic resource allocation, which are generally formulated as hard optimization problems. At the same time, successful training and deployment of DRL agents in real-world scenarios face a number of challenges that we outline and address in this article. Tackling the case of wireless local area network radio resource management, we report guidelines that extend to other usecases and more general contexts.

Radio resource management in energy harvesting cooperative cognitive UAV assisted IoT networks: A multi-objective approach

Cooperative communication through energy harvested relays in Cognitive Internet of Things (CIoT) has been envisioned as a promising solution to support massive connectivity of Cognitive Radio (CR) based IoT devices and to achieve maximal energy and spectral efficiency in upcoming wireless systems. In this work, a cooperative CIoT system is contemplated, in which a source acts as a satellite, communicating with multiple CIoT devices over numerous relays. Unmanned Aerial Vehicles (UAVs) are used as relays, which are equipped with onboard Energy Harvesting (EH) facility. We adopted a Power Splitting (PS) method for EH at relays, which are harvested from the Radio frequency (RF) signals. In conjunction with this, the Decode and Forward (DF) relaying strategy is used at UAV relays to transmit the messages from the satellite source to the CIoT devices. We developed a Multi-Objective Optimization (MOO) framework for joint optimization of source power allocation, CIoT device selection, UAV relay assignment, and PS ratio determination. We formulated three objectives: maximizing the sum rate and the number of admitted CIoT in the network and minimizing the carbon dioxide emission. The MOO formulation is a Mixed-Integer Non-Linear Programming (MINLP) problem, which is challenging to solve. To address the joint optimization problem for an epsilon optimal solution, an Outer Approximation Algorithm (OAA) is proposed with reduced complexity. The simulation results show that the proposed OAA is superior in terms of CIoT device selection and network utility maximization when compared to those obtained using the Nonlinear Optimization with Mesh Adaptive Direct-search (NOMAD) algorithm.

Energy-Efficient Resource Allocation Based on Deep Q-Network in V2V Communications.

Recently, with the development of autonomous driving technology, vehicle-to-everything (V2X) communication technology that provides a wireless connection between vehicles, pedestrians, and roadside base stations has gained significant attention. Vehicle-to-vehicle (V2V) communication should provide low-latency and highly reliable services through direct communication between vehicles, improving safety. In particular, as the number of vehicles increases, efficient radio resource management becomes more important. In this paper, we propose a deep reinforcement learning (DRL)-based decentralized resource allocation scheme in the V2X communication network in which the radio resources are shared between the V2V and vehicle-to-infrastructure (V2I) networks. Here, a deep Q-network (DQN) is utilized to find the resource blocks and transmit power of vehicles in the V2V network to maximize the sum rate of the V2I and V2V links while reducing the power consumption and latency of V2V links. The DQN also uses the channel state information, the signal-to-interference-plus-noise ratio (SINR) of V2I and V2V links, and the latency constraints of vehicles to find the optimal resource allocation scheme. The proposed DQN-based resource allocation scheme ensures energy-efficient transmissions that satisfy the latency constraints for V2V links while reducing the interference of the V2V network to the V2I network. We evaluate the performance of the proposed scheme in terms of the sum rate of the V2X network, the average power consumption of V2V links, and the average outage probability of V2V links using a case study in Manhattan with nine blocks of 3GPP TR 36.885. The simulation results show that the proposed scheme greatly reduces the transmit power of V2V links when compared to the conventional reinforcement learning-based resource allocation scheme without sacrificing the sum rate of the V2X network or the outage probability of V2V links.

Optimal Learning Paradigm and Clustering for Effective Radio Resource Management in 5G HetNets

Ultra-dense heterogeneous networks (UDHN) based on small cells are a requisite part of the future cellular networks as they are proposed as one of the enabling technologies to handle coverage and capacity problems. But co-tier and cross-tier interferences in UDHN severely degrade the quality of service due to K-tiered architecture. Machine learning based radio resource management either through independent learning or cooperative learning is a proven efficient scheme for interference mitigation and quality of service provision in UDHN in a both distributive and cooperative manner. However, an optimal learning paradigm selection, i.e., either independent or cooperative learning and optimal cooperative cluster size in cooperative learning for efficient radio resource management in UDHN is still an open research problem. In this article, a Q-learning based radio resource management scheme is proposed and evaluated for both distributive and cooperative schemes using independent and cooperative learning. The proposed Q-learning solution follows the ϵ-greedy policy for optimal convergence. The simulation results for the UDHN in an urban setup show that in comparison to the independent learning paradigm, cooperative learning has no significant impact on macro cell user capacity. However, there is a significant improvement in small cell user capacity and the sum capacity of the cooperating small cells in the cluster. A significant increase of 48.57% and 37.9% is observed in the small cell user capacity, and sum capacity of the cooperating small cells, respectively, using cooperative learning as compared to independent learning which sets cooperative learning as an optimal learning strategy in UDHN. The improvement in small cell user capacity is at cost of increased computational time which is directly proportional to the number of cooperating small cells. To solve the issue of computational time in cooperative learning, an optimal clustering algorithm is proposed. The proposed optimal clustering reduced the computational time by four times in cooperative Q-learning.

Learning Resilient Radio Resource Management Policies With Graph Neural Networks

We consider the problems of user selection and power control in wireless interference networks, comprising multiple access points (APs) communicating with a group of user equipment devices (UEs) over a shared wireless medium. To achieve a high aggregate rate, while ensuring fairness across all users, we formulate a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">resilient</i> radio resource management (RRM) policy optimization problem with per-user minimum-capacity constraints that <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">adapt</i> to the underlying network conditions via learnable slack variables. We reformulate the problem in the Lagrangian dual domain, and show that we can parameterize the RRM policies using a finite set of parameters, which can be trained alongside the slack and dual variables via an unsupervised primal-dual approach thanks to a provably small duality gap. We use a scalable and permutation-equivariant graph neural network (GNN) architecture to parameterize the RRM policies based on a graph topology derived from the instantaneous channel conditions. Through experimental results, we verify that the minimum-capacity constraints adapt to the underlying network configurations and channel conditions. We further demonstrate that, thanks to such adaptation, our proposed method achieves a superior tradeoff between the average rate and the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$5^\mathrm{th}$</tex-math></inline-formula> percentile rate—a metric that quantifies the level of fairness in the resource allocation decisions—as compared to baseline algorithms.

Twin Delayed DDPG based Dynamic Power Allocation for Mobility in IoRT

The internet of robotic things (IoRT) is a modern as well as fast-evolving technology employed in abundant socio-economical aspects which connect user equipment (UE) for communication and data transfer among each other. For ensuring the quality of service (QoS) in IoRT applications, radio resources, for example, transmitting power allocation (PA), interference management, throughput maximization etc., should be efficiently employed and allocated among UE. Traditionally, resource allocation has been formulated using optimization problems, which are then solved using mathematical computer techniques. However, those optimization problems are generally nonconvex as well as nondeterministic polynomial-time hardness (NP-hard). In this paper, one of the most crucial challenges in radio resource management is the emitting power of an antenna called PA, considering that the interfering multiple access channel (IMAC) has been considered. In addition, UE has a natural movement behavior that directly impacts the channel condition between remote radio head (RRH) and UE. Additionally, we have considered two well-known UE mobility models i) random walk and ii) modified Gauss-Markov (GM). As a result, the simulation environment is more realistic and complex. A data-driven as well as model-free continuous action based deep reinforcement learning algorithm called twin delayed deep deterministic policy gradient (TD3) has been proposed that is the combination of policy gradient, actor-critics, as well as double deep Q-learning (DDQL). It optimizes the PA for i) stationary UE, ii) the UE movements according to random walk model, and ii) the UE movement based on the modified GM model. Simulation results show that the proposed TD3 method outperforms model-based techniques like weighted MMSE (WMMSE) and fractional programming (FP) as well as model-free algorithms, for example, deep Q network (DQN) and DDPG in terms of average sum-rate performance.