Radio Access Network Slicing Research Articles

Towards efficient RAN slicing: A deep hierarchical reinforcement learning approach

Radio access network (RAN) slicing can significantly improve network flexibility and resource utilization efficiency. Generally, deep reinforcement learning (DRL) is a prevailing approach to implementing practicable resource management in RAN slicing. However, this demand-aware resource allocation suffers from long training time and slow execution efficiency. This can lead to the inability to quickly achieve the desired spectral efficiency and service level agreement satisfaction rate. To tackle this issue, we propose a novel RAN slicing resource allocation framework based on a deep hierarchical RL framework for efficient resource scheduling. Structurally, our framework consists of a policy selection network and a policy evaluation network. In particular, a newly built action acceleration unit can achieve quick reward accumulation, thus speeding up the optimal policy search. Extensive simulations show that our proposal has higher system utility and faster convergence compared to the state-of-the-art DRL algorithms.

Hierarchical Slicing: A New Paradigm of Radio Resource Management for Mobile Networks

Radio access network slicing is considered to be highly challenging due to the complexity of mobile environments and the diversity of mobile services. Existing works in this area mainly decompose the radio resource management problem into slice-level resource management and user-level resource scheduling. In this paper, we propose a hierarchical three-tier slicing architecture, where an additional tier is introduced to merge the gap between the conventional two tiers in time and spatial scales. Specifically, the medium-scale tier provides intercell and inter-slice coordination to address short-term dynamics caused by local user mobility and traffic variations, which highly simplifies the upper and lower tiers by allowing them to focus on long-term traffic distribution and instant user demands, respectively. Thus, it allows each independent iter to apply more flexible and more efficient radio resource management methods with appropriate time and spatial scales. A proof of concept is provided to show that the proposed three-tier architecture can achieve a better tradeoff between slice isolation and slice capacity as compared to the state-of-the-art two-tier solution.

Hierarchical Intelligent Radio Access Network Slicing for Differential Service Level Agreement Guaranteeing

Hierarchical Intelligent Radio Access Network Slicing for Differential Service Level Agreement Guaranteeing

Holistic Roadmap of Trends in Radio Access Network Slicing: A Survey

Holistic Roadmap of Trends in Radio Access Network Slicing: A Survey

Security-aware 5G RAN slice mapping with tiered isolation in physical-layer secured metro-aggregation elastic optical networks using heuristic-assisted DRL

The optical transport network (OTN) encryption technology is attractive to solve the physical-layer security in services for the light-path provision process. This paper mainly explores the security-aware 5G radio access network (RAN) slice mapping problem with the tiered isolation (TI) policy, which decides the solution for aggregating service into the physical-layer secured metro-aggregation elastic optical networks (MA-EONs). We first introduce the physical-layer secured OTNs and illustrate their differences from the traditional optical networks. Then, we formulate the 5G RAN slice mapping problem in physical-layer secured MA-EONs as an exact integer linear programming (ILP) model to minimize the average cost (AC), which consists of the number of utilized processing pools (PPs)/general-purpose processors (GPPs)/virtual machines (VMs), and maximum frequency slot index (MFSI) on the light-paths, meanwhile satisfying the given slice’s latency, isolation, and security requirements. After that, to overcome the non-scalability problem of the ILP model, a heuristic-assisted deep reinforcement learning (HA-DRL) algorithm is proposed to obtain a near-optimal solution for large-scale network scenarios, where the classical shortest path algorithm is employed in the DRL to shrink the size of the exploration space and accelerate the convergence process. Finally, we evaluate the proposed ILP model and HA-DRL algorithm through extensive simulations. Simulation results indicate that our proposed HA-DRL method can find approximate solutions to the ILP model in the small-scale network scenario. Furthermore, the HA-DRL method can also achieve higher resource efficiency compared with benchmark heuristic first-fit algorithms in the large-scale network scenario. In comparison to the first-fit algorithm benchmark, the proposed HA-DRL can achieve up to 9.4% AC reduction in large-scale network scenarios.

Knowledge Transfer and Reuse: A Case Study of Ai-Enabled Resource Management in RAN Slicing

An efficient resource management scheme is critical to enable network slicing in 5G networks and in envisioned 6G networks, and artificial intelligence (AI) techniques offer promising solutions. Considering the rapidly emerging new machine learning techniques, such as graph learning, federated learning, and transfer learning, a timely survey is needed to provide an overview of resource management and network slicing techniques of AI-enabled wireless networks. This article provides such a survey along with an application of knowledge transfer in radio access network (RAN) slicing. In particular, we first provide some background on resource management and network slicing, and review relevant state-of-the-art AI and machine learning (ML) techniques and their applications. Then, we introduce our AI-enabled knowledge transfer and reuse-based resource management (AKRM) scheme, where we apply transfer learning to improve system performance. Compared with most existing works, which focus on the training of standalone agents from scratch, the main difference of AKRM lies in its knowledge transfer and reuse capability between different tasks. Our paper aims to be a roadmap for researchers to use knowledge transfer schemes in AI-enabled wireless networks, and we provide a case study over the resource allocation problem in RAN slicing.

Consortium Blockchain-Based Spectrum Trading for Network Slicing in 5G RAN: A Multi-Agent Deep Reinforcement Learning Approach

Network slicing (NS) is envisioned as an emerging paradigm for accommodating different virtual networks on a common physical infrastructure. Considering the integration of blockchain and NS, a secure decentralized spectrum trading platform can be established for autonomous radio access network (RAN) slicing. Moreover, the realization of proper incentive mechanisms for fair spectrum trading is crucial for effective RAN slicing. This paper proposes a novel hierarchical framework for blockchain-empowered spectrum trading for NS in RAN. Specifically, we deploy a consortium blockchain platform for spectrum trading among spectrum providers and buyers for slice creation, and autonomous slice adjustment. For slice creation, the spectrum providers are infrastructure providers (InPs) and buyers are mobile virtual network operators (MVNOs). Then, underloaded MVNOs with extra spectrum to spare, trade with overloaded MVNOs, for slice spectrum adjustment. For proper incentive maximization, we propose a three-stage Stackelberg game framework among InPs, seller MVNOs, and buyer MVNOs, for joint optimal pricing and demand prediction strategies. Then, a multi-agent deep reinforcement learning (MADRL) method is designed to achieve a Stackelberg equilibrium (SE). Security assessment and extensive simulation results confirm the security and efficacy of our proposed method in terms of players’ utility maximization and fairness, compared with other baselines.

Joint Multi-Objective Optimization for Radio Access Network Slicing Using Multi-Agent Deep Reinforcement Learning

Radio access network (RAN) slices can provide various customized services for next-generation wireless networks. Thus, multiple performance metrics of different types of RAN slices need to be jointly optimized. However, existing efforts in multi-objective optimization problem (MOOP) for RAN slicing are only in the scalar form, which is difficult to achieve simultaneous optimization. In this paper, we consider a non-scalar MOOP for RAN slicing with three types of slices, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> , the high-bandwidth slice, the low-delay slice, and the wide-coverage slice over the same underlying physical network. We jointly optimize the throughput, the transmission delay, and the coverage area by user-oriented dynamic virtual base stations (vBSs)' deployment, and sub-channel and power allocation. An improved multi-agent deep deterministic policy gradient (IMADDPG) algorithm, having the characteristics of centralized training and distributed execution, is proposed to solve the above non-deterministic polynomial-time hard (NP-hard) problem. The rank voting method is introduced in the inference process to obtain near-Pareto optimal solutions. Simulation results verify that the proposed scheme can ensure better performance than the traditional scalar utility method and other benchmark algorithms. The proposed scheme has the advantage of flexibly approaching any point of the Pareto boundary, while the traditional scalar method only subjectively approaches one of the Pareto optimal solutions. Furthermore, our proposal strikes a compelling tradeoff among three types of RAN slices due to the non-dominance between Pareto optimal solutions.

Deep Reinforcement Learning for NextG Radio Access Network Slicing With Spectrum Coexistence

Reinforcement learning (RL) is applied for dynamic admission control and resource allocation in NextG radio access network slicing. When sharing the spectrum with an incumbent user (that dynamically occupies frequency-time blocks), communication and computational resources are allocated to slicing requests, each with priority (weight), throughput, latency, and computational requirements. RL maximizes the total weight of granted requests over time beyond myopic, greedy, random, and first come, first served solutions. As the state-action space grows, Deep Q-network effectively admits requests and allocates resources as a low-complexity solution that is robust to sensing errors in detecting the incumbent user activity.

Handoff Control and Resource Allocation for RAN Slicing in IoT Based on DTN: An Improved Algorithm Based on Actor–Critic Framework

As a three-layer association of Internet of things equipment (IoTE) -network slicing (NS) -base station (BS) in radio access network (RAN) slicing, handoff control and resource allocation has become an important but complicated issue. In addition, the centralized controller has a difficult grasping the network situation in real time. In view of this, the problem of handoff control in the RAN slicing are investigated in the digital twin network (DTN), with the goal of maximizing the long-term utility about user satisfaction and handoff cost. Then an improved algorithm based on actor-critic framework is suggested, which is called HCRA. Specifically, the actor component contains neural networks for Handoff Control and an optimizer for Resource Allocation, and then the critic component evaluates the handoff and resource allocation actions of the actor component to guide the optimization of actions in the actor component. The simulation results show that HCRA can obtain better performance than benchmark algorithms.

SNAF: DRL-Based Interdependent E2E Resource Slicing Scheme for a Virtualized Network

The fifth generation (5G) mobile cellular network relies on network slicing (NS) to satisfy the diverse quality of service (QoS) requirements of various service providers operating on a standard shared infrastructure. However, the synchronization of radio access network (RAN) and core network (CN) slicing has not been well-studied as an interdependent resource allocation problem. This work proposes a novel slice-to-node access factor (SNAF)-based end-to-end (E2E) slice resource provisioning scheme and deep reinforcement learning (DRL)-based real-time resource allocation algorithm for E2E interdependent resource slicing and allocation, respectively, specifically for RAN and CN. To ensure effective resource slicing and allocation, we consider the versatile user equipment (UEs) QoS requirements on transmission delay and data rate. Notably, the SNAF-based scheme provides proper resource provisioning and traffic synchronization, while the DRL-based algorithm allocates radio resources based on affordable traffic and backhaul resources. Based on the 5G air interface, we conduct system-level simulations to evaluate the performance of our proposed methods from various perspectives. Simulation results confirm that our proposed SNAF and DRL-based interdependent E2E resource slicing and allocation techniques achieve better E2E traffic-resource synchronization, and improve the QoS satisfaction with minimal resource utilization compared to other existing benchmark schemes.

Cooperative-Aware Radio Resource Allocation Scheme for 5G Network Slicing in Cloud Radio Access Networks.

The 5G network is designed to serve three main use cases: enhanced mobile broadband (eMBB), massive machine-type communications (mMTC), and ultra-reliable and low-latency communications (uRLLC). There are many new technological enablers, including the cloud radio access network (C-RAN) and network slicing, that can support 5G and meet its requirements. The C-RAN combines both network virtualization and based band unit (BBU) centralization. Using the network slicing concept, the C-RAN BBU pool can be virtually sliced into three different slices. 5G slices require a number of Quality of service (QoS) metrics, such as average response time and resource utilization. In order to enhance the C-RAN BBUs utilization while protecting the minimum QoS of the coexisting three slices, a priority-based resource allocation with queuing model is proposed. The uRLLC is given the highest priority, while eMBB has a higher priority than mMTC services. The proposed model allows the eMBB and mMTC to be queued and the interrupted mMTC to be restored in its queue to increase its chance to reattempt the service later. The proposed model's performance measures are defined and derived using a continuous-time Markov chain (CTMC) model and evaluated and compared using different methodologies. Based on the results, the proposed scheme can increase C-RAN resource utilization without degrading the QoS of the highest-priority uRLLC slice. Additionally, it can reduce the forced termination priority of the interrupted mMTC slice by allowing it to re-join its queue. Therefore, the comparison of the results shows that the proposed scheme outperforms the other states of the art in terms of improving the C-RAN utilization and enhancing the QoS of eMBB and mMTC slices without degrading the QoS of the highest priority use case.

Heuristic Radio Access Network Subslicing with User Clustering and Bandwidth Subpartitioning

In 5G and beyond, the network slicing is a crucial feature that ensures the fulfillment of service requirements. Nevertheless, the impact of the number of slices and slice size on the radio access network (RAN) slice performance has not yet been studied. This research is needed to understand the effects of creating subslices on slice resources to serve slice users and how the performance of RAN slices is affected by the number and size of these subslices. A slice is divided into numbers of subslices of different sizes, and the slice performance is evaluated based on the slice bandwidth utilization and slice goodput. A proposed subslicing algorithm is compared with k-means UE clustering and equal UE grouping. The MATLAB simulation results show that subslicing can improve slice performance. If the slice contains all UEs with a good block error ratio (BLER), then a slice performance improvement of up to 37% can be achieved, and it comes more from the decrease in bandwidth utilization than the increase in goodput. If a slice contains UEs with a poor BLER, then the slice performance can be improved by up to 84%, and it comes only from the goodput increase. The most important criterion in subslicing is the minimum subslice size in terms of resource blocks (RB), which is 73 for a slice that contains all good-BLER UEs. If a slice contains UEs with poor BLER, then the subslice can be smaller.

Reinforcement Learning for Radio Resource Management in RAN Slicing: A Survey

Dynamic radio resource allocation to network slices in mobile networks is challenging due to the diverse requirements of RAN slices and the dynamic environment of wireless networks. Reinforcement learning (RL) has been successfully applied to solve different network resource allocation problems where an agent learns how to choose the best action from the interactions with the environment. This survey studies the state-of-the-art RL approaches that address radio resource management in radio access network (RAN) slicing. To this end, we first categorize different problem definitions based on the network environment. Then, we explain how each environment can be modeled as a Markov decision process (MDP) and what RL algorithms can be used to solve them. In addition, we discuss the challenges present in existing works and suggest strategies to address them.

Federated Deep Reinforcement Learning for Open RAN Slicing in 6G Networks

Radio access network (RAN) slicing is a key element in enabling current 5G networks and next-generation networks to meet the requirements of different services in various verticals. However, the heterogeneous nature of these services' requirements, along with the limited RAN resources, makes RAN slicing very complex. Indeed, the challenge that mobile virtual network operators (MVNOs) face is to rapidly adapt their RAN slicing strategies to the frequent changes of the environment constraints and service requirements. Machine learning techniques, such as deep reinforcement learning (DRL), are increasingly considered a key enabler for automating the management and orchestration of RAN slicing operations. Nerveless, the ability to generalize DRL models to multiple RAN slicing environments may be limited due to their strong dependence on the environment data on which they are trained. Federated learning enables MVNOs to leverage more diverse training inputs for DRL without the high cost of collecting this data from different RANs. In this article, we propose a federated deep reinforcement learning approach for Open RAN Slicing. In this approach, MVNOs collaborate to improve the performance of their DRL-based RAN slicing models. Each MVNO trains a DRL model and sends it for aggregation. The aggregated model is then sent back to each MVNO for immediate use and further training. The simulation results show the effectiveness of the proposed DRL approach.

Potential-Game-Based 5G RAN Slice Planning for GBR Services

The Radio Access Network (RAN) slice planning is a key phase within the RAN slice management and orchestration process. Based on the performance requirements of requested RAN slices and key performance indicators of the RAN and existing RAN slices, the RAN slice planning mainly consists of deciding (a) the feasibility of deploying new RAN slices; (b) re-configuring the existing RAN accordingly; and (c) the need to renegotiate the Service Level Agreements (SLAs) and/or expand the RAN (i.e., radio resources, carriers, cells etc) if one or more RAN slices cannot be accommodated in a first attempt. Under this context, we propose a framework for planning RAN slices which require their data sessions get a Guaranteed Bit Rate (GBR) and the probability of blocking such sessions is below a threshold. To meet such requirements, our framework plans the amount of prioritized radio resources for new and already deployed RAN slices. We formulate the RAN slice planning as multiple ordinal potential games and demonstrate the existence of a Nash Equilibrium solution which minimizes the average probability of blocking data sessions for all the RAN slices. We perform detailed simulations to demonstrate the effectiveness of the proposed solution in terms of performance, and renegotiation capability.

How to Attack and Defend NextG Radio Access Network Slicing With Reinforcement Learning

In this paper, reinforcement learning (RL) for network slicing is considered in next generation (NextG) radio access networks, where the base station (gNodeB) allocates resource blocks (RBs) to the requests of user equipments and aims to maximize the total reward of accepted requests over time. Based on adversarial machine learning, a novel over-the-air attack is introduced to manipulate the RL algorithm and disrupt NextG network slicing. The adversary observes the spectrum and builds its own RL based surrogate model that selects which RBs to jam subject to an energy budget with the objective of maximizing the number of failed requests due to jammed RBs. By jamming the RBs, the adversary reduces the RL algorithm's reward. As this reward is used as the input to update the RL algorithm, the performance does not recover even after the adversary stops jamming. This attack is evaluated in terms of both the recovery time and the (maximum and total) reward loss, and it is shown to be much more effective than benchmark (random and myopic) jamming attacks. Different reactive and proactive defense schemes such as suspending the RL algorithm's update once an attack is detected, introducing randomness to the decision process in RL to mislead the learning process of the adversary, or manipulating the feedback (NACK) mechanism such that the adversary may not obtain reliable information are introduced to show that it is viable to defend NextG network slicing against this attack, in terms of improving the RL algorithm's reward.

Knowledge-based flexible resource allocation optimisation strategy for multi-tenant radio access network slicing in 5G and B5G

Formalisation of the network slice as a resource allocation unit is considered a promising aspect that enables scalable and flexible resource allocation among many tenants in 5G and beyond 5G (B5G) communication networks. However, the user traffic has to be passed through the central administration for processing, which leads to latency problems. To solve this problem, recent research works have suggested fixed central-to-edge resource allocation ratios as per the service type. However, this approach leads to over-provisioning of some resources. This paper provides a flexible resource allocation approach for 5G slice networks operating in a heterogeneous environment with multiple tenants and tiers. A radial basis-neural network is used to convert abstract specifications of simulation activities into precise resource needs, and then a genetic algorithm-based flexible multi-resource allocation scheme is proposed, where a versatile optimisation framework is used. The results show that the proposed approach outperforms such existing schemes.

Knowledge-based flexible resource allocation optimisation strategy for multi-tenant radio access network slicing in 5G and B5G

Knowledge-based flexible resource allocation optimisation strategy for multi-tenant radio access network slicing in 5G and B5G

Toward an Efficient and Dynamic Allocation of Radio Access Network Slicing Resources for 5G Era

Toward an Efficient and Dynamic Allocation of Radio Access Network Slicing Resources for 5G Era