Utilization Of Network Resources Research Articles

Deep reinforcement learning‐based routing and resource assignment in quantum key distribution‐secured optical networks

AbstractIn quantum key distribution‐secured optical networks (QKD‐ONs), constrained network resources limit the success probability of QKD lightpath requests (QLRs). Thus, the selection of an appropriate route and the efficient utilisation of network resources for establishment of QLRs are the essential and challenging problems. This work addresses the routing and resource assignment (RRA) problem in the quantum signal channel of QKD‐ONs. The RRA problem of QKD‐ONs is a complex decision making problem, where appropriate solutions depend on understanding the networking environment. Motivated by the recent advances in deep reinforcement learning (DRL) for complex problems and also because of its capability to learn directly from experiences, DRL is exploited to solve the RRA problem and a DRL‐based RRA scheme is proposed. The proposed scheme learns the optimal policy to select an appropriate route and assigns suitable network resources for establishment of QLRs by using deep neural networks. The performance of the proposed scheme is compared with the deep‐Q network (DQN) method and two baseline schemes, namely, first‐fit (FF) and random‐fit (RF) for two different networks, namely The National Science Foundation Network (NSFNET) and UBN24. Simulation results indicate that the proposed scheme reduces blocking by 7.19%, 10.11%, and 33.50% for NSFNET and 2.47%, 3.20%, and 19.60% for UBN24 and improves resource utilisation up to 3.40%, 4.33%, and 7.18% for NSFNET and 1.34%, 1.96%, and 6.44% for UBN24 as compared with DQN, FF, and RF, respectively.

Load-aware dynamic controller placement based on deep reinforcement learning in SDN-enabled mobile cloud-edge computing networks

Software-defined networks (SDNs) can improve network resource utilization and optimize the performance of mobile cloud-edge computing networks (MECCNs) through unified and flexible network management. However, network traffic in MECCNs can change over time and space, which affects the performance of the control plane in an SDN. Further, an MECCN may require the temporary addition of network access points, further reducing the network management capabilities of the control plane. To ensure that the control plane can handle the constant changes in network traffic, adapt to dynamic changes in network access points, and provide continuous and efficient network management functions, this study focuses on the dynamic controller placement problem in SDN-enabled MECCNs. We study the deployment of a two-layer control plane and accordingly construct the corresponding delay, load balancing, and control reliability models. Next, we construct a joint optimization problem considering the developed delay, load balancing, and control reliability, and solve this problem using an algorithm based on the deep deterministic policy gradient algorithm. The experimental results demonstrate that the proposed algorithm outperforms other algorithms on a variable-node network.

Reveal: Robustness-aware VNF placement and request scheduling in edge clouds

In the edge cloud network, service providers place virtual network functions (VNFs) in edge clouds to serve users’ requests. Thus, it is essential to consider VNF placement and request scheduling in edge clouds. Existing works often focus on minimizing request completion time or maximizing network throughput to utilize network resources and ensure users’ QoS efficiently. However, they ignore two practical factors: malicious users and failed VNFs, leading to poor network robustness. To this end, this paper studies robustness-aware VNF placement and request scheduling, named Reveal. Specifically, we limit the number of VNFs each user can access and the number of users each VNF can serve to control the influence scope of malicious users and VNF failures. Since placing VNFs is time-consuming and requests arrive dynamically, we solve this problem through two phases: robust VNF placement and assignment, and online request scheduling. For the first phase, we design an efficient knapsack-based rounding algorithm with bounded approximation factors. For online request scheduling, we propose a primal–dual based algorithm with a competitive ratio of 1−ϵ,O(log1/ϵ) where ϵ∈(0,1). Experiment and simulation results show that Reveal can achieve better performance and robustness than other alternatives.

Optimal Device Selection in Federated Learning for Resource-Constrained Edge Networks

Low latency, resource efficiency, and data privacy are some of the crucial requirements in modern communication networks. Federated learning can efficiently address these issues by utilizing the data at the network edge and processing massive amounts of data in parallel at the edge devices, thus ensuring data privacy and low latency. For a large-scale federated learning task spanning many devices, challenges arise due to device heterogeneity, data variability, and limited network resources. Optimal selection of edge devices participating in federated learning is essential to attaining resilient, reliable, and resource-efficient edge networks. In this context, this paper proposes an optimal device selection method to minimize redundant data training and improve network resource utilization without affecting the performance of federated learning over resource-constrained edge networks. The proposed optimal device selection method aims to minimize network resource demands while maximizing data diversity within the aggregated model. The performance of the proposed federated learning framework is evaluated using a publicly available image dataset of handwritten digits, EMNIST (an extended version of the MNIST dataset). Experimental results indicate that the proposed framework can obtain accuracy convergence performance on par with conventional federated learning methods while significantly reducing device usage (up to 50%) and resource utilization (up to 30%) while reaching 99% of achievable accuracy. The proposed method can therefore be effectively applied to resource-constrained edge networks.

Fairness Guaranteed and Auction-Based x-Haul and Cloud Resource Allocation in Multi-Tenant O-RANs

The open-radio access network (O-RAN) embraces cloudification and network function virtualization for base-band function processing by dis-aggregated radio units (RUs), distributed units (DUs), and centralized units (CUs). These enable the cloud-RAN vision in full, where multiple mobile network operators (MNOs) can install their proprietary or open RUs, but lease on-demand computational resources for DU-CU functions from commonly available open-clouds via open x-haul interfaces. In this paper, we propose and compare the performances of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">min-max fairness</i> and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Vickrey-Clarke-Groves (VCG) auction</i> -based x-haul and DU-CU resource allocation mechanisms to create a multi-tenant O-RAN ecosystem that is sustainable for small, medium, and large MNOs. The min-max fair approach <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">minimizes the maximum OPEX of RUs</i> through cost-sharing proportional to their demands, whereas the VCG auction-based approach <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">minimizes the total OPEX for all resources utilized while extracting truthful demands from RUs</i> . We consider time-wavelength division multiplexed (TWDM) passive optical network (PON)-based x-haul interfaces where PON virtualization technique is used to flexibly provide optical connections among RUs and edge-clouds at macro-cell RU locations as well as open-clouds at the central office locations. Moreover, we design efficient heuristics that yield significantly better economic efficiency and network resource utilization than conventional greedy resource allocation algorithms and reinforcement learning-based algorithms.

PARS: A Priority-Aware Resource Sharing Scheme for Services With Differential QoS Requirements in B5G Edge Network

Next-generation networks are expected to address scenarios demanding varying QoS, such as higher bandwidth, lower latency, and reliable data delivery. A multitude of B5G applications like AR/VR/XR and health-care needs differential QoS requirements and higher reliability of internet services. In this context, we develop a novel approach, PARS, for resource provisioning among slices for facilitating efficient resource utilization in an edge network. Further, we propose an adaptive SFC-RA algorithm to maximize user admission under computation and bandwidth resource constraints while satisfying SLA requirements of health-care services. Moreover, we show efficacy of our solution by comparing it with existing state-of-the-art methods.

Efficiency-optimized 6G: A virtual network resource orchestration strategy by enhanced particle swarm optimization

The future sixth-generation (6G) wireless systems are expected to encounter emerging services with diverse requirements. In this paper, 6G network resource orchestration is optimized to support customized network slicing of services, and place network functions generated by heterogeneous devices into available resources. This is a combinatorial optimization problem that is solved by developing a particle swarm optimization (PSO) based scheduling strategy with enhanced inertia weight, particle variation, and nonlinear learning factor, thereby balancing the local and global solutions and improving the convergence speed to globally near-optimal solutions. Simulations show that the method improves the convergence speed and the utilization of network resources compared with other variants of PSO.

Performance Analysis of Automatic Hidden Ligthpaths in Multi-Layer Networks

The problem of resource provisioning in multi-layer networks is an important issue. We discuss and analyze resource provisioning in terms of the visibility of optical resources for the virtual layer in a network. In this context, we propose the diversification of the optical spectrum in such a way that some resources are hidden for the virtual layer and used when needed. We introduce and investigate automatic hidden lightpaths to improve resource utilization in multi-layer networks. Our solution uses a software-defined networking concept to automatically set up and release hidden lightpaths, which are not visible at the network layer. We evaluate and analyze two main approaches to resource provisioning in terms of the blocking probability. Future research should focus on resource diversification and the usage of the hidden elastic optical spectrum in modern networks as a promising solution.

A Multiple QoS Metrics-Aware Virtual Network Embedding Algorithm

Abstract As a key issue of network virtualisation, virtual network embedding (VNE) aims to embed multiple virtual network requests (VNRs) from different applications onto the substrate network effectively. In real networks, about 90% of traffic is generated by different quality of service (QoS) sensitive applications. However, most existing VNE algorithms do not account for the difference. Although several VNE algorithms considered the delay metric of applications, they usually provide strict delay guarantees for all VNRs, leading to a low VNR acceptance ratio. In this paper, we focus on the VNE problem involving multiple QoS metrics and propose a multiple QoS metrics-aware VNE algorithm based on reinforcement learning (RLQ-VNE). We first classify VNRs according to their different requirements for multiple QoS metrics including delay, jitter and packet loss rate, and then introduce reinforcement learning to implement differentiated VNE. Specifically, RLQ-VNE provides strict QoS guarantees for the VNRs with high-level QoS requirements and provides lower QoS guarantees for the VNRs with low-level QoS requirements, thus balancing the QoS guarantee and request acceptance ratio. Simulation results from multiple experimental scenarios show that RLQ-VNE improves the request acceptance ratio and network resource utilisation by sacrificing less QoS.

Adaptive Random Access and Data Transmission Scheme With Mixed Traffic in NGSO Satellite Networks

In non-geostationary orbit satellite networks, the resource is limited and the traffic under each satellite is in variation due to the motion of satellites and the non-uniform traffic on the ground. Generally, the uplink communication is guaranteed by successful random access (RA) and sufficient data transmission (DT) resources for various traffic types. Different traffic types usually have different requirements for resources, and joint resource management for mixed traffic tries to meet the needs of various traffic types at the same time and may improve uplink resource efficiency. However, existing strategies have not given integrative consideration to jointly allocate resources for RA and DT under dynamic mixed traffic demands, which leads to inefficient network resource utilization. Considering such a problem, we propose an adaptive random access and data transmission scheme for NGSO satellite networks in the mixed traffic scenario, considering two types of users, i.e., real-time users and best-effort users. Specifically, the utility functions of different types of users are introduced, and a utility-based access control and resource allocation algorithm is developed by solving the network utility maximization problem. Our scheme aims to achieve optimal uplink network utilization by adaptively controlling random access and allocating resources according to the dynamic demands of mixed traffic. Simulation results show that the proposed scheme outperforms other methods in terms of network utility, and exhibits near-optimal performance.

A Resource-Adaptive Routing Scheme with Wavelength Conflicts in Quantum Key Distribution Optical Networks.

Quantum key distribution (QKD) has great potential in ensuring data security. Deploying QKD-related devices in existing optical fiber networks is a cost-effective way to practically implement QKD. However, QKD optical networks (QKDON) have a low quantum key generation rate and limited wavelength channels for data transmission. The simultaneous arrival of multiple QKD services may also lead to wavelength conflicts in QKDON. Therefore, we propose a resource-adaptive routing scheme (RAWC) with wavelength conflicts to achieve load balancing and efficient utilization of network resources. Focusing on the impact of link load and resource competition, this scheme dynamically adjusts the link weights and introduces the wavelength conflict degree. Simulation results indicate that the RAWC algorithm is an effective approach to solving the wavelength conflict problem. Compared with the benchmark algorithms, the RAWC algorithm can improve service request success rate (SR) by up to 30%.

Dynamic multi-hop multipath RMSA and traffic grooming in OFDM-based sliceable elastic optical networks

In elastic optical networks (EON), routing, modulation selection, and spectrum assignment (RMSA) is crucial in provisioning connection requests. Multipath RMSA offers a number of benefits including provisioning of ultra-high bandwidth demands and better utilization of fragmented spectrum resource. By Combining with traffic grooming, multipath RMSA and traffic grooming is able to provide better utilization of network resource in provisioning connection requests. Adding multi-hop routing mechanism to multipath RMSA and traffic grooming increases the flexibility for selecting paths resulting in higher probability of successfully finding routing paths for connection requests. Dynamic multipath RMSA problem in EON has been investigated extensively in the literature. Dynamic multi-hop multipath RMSA and traffic grooming problem in EON is far from been well studied. This paper proposes an algorithm for the dynamic multi-hop multipath RMSA and traffic grooming problem in OFDM-based elastic optical networks with sliceable bandwidth-variable transponders. Performance of the proposed algorithm is studied via simulation. Our simulation results show that the proposed algorithm yields lower bandwidth blocking ratio than an existing algorithm.

Decentralized Deep Reinforcement Learning Meets Mobility Load Balancing

Mobility load balancing (MLB) aims to solve the problem of uneven resource utilization in cellular networks. Since network dynamics are usually complicated and non-stationary, conventional model-based MLB methods fail to cover all scenarios of cellular networks. On the other hand, deep reinforcement learning (DRL) can provide a flexible framework to learn to distribute cell load evenly without explicit modeling of the underlying network dynamics. In this paper, we introduce a novel decentralized DRL-based MLB method where each cell has a DRL agent to learn its handover parameters and antenna tilt angle. As the number of cells increases, the decentralized framework is more computationally efficient than its centralized counterpart by dividing the action space. Furthermore, our designed decentralized DRL architecture only requires readily known information defined in existing cellular standards, and it can achieve a more balanced cell load distribution than the centralized DRL one by using individual reward functions. To provide realistic performance evaluation, a network simulator is introduced strictly following the Third Generation Partnership Project (3GPP) specifications. Furthermore, field data is used to construct the underlying cellular environment. Extensive evaluations have been conducted to demonstrate the fact that the introduced decentralized DRL-based MLB method can achieve a more balanced cell load distribution and a better performance of edge users than the state-of-the-art MLB methods.

Network slicing in virtualized 5G Core with VNF sharing

Through multiplexing separate virtual networks on the same network infrastructure, network slicing will lead to customization, scalability, flexibility, and isolation of services in different domains of the network. It also results in lower CAPEX/OPEX via increased utilization of network resources. In this paper, we have formulated an Integer Linear Programming (ILP) model to create network slices in the 5G virtualized core by considering the sharing of virtual network functions (VNFs) between different network slices to minimize cost. The ILP model aims to minimize the number of VNF instances and resource consumption while meeting the network slice requirements. The main motivation for sharing the 5G core VNFs among different network slices is to reduce the number of VNF instances, which results in a lower management overhead of VNFs. As solving the optimization problems is time-consuming and does not scale for larger deployments, we also developed a Heuristic Backtracking Algorithm for Network Slicing (HBA-NS). Furthermore, in order to account for the dynamic nature of traffic, our proposed network slicing framework also includes a CNN-LSTM deep learning model to predict UEs’ requests in each network slice. Based on the predicted requests, the HBA-NS algorithm scales allocated resources of different network slices to meet the dynamic SLA requirements of users. Our evaluation results show that the solution achieved by the HBA-NS algorithm is on average 6% worse than the optimal solution in the worst case but derived at a significantly lower run time. We also evaluated the proposed network slicing framework in terms of different criteria including its isolation property. Our simulation results show that network slicing can provide better performance compared to competitive approaches.

Hardware-accelerated Real-time Drift-awareness for Robust Deep Learning on Wireless RF Data

Proactive and intelligent management of network resource utilization (RU) using deep learning (DL) can significantly improve the efficiency and performance of the next generation of wireless networks. However, variations in wireless RU are often affected by uncertain events and change points due to the deviations of real data distribution from that of the original training data. Such deviations, which are known as dataset drifts, can subsequently lead to a shift in the corresponding decision boundary degrading the DL model prediction performance. To address these challenges, we present hardware-accelerated real-time radio frequency (RF) analytics and drift-awareness modules for robust DL predictions. We have prototyped the proposed design on a Zynq-7000 System-on-Chip that contains an FPGA and an embedded ARM processor. We have used Xilinx Vivado design suite for synthesis and analysis of the HDL design for the proposed solution. To detect dataset drifts, the proposed solution adopts a distance-based technique on FPGA to quantify in real-time the change between the prediction distribution obtained from DL predictions and data distribution of input streaming samples. Using various performance metrics, we have extensively evaluated the performance of the proposed solution and shown that it can significantly improve the DL model robustness in the presence of dataset drifts.

An optimal algorithm for energy harvesting in optical networks

Optical fiber cables, which make up the majority of the world's telecommunications infrastructure, now carry the vast majority of digital data. However, because to the Internet's exponential expansion and a slew of bandwidth-hungry new services like online gaming, 3D or high-definition television, and cloud computing, this infrastructure will soon run out of capacity. The growing need for faster transmission speeds by clients has led to the development of heterogeneous optical networks. High throughput, portability, and future information interconnects may all be accommodated by such a system. Delivering such high-capacity connections to a mobile client should need heterogeneous optical networks. Different multiple access user technologies have been suggested to meet the needs of a group of future heterogeneous optical network users. In order to address the high energy consumption, poor QoS and low resource utilization in cloud radio access network, this paper proposed a novel algorithm. The main idea is to allocate virtual resources with energy-awareness and hybrid energy supply. Then, the energy arrival and consumption models were established based on different network devices. Simulation results show that the proposed algorithm effectively improved the energy consumption performance and QoS as compared with existing methods.

An Intelligent Network Traffic Prediction Scheme Based on Ensemble Learning of Multi-Layer Perceptron in Complex Networks

At present, the amount of network equipment, servers, and network traffic is increasing exponentially, and the way in which operators allocate and efficiently utilize network resources has attracted considerable attention from traffic forecasting researchers. However, with the advent of the 5G era, network traffic has also shown explosive growth, and network complexity has increased dramatically. Accurately predicting network traffic has become a pressing issue that must be addressed. In this paper, a multilayer perceptron ensemble learning method based on convolutional neural networks (CNN) and gated recurrent units (GRU) spatiotemporal feature extraction (MECG) is proposed for network traffic prediction. First, we extract spatial and temporal features of the data by convolutional neural networks (CNN) and recurrent neural networks (RNN). Then, the extracted temporal features and spatial features are fused into new spatiotemporal features through integrated learning of a multilayer perceptron, and a spatiotemporal prediction model is built in the sequence-to-sequence framework. At the same time, the teacher forcing mechanism and attention mechanism are added to improve the accuracy and convergence speed of the model. Finally, the proposed method is compared with other deep learning models for experiments. The experimental results show that the proposed method not only has apparent advantages in accuracy but also shows some superiority in time training cost.

Multi-Agent Deep Q-Network Based Dynamic Controller Placement for Node Variable Software-Defined Mobile Edge-Cloud Computing Networks

Software-defined networks (SDN) can use the control plane to manage heterogeneous devices efficiently, improve network resource utilization, and optimize Mobile Edge-Cloud Computing Networks (MECCN) network performance through decisions based on global information. However, network traffic in MECCNs can change over time and affect the performance of the SDN control plane. Moreover, the MECCN network may need to temporarily add network access points when the network load is excessive, and it is difficult for the control plane to form effective management of temporary nodes. This paper investigates the dynamic controller placement problem (CPP) in SDN-enabled Mobile Edge-Cloud Computing Networks (SD-MECCN) to enable the control plane to continuously and efficiently serve the network under changing network load and network access points. We consider the deployment of a two-layer structure with a control plane and construct the CPP based on this control plane. Subsequently, we solve this problem based on multi-agent DQN (MADQN), in which multiple agents cooperate to solve CPP and adjust the number of controllers according to the network load. The experimental results show that the proposed dynamic controller deployment algorithm based on MADQN for node-variable networks in this paper can achieve better performance in terms of delay, load difference, and control reliability than the Louvain-based algorithm, single-agent DQN-based algorithm, and MADQN- (without node-variable networks consideration) based algorithm.

Joint Routing and Scheduling With Cyclic Queuing and Forwarding for Time-Sensitive Networks

Various Industrial Internet of Things (IIoT) devices and their compelling applications have witnessed exponential growth over the past few years. Compared to the consumer internet, the industrial IoT often faces tough communication latency boundaries and low packet loss. Recently, time-sensitive networks (TSN) technique has been considered the most promising solution to tackle these issues, providing deterministic communication guarantees for time-sensitive applications. In TSN, to achieve bounded low latency, it introduces the Cyclic Queuing and Forwarding (CQF) mechanism to shape and regulate the transmission selection to provide the protected transmission window for time-triggered (TT) flows. Currently, a considerable amount of literature has explored traffic scheduling with the CQF mechanism in TSN. However, these methods ignore the impact of routing policy on scheduling optimization. In this paper, we design two algorithms to tackle the problem of joint routing and scheduling optimization with the CQF mechanism for TSN. We propose a TT flow sorting algorithm (JDPS) to measure the scheduling priority weight of each TT flow, which uses the normalization method to calculate the weight based on the flow's deadline, period, and size. Then, we propose a joint routing and scheduling optimization with load balance algorithm (JRS-LB) to improve the scheduling success radio and the network resource utilization, which jointly considers the injection time slot offset, scheduling, and routing of TT traffic. The evaluation results mainly focus on the scheduling success ratio, execution time, and resource utilization for the proposed algorithm compared to the Tabu and Naive algorithms. Extensive simulations indicate that the proposed JDPS and JRS-LB algorithms can improve the scheduling success radio compared with the other algorithms.

Online Coordinated NFV Resource Allocation via Novel Machine Learning Techniques

Thanks to Network Function Virtualization (NFV), Internet Service Providers (ISPs) can improve network resource utilization with significantly reduced capital and operational expenditures. To dig deeper into the potential of NFV, an important challenge is the resource allocation problem in NFV (NFV-RA), which can be divided into three stages: VNFs chain composition, VNF forwarding graph embedding, and VNFs scheduling. The key to the NFV-RA problem is to design an effective and coordinated resource allocation algorithm for the three stages. Besides, the NFV-RA problem has been proved to be NP-Hard, and thus most existing approaches focus on heuristic and meta-heuristic algorithms. In this paper, we propose an NFV online coordinated resource allocation framework (OCRA) that completes the three stages simultaneously in a coordinated manner by combining parallel Multi-Agent Deep Reinforcement Learning with novel neural networks and RL training techniques. The extensive experimental results show that compared with the state-of-the-art solutions, OCRA is highly-efficient in terms of time, with up to 50% and 10.8% improvement on resource overhead and acceptance ratio, respectively.