Virtual Network Function Scheduling Research Articles

Low-latency Virtual Network function Scheduling Algorithm Based on Deep Reinforcement Learning

This paper addresses the problem of mapping, scheduling, and routing of virtual network functions (VNF) on a service function chain (SFC) that is sensitive to latency in a virtual network. A scheduling algorithm for VNF is proposed, which aims to minimize the SFC rejection rate while taking into account VNF mapping, scheduling, and traffic routing during the scheduling process. To achieve this goal, a Markov decision process (MDP)-based VNF scheduling model is established that guarantees SFC resource requirements are met. The model uses the D3QN (Dueling Double DQN) algorithm based on composite rules to select the SFC at each scheduling time point, and selects virtual nodes and routes using a routing optimization algorithm to minimize the SFC rejection rate. We compare our algorithm with the single rule, DQN and genetic algorithm, and the simulation results show that the proposed algorithm can reduce the rejection rate of SFC by approximately 8% compared to genetic algorithms.

AI-Based Mobility-Aware Energy Efficient Resource Allocation and Trajectory Design for NFV Enabled Aerial Networks

In this paper, we propose a novel joint intelligent trajectory design and resource allocation algorithm based on users’ mobility and their requested services for unmanned aerial vehicles (UAVs) assisted networks, where UAVs act as nodes of a network function virtualization (NFV) enabled network. Our objective is to maximize energy efficiency and minimize the average delay on all services by allocating the limited radio and NFV resources. In addition, due to the traffic conditions and mobility of users, we let some virtual network functions (VNFs) migrate from their current locations to other locations to satisfy the Quality of Service requirements. We formulate our problem to find near-optimal locations of UAVs, transmit power, subcarrier assignment, VNF placement, and VNF scheduling over the UAVs and perform suitable VNF migration. Then we propose a novel hierarchical hybrid continuous and discrete action (HHCDA) deep reinforcement learning method to solve the proposed problem. Finally, the convergence and computational complexity of the proposed algorithm and its performance is analyzed for different parameters. Simulation results show that our proposed HHCDA method decreases the average end-to-end delay by 31.5% and increases the energy efficiency by 40% compared to the state-of-the-art Deep Deterministic Policy Gradient method.

Equilibrated and Fast Resources Allocation for Massive and Diversified MTC Services Using Multiagent Deep Reinforcement Learning

Massive and diversified machine type communication (MTC) service is one of the development trends of MTC in Internet of Things (IoT). Meanwhile, realizing network functions virtualization (NFV) is inseparable from reasonable virtual network function (VNF) scheduling and resource allocation. For VNF scheduling and resource allocation of MTC services, recently, deep reinforcement learning (DRL) has become one of the feasible solutions. However, existing DRL solutions have problems of inapplicability to the environment with both discrete and continuous variables, long training, time and nonequilibrium resource allocation. In this article, we first model the end-to-end (E2E) VNF scheduling and resource allocation of core network nodes, links, and access network subcarriers with different strategies, respectively, and propose a compound variable optimization problem aiming at maximizing the net income of the network provider. Then, we propose the mapping scheme of the absolute value of the signum function (ASgn mapping scheme) to simplify the compound variables into continuous variables of the optimization problem, so that the DRL algorithm is applicable. Moreover, we propose a model paralleling multiagent twin delayed deep deterministic (MPMA-TD3) policy gradient algorithm to handle massive services, reduce training time, and action space of agents. Finally, we improve the MPMA-TD3 algorithm to handle diversified services, solve the problem of nonequilibrium resources allocation, and realize the reasonable resource allocation for each service. Simulation results show that the proposed algorithms are better than other algorithms in reward, delay, cost, and training time for massive services. Further, the Improved MPMA-TD3 algorithm has the best service equilibrating ability.

Supporting 5G service orchestration with formal verification

The 5G communication technology has the ability to create logical networks, called network slices, which are specifically carved to serve particular application domains. Due to the mix of different application criticality, it becomes crucial to verify if the applications? service level agreements are met. In this paper, we propose a novel framework for modeling and verifying 5G orchestration, considering simultaneous access and admission of new requests to slices as well as virtual network function scheduling and routing. By combining modeling in userfriendly UML, with UPPAAL model checking and satisfiability-modulo-theories based model finding, our framework supports both modeling and formal verification of service orchestration. We demonstrate our approach on a e-health case study showing how a user, with no knowledge of formal methods, can model a system in UML and verify that the application meets its requirements.

KDN-Based Fault-Tolerant Scheduling for VNFs in Data Centers

Network Functions Virtualization (NFV) aims to decouple network functionality from dedicated physical devices, thereby allowing Virtual Machines (VMs) to host services such as firewalls and load balancers. Hosting these services on VMs will provide new business opportunities for data centers. Currently, the available physical resources are limited, thus making efficient resource scheduling the most challenging issue towards the successful adoption of NFV. Therefore, optimizing resource allocation is crucial. This paper proposes an integrated, intelligent framework for VNF (Virtual Network Function) scheduling. In particular, the proposed solution is based on Knowledge Defined Networking (KDN) with Long Short-Term Memory (LSTM) to operate and optimize VNF scheduling. For this reason, our solution considers VNF affinity rules in a data center with LSTM- based multi-step ahead time forecast. Experimental analysis through benchmark data shows that the proposed fault-tolerant aware scheduling framework based on LSTM and affinity techniques achieves promising results.

Service Function Chaining in MEC: A Mean-Field Game and Reinforcement Learning Approach

Multiaccess edge computing (MEC) and network virtualization technologies are important enablers for fifth generation networks to deliver diverse services. Services are often provided as fully connected virtual network functions (VNFs), through service function chaining (SFC). However, the problem of allocating SFC resources at the edge still faces many challenges related to the way VNFs are placed, chained, and scheduled. In this article, to solve these problems, we propose a game theory-based approach with the objective to reduce service latency in the context of SFC at the edge. The problem of allocating SFC resources can be divided into two subproblems: 1) the VNF placement and routing subproblem, and 2) the VNF scheduling subproblem. For the former subproblem, we formulate it as a mean-field game in which VNFs are contending over edge resources aiming at reducing the resource consumption of MEC nodes and reducing latency for users. We also propose a reinforcement learning-based technique, where the Ishikawa--Mann learning algorithm is used. For the later subproblem, we formulate it as a matching game between VFNs and edge resources to find the execution order of the VNFs while reducing the latency. To efficiently solve it, we propose a modified version of the many-to-one deferred acceptance algorithm (DAA), called the enhanced multistep DAA. To illustrate the performance of the proposed approaches, we perform extensive simulations. The results show that the approaches achieve up to 40% less resource consumption, and up to 38% less latency than the benchmarked state-of-the-art methods.

Age of Information Aware VNF Scheduling in Industrial IoT Using Deep Reinforcement Learning

In delay-sensitive industrial Internet of Things (IIoT) applications, the age of information (AoI) is employed to characterize the freshness of information. Meanwhile, the emerging network function virtualization provides flexibility and agility for service providers to deliver a given network service using a sequence of virtual network functions (VNFs). However, suitable VNF placement and scheduling in these schemes is NP-hard and finding a globally optimal solution by traditional approaches is complex. Recently, deep reinforcement learning (DRL) has appeared as a viable way to solve such problems. In this paper, we first utilize single agent low-complex compound action actor-critic RL to cover both discrete and continuous actions and jointly minimize VNF cost and AoI in terms of network resources under end-to-end Quality of Service constraints. To surmount the single-agent capacity limitation for learning, we then extend our solution to a multi-agent DRL scheme in which agents collaborate with each other. Simulation results demonstrate that single-agent schemes significantly outperform the greedy algorithm in terms of average network cost and AoI. Moreover, multi-agent solution decreases the average cost by dividing the tasks between the agents. However, it needs more iterations to be learned due to the requirement on the agents' collaboration.

Delay-Aware VNF Scheduling: A Reinforcement Learning Approach With Variable Action Set

Software defined networking (SDN) and network function virtualization (NFV) are the key enabling technologies for service customization in next generation networks to support various applications. In such a circumstance, virtual network function (VNF) scheduling plays an essential role in enhancing resource utilization and achieving better quality-of-service (QoS). In this paper, the VNF scheduling problem is investigated to minimize the makespan (i.e., overall completion time) of all services, while satisfying their different end-to-end (E2E) delay requirements. The problem is formulated as a mixed integer linear program (MILP) which is NP-hard with exponentially increasing computational complexity as the network size expands. To solve the MILP with high efficiency and accuracy, the original problem is reformulated as a Markov decision process (MDP) problem with variable action set. Then, a reinforcement learning (RL) algorithm is developed to learn the best scheduling policy by continuously interacting with the network environment. The proposed learning algorithm determines the variable action set at each decision-making state and captures different execution time of the actions. The reward function in the proposed algorithm is carefully designed to realize delay-aware VNF scheduling. Simulation results are presented to demonstrate the convergence and high accuracy of the proposed approach against other benchmark algorithms.

Virtual network function scheduling via multilayer encoding genetic algorithm with distributed bandwidth allocation

Network function virtualization represents a revolutionary approach to network service deployment. This software-oriented approach for virtual network functions (VNFs) deployment enables more flexible and dynamic network services to meet diversified demands. To minimize the execution time of all VNFs in service function chains, VNF scheduling must be addressed. In this paper, we improve upon the flexible job-shop model by introducing the process of bandwidth allocation. First, we propose a multilayer encoding genetic algorithm to solve the VNF scheduling model. In addition, we design a distributed method for bandwidth allocation based on the Nash bargaining solution. Finally, by combining the genetic algorithm with distributed bandwidth allocation, we present a heuristic algorithm that solves the VNF scheduling problem in one stage. Using a multilayer encoding genetic algorithm, we simplify the constraints of the VNF scheduling problem and reduce its time complexity. At the same time, our Nash game solution refines the granularity of bandwidth allocation to further reduce the transmission delay between VNFs. The effectiveness of our proposed heuristic algorithm is verified through numerical evaluation. Compared with existing approaches, our method exhibits shorter scheduling time and reduces CPU time by 45% in simulated scenarios.

Virtual Network Function Scheduling: A Matching Game Approach

Network function virtualization is a promising technique for telecom providers to efficiently manage network services at low cost. However, existing works mainly focus on resource allocation and thus leave behind an important issue: the virtual network function (VNF) scheduling. Current approaches, e.g., round-robin scheduling or heuristic algorithms, still expose some unsolved issues, such as high computational cost and inability to perform online scheduling. In this letter, we propose a matching-based algorithm to solve the NP-hard VNF scheduling problem. This approach can guarantee a stable scheduling, in which all network services are satisfied with the assignment. Finally, the effectiveness of our method is verified through numerical evaluation, showing that our approach can increase the number of completed VNFs by 36.8% compared with the current round-robin method.

A Generalized VNF Sharing Approach for Service Scheduling

Network function virtualization enables a flexible service provision by introducing the concept of virtual network function (VNF). Despite the importance of VNF scheduling, it is largely unexplored. In this letter, we propose a new model for VNF scheduling based on the min-plus algebra theory. The main idea of this model is to share the deployed VNF instances among different services, thus to improve the resource utilization and reduce the resource fragmentation generated by deploying many VNF instances. In addition, a weight-based VNF sharing scheduling approach is proposed in this model to achieve a fair scheduling among the arriving service requests. The experimental results show that the proposed model and approach are effective and efficient.

Delay-Aware Scheduling and Resource Optimization With Network Function Virtualization

To accelerate the implementation of network functions/middle boxes and reduce the deployment cost, recently, the concept of network function virtualization (NFV) has emerged and become a topic of much interest attracting the attention of researchers from both industry and academia. Unlike the traditional implementation of network functions, a software-oriented approach for virtual network functions (VNFs) creates more flexible and dynamic network services to meet a more diversified demand. Software-oriented network functions bring along a series of research challenges, such as VNF management and orchestration, service chaining, VNF scheduling for low latency and efficient virtual network resource allocation with NFV infrastructure, among others. In this paper, we study the VNF scheduling problem and the corresponding resource optimization solutions. Here, the VNF scheduling problem is defined as a series of scheduling decisions for network services on network functions and activating the various VNFs to process the arriving traffic. We consider VNF transmission and processing delays and formulate the joint problem of VNF scheduling and traffic steering as a mixed integer linear program. Our objective is to minimize the makespan/latency of the overall VNFs’ schedule. Reducing the scheduling latency enables cloud operators to service (and admit) more customers, and cater to services with stringent delay requirements, thereby increasing operators’ revenues. Owing to the complexity of the problem, we develop a genetic algorithm-based method for solving the problem efficiently. Finally, the effectiveness of our heuristic algorithm is verified through numerical evaluation. We show that dynamically adjusting the bandwidths on virtual links connecting virtual machines, hosting the network functions, reduces the schedule makespan by 15%–20% in the simulated scenarios.

Joint Optimization of Service Function Chaining and Resource Allocation in Network Function Virtualization

Network function virtualization (NFV) has already been a new paradigm for network architectures. By migrating NFs from dedicated hardware to virtualization platform, NFV can effectively improve the flexibility to deploy and manage service function chains (SFCs). However, resource allocation for requested SFC in NFV-based infrastructures is not trivial as it mainly consists of three phases: virtual network functions (VNFs) chain composition, VNFs forwarding graph embedding, and VNFs scheduling. The decision of these three phases can be mutually dependent, which also makes it a tough task. Therefore, a coordinated approach is studied in this paper to jointly optimize NFV resource allocation in these three phases. We apply a general cost model to consider both network costs and service performance. The coordinate NFV-RA is formulated as a mixed-integer linear programming, and a heuristic-based algorithm (JoraNFV) is proposed to get the near optimal solution. To make the coordinated NFV-RA more tractable, JoraNFV is divided into two sub-algorithms, one-hop optimal traffic scheduling and a multi-path greedy algorithm for VNF chain composition and VNF forwarding graph embedding. Last, extensive simulations are performed to evaluate the performance of JoraNFV, and results have shown that JoraNFV can get a solution within 1.25 times of the optimal solution with reasonable execution time, which indicates that JoraNFV can be used for online NFV planning.