Routing In Software-defined Networks Research Articles

Delay-Aware Routing in Software-Defined Networks via Network Tomography and Reinforcement Learning

Delay-Aware Routing in Software-Defined Networks via Network Tomography and Reinforcement Learning

Reinforcement learning-based SDN routing scheme empowered by causality detection and GNN.

In recent years, with the rapid development of network applications and the increasing demand for high-quality network service, quality-of-service (QoS) routing has emerged as a critical network technology. The application of machine learning techniques, particularly reinforcement learning and graph neural network, has garnered significant attention in addressing this problem. However, existing reinforcement learning methods lack research on the causal impact of agent actions on the interactive environment, and graph neural network fail to effectively represent link features, which are pivotal for routing optimization. Therefore, this study quantifies the causal influence between the intelligent agent and the interactive environment based on causal inference techniques, aiming to guide the intelligent agent in improving the efficiency of exploring the action space. Simultaneously, graph neural network is employed to embed node and link features, and a reward function is designed that comprehensively considers network performance metrics and causality relevance. A centralized reinforcement learning method is proposed to effectively achieve QoS-aware routing in Software-Defined Networking (SDN). Finally, experiments are conducted in a network simulation environment, and metrics such as packet loss, delay, and throughput all outperform the baseline.

Dynamic routing optimization in software-defined networking based on a metaheuristic algorithm

Optimizing resource allocation and routing to satisfy service needs is paramount in large-scale networks. Software-defined networking (SDN) is a new network paradigm that decouples forwarding and control, enabling dynamic management and configuration through programming, which provides the possibility for deploying intelligent control algorithms (such as deep reinforcement learning algorithms) to solve network routing optimization problems in the network. Although these intelligent-based network routing optimization schemes can capture network state characteristics, they are prone to falling into local optima, resulting in poor convergence performance. In order to address this issue, this paper proposes an African Vulture Routing Optimization (AVRO) algorithm for achieving SDN routing optimization. AVRO is based on the African Vulture Optimization Algorithm (AVOA), a population-based metaheuristic intelligent optimization algorithm with global optimization ability and fast convergence speed advantages. First, we improve the population initialization method of the AVOA algorithm according to the characteristics of the network routing problem to enhance the algorithm’s perception capability towards network topology. Subsequently, we add an optimization phase to strengthen the development of the AVOA algorithm and achieve stable convergence effects. Finally, we model the network environment, define the network optimization objective, and perform comparative experiments with the baseline algorithms. The experimental results demonstrate that the routing algorithm has better network awareness, with a performance improvement of 16.9% compared to deep reinforcement learning algorithms and 71.8% compared to traditional routing schemes.

A new intelligent cross-domain routing method in SDN based on a proposed multiagent reinforcement learning algorithm

PurposeA cross-domain intelligent software-defined network (SDN) routing method based on a proposed multiagent deep reinforcement learning (MDRL) method is developed.Design/methodology/approachFirst, the network is divided into multiple subdomains managed by multiple local controllers, and the state information of each subdomain is flexibly obtained by the designed SDN multithreaded network measurement mechanism. Then, a cooperative communication module is designed to realize message transmission and message synchronization between the root and local controllers, and socket technology is used to ensure the reliability and stability of message transmission between multiple controllers to acquire global network state information in real time. Finally, after the optimal intradomain and interdomain routing paths are adaptively generated by the agents in the root and local controllers, a network traffic state prediction mechanism is designed to improve awareness of the cross-domain intelligent routing method and enable the generation of the optimal routing paths in the global network in real time.FindingsExperimental results show that the proposed cross-domain intelligent routing method can significantly improve the network throughput and reduce the network delay and packet loss rate compared to those of the Dijkstra and open shortest path first (OSPF) routing methods.Originality/valueMessage transmission and message synchronization for multicontroller interdomain routing in SDN have long adaptation times and slow convergence speeds, coupled with the shortcomings of traditional interdomain routing methods, such as cumbersome configuration and inflexible acquisition of network state information. These drawbacks make it difficult to obtain global state information about the network, and the optimal routing decision cannot be made in real time, affecting network performance. This paper proposes a cross-domain intelligent SDN routing method based on a proposed MDRL method. First, the network is divided into multiple subdomains managed by multiple local controllers, and the state information of each subdomain is flexibly obtained by the designed SDN multithreaded network measurement mechanism. Then, a cooperative communication module is designed to realize message transmission and message synchronization between root and local controllers, and socket technology is used to ensure the reliability and stability of message transmission between multiple controllers to realize the real-time acquisition of global network state information. Finally, after the optimal intradomain and interdomain routing paths are adaptively generated by the agents in the root and local controllers, a prediction mechanism for the network traffic state is designed to improve awareness of the cross-domain intelligent routing method and enable the generation of the optimal routing paths in the global network in real time. Experimental results show that the proposed cross-domain intelligent routing method can significantly improve the network throughput and reduce the network delay and packet loss rate compared to those of the Dijkstra and OSPF routing methods.

Neural Network Multipath Routing in Software Defined Networks Based on Genetic Algorithm

Currently, a wide demand for the implementation and use of various cloud solutions is a modern trend and the driving force behind the development of network technologies. The growth of cloud application services delivered through data centers with varying network traffic needs demonstrates the limitations of traditional routing and load balancing methods. The combination of the advantages of software defined networks (SDN) technology and artificial intelligence (AI) methods ensures efficient management and operation of computer network resources. The paper proposes an approach to neural network multipath routing in SDN based on a genetic algorithm. The architecture and model of an artificial neural network has been developed to solve the problem of multipath routing in the SDN, which is able to predict the shortest paths based on the metrics of communication links. To optimize the hyperparameters of the neural network model, it is proposed to use a modified genetic algorithm. A visual software system SDNLoadBalancer has been developed and an experimental SDN topology has been designed, which makes it possible to study in detail the processes of neural network multipath routing in SDN based on the proposed approach. The obtained results show that the proposed neural network model has the ability to predict routes with high accuracy in real time, which makes it possible to implement various load balancing schemes in order to increase performance of SDN.

Multi-Path Routing Algorithm Based on Deep Reinforcement Learning for SDN

Software-Defined Networking (SDN) enhances network control but faces Distributed Denial of Service (DDoS) attacks due to centralized control and flow-table constraints in network devices. To overcome this limitation, we introduce a multi-path routing algorithm for SDN called Trust-Based Proximal Policy Optimization (TBPPO). TBPPO incorporates a Kullback–Leibler divergence (KL divergence) trust value and a node diversity mechanism as the security assessment criterion, aiming to mitigate issues such as network fluctuations, low robustness, and congestion, with a particular emphasis on countering DDoS attacks. To avoid routing loops, differently from conventional ‘Next Hop’ routing decision methodology, we implemented an enhanced Depth-First Search (DFS) approach involving the pre-computation of path sets, from which we select the best path. To optimize the routing efficiency, we introduced an improved Proximal Policy Optimization (PPO) algorithm based on deep reinforcement learning. This enhanced PPO algorithm focuses on optimizing multi-path routing, considering security, network delay, and variations in multi-path delays. The TBPPO outperforms traditional methods in the Germany-50 evaluation, reducing average delay by 20%, cutting delay variation by 50%, and leading in trust value by 0.5, improving security and routing efficiency in SDN. TBPPO provides a practical and effective solution to enhance SDN security and routing efficiency.

Traffic-aware optimal routing in software defined networks by predicting traffic using neural network

Network infrastructure management has been completely transformed by Software-Defined Networking (SDN), allowing for centralized control and programmability. The significant challenge in SDN is to provide optimal routing decisions based on real-time network performance metrics. In this work, it is proposed to have Gated Recurrent Unit (GRU) based traffic prediction to dynamically adjust link weights in SDN. It facilitates real-time adaptation to traffic changes and optimal routing decisions for improved Quality of Service (QoS). This work is to provide optimal routing in SDN by leveraging the capability of GRU to predict future traffic based on network performance metrics sampled at different time sequences. It enables the network to adapt the changing traffic patterns in real-time. The predicted traffic is then used to determine the edge weights between links in the network, which are updated dynamically to reflect changes in the network. When an outbound packet is received, it can be routed optimally by selecting a path with less traffic, thereby reducing latency and improving the QoS of routing the packets. The simulation results reveals that the proposed methodology has the potential to reduce delay up to 9.16% and jitter up to 32.31%, coupled with significant throughput enhancements up to 26.81% compared to the default shortest path. These quantitative findings highlight its effective contribution towards optimizing network performance and QoS.

A reinforcement learning approach for widest path routing in software-defined networks

In this paper, a routing method based on reinforcement learning (RL) under software-defined networks (SDN), namely the Q-learning widest-path routing algorithm (Q-WPRA), is proposed. This algorithm processes the reward function according to the link bandwidth in the execution environment to find the optimal (i.e., widest) transmission path with the maximum bandwidth between the source and the destination through RL. The experimental results reveal that the Q-WPRA is outperformance than Dijkstra’s algorithm and Dijkstra’s widest-path algorithm to find the widest transmission path in SDN environment under different bandwidths, loss rates, and background traffic.

A Graph reinforcement learning based SDN routing path selection for optimizing long-term revenue

Software-Defined Network (SDN) paradigm decouples control plane from data plane and provides a logically-centralized control to whole underlying network, which enables the controller to centrally configure the network, and has demonstrated great potential in fully utilizing network capacity. However, the traditional SDN does not consider the accumulative long-term revenue in routing configuration for network service provider (SP). For example, it is prone to choosing the path passing through important nodes, which may increase the possibility of future congestion, and hamper the SP’s long-term revenue. Deep reinforcement learning (DRL) has brought about the transformative advances in decision-making when facing uncertainty. As a well-known type of DRL, Deep Q-network (DQN) can explore optimal routing schemes from high dimensional network state. However, most of existing DRL based methods fail to provide satisfactory configuration when confronted with a topology unseen during training. Meanwhile, many existing network optimization schemes ignore the impact of nodes and global network attributes on the routing configuration. Consequently, they may perform poorly in real scenarios. In this paper, we propose a graph reinforcement learning (GRL) based SDN routing selection scheme, GN-DQN for optimizing SP’s accumulative long-term revenue. First, besides the link embeddings, GN-DQN explicitly learns the representations of the routers and globe network topology with full graph neural network block. Then, through utilizing the formed network representation as the state, DQN is used to learn to sequentially choose the optimal routing path from the candidate path set to maximize the accumulative long-term revenue. Thorough experiments on multiple real network topologies show that our proposed GN-DQN outperforms other GRL based and traditional routing selection schemes, and can generalize over arbitrary topologies. Moreover, GN-DQN also demonstrates great robustness to the link failure scenario.

OPTIMIZATION OF ROUTING USING TRAFFIC CLASSIFICATION IN SOFTWARE DEFINED NETWORKING

Efficient routing is an essential task for any network as it directly impacts the network’s performance. In this paper, we have used traffic classification techniques to optimize routing in Software Defined Network (SDN) and provided a cost aware routing framework. Instead of classifying traffic based on a particular parameter, we perform the traffic classification using a hybrid approach that is based on machine learning techniques. Our framework uses supervised machine learning techniques to classify network flow and detect elephant flows which are too heavy in size and require significant bandwidth. As the requirements of each elephant flow vary with the application, we further perform Quality of Service (QoS) based traffic classification, which classifies these elephant flow into QoS classes as per their QoS requirements. For this purpose, we have used semi-supervised machine learning algorithms. Further, we have proposed a routing algorithm that makes use of the Dijkstra algorithm to compute the best possible shortest path based on the QoS requirements of the elephant flow. The proposed method was implemented and tested on Mininet using a POX controller. The simulation results show that our framework successfully classifies elephant flows with an accuracy of 80% and also computes a low-cost path for each elephant flow based on the actual internet data set.

A fuzzy delay-bandwidth guaranteed routing algorithm for video conferencing services over SDN networks.

Video conferencing is one of the advanced technologies for users that allows online communication despite long distances. High quality communication and ongoing support for the principles of video conferencing service that can be achieved through Software-Defined Networking (SDN). SDN is a new architecture for computer networks that separates the control plane from the data plane to improve network resources and reduce operating costs. All routing decisions and control mechanisms are made by a device called a controller. Traffic engineering can be well implemented in SDN because the entire network topology is known to the controller. Considering SDN features, user requests can be dynamically routed according to current network status and Quality of Service (QoS) requirements. In general, the purpose of SDN routing algorithms is to maximize the acceptance rate of user requests by considering QoS requirements. In this literature, most routing studies to provide satisfactory video conferencing services have focused solely on bandwidth. Nevertheless, some studies have considered both delay and bandwidth constraints. In this paper, a Fuzzy Delay-Bandwidth Guaranteed Routing (FDBGR) algorithm is proposed that considers both delay and bandwidth constraints in routing. The proposed fuzzy system is based on rules that can postpone requests with high resource demands. Also, the purpose of the FDBGR is to distribute the network workload evenly for all requests, where this is done by maintaining the capacity to accept future requests. The combination of conventional routing algorithms and SDN provides remarkable improvements in mobility, scalability and the overall performance of the networks. Simulations are performed on different scenarios to evaluate the performance of the FDBGR compared to state-of-the-art methods. Besides, FDBGR has been compared with a number of most related previous works such as H-MCOP, MH-MCOP, QoMRA, QROUTE and REDO based on criteria such as number of accepted requests, average path length, energy consumption, load balancing, and average delay. The simulation results clearly prove the superiority of the proposed algorithm with an average delay of 48ms in different topologies for video conferencing applications.

Artificial Intelligence Based Smart Routing in Software Defined Networks

In a non-static information exchange network, routing is an overly complex task to perform, which has to satisfy all the needs of the network. Software Defined Network (SDN) is the latest and widely used technology in the future communication networks, which would provide smart routing that is visible universally. The various features of routing are supported by the information centric network, which minimizes the congestion in the dataflow in a network and provides the content awareness through its mined mastery. Due to the advantages of the information centric network, the concepts of the information-centric network has been used in the paper to enable an optimal routing in the software-defined networks. Although there are many advantages in the information-centric network, there are some disadvantages due to the non-static communication properties, which affects the routing in SDN. In this regard, artificial intelligence methodology has been used in the proposed approach to solve these difficulties. A detailed analysis has been conducted to map the content awareness with deep learning and deep reinforcement learning with routing. The novel aligned internet investigation technique has been proposed to process the deep reinforcement learning. The performance evaluation of the proposed systems has been conducted among various existing approaches and results in optimal load balancing, usage of the bandwidth, and maximization in the throughput of the network.

DRSIR: A Deep Reinforcement Learning Approach for Routing in Software-Defined Networking

Traditional routing protocols employ limited information to make routing decisions, which leads to slow adaptation to traffic variability and restricted support to the quality of service requirements of applications. To address these shortcomings, in previous work, we proposed RSIR, a routing solution based on Reinforcement Learning (RL) in Software-Defined Networking (SDN). However, RL-based solutions usually suffer an increase in time during the learning process when dealing with large action and state spaces. This paper introduces a different routing approach, called Deep Reinforcement Learning and Software-Defined Networking Intelligent Routing (DRSIR). DRSIR defines a routing algorithm based on Deep RL (DRL) in SDN that overcomes the limitations of RL-based solutions. DRSIR considers path-state metrics to produce proactive, efficient, and intelligent routing that adapts to dynamic traffic changes. DRSIR was evaluated by emulation using real and synthetic traffic matrices. The results show that this solution outperforms the routing algorithms based on Dijkstra’s algorithm and RSIR in relation to stretch, packet loss, and delay. Moreover, the results obtained demonstrate that DRSIR provides a practical and feasible solution for routing in SDN.

Intelligent Multipath Routing in Software Defined Networks Based on Ant Colony Optimization Algorithms

The software defined networks (SDN) are a modern trend in building of new generation computer networks. The SDN provide flexibility and control in management of data flows in the network, as well as requirement level of quality of service for applications. An approach to intelligent multipath routing in SDN based on ant colony optimization algorithms is proposed in this paper. Research and analysis of two types of ant colony optimization algorithms was carried out: ant system (AS) and ant colony system (ACS). The visual software system SDNLoadBalancer was developed and the experimental SDN topology was designed. They make it possible to research in detail the processes of multipath routing in SDN based on the proposed approach. Also comparative analysis of the proposed approach, results of the Yen algorithm and the genetic algorithm was carried out. The results of experimental research show the effectiveness of the use of ant colony optimization algorithms in solving the problem of multipath routing in SDN, allow to obtain results close to optimal in an acceptable time, and also reduce the transmission delay jitter in the network.

Intelligent Multipath Routing in Software Defined Networks Based on Artificial Bee Colony Algorithm

At present a new generation network paradigm — software defined networks (SDN) has gained wide popularity to ensure the quality of service in computer networks. SDNs provide flexibility in managing data flows through a centralized view of the entire network and the ability to program network services. To ensure the quality of service of multimedia content and video traffic, the most effective mechanism is the use of multipath routing methods. An approach to intelligent multipath routing in the SDN based on the artificial bee colony algorithm is proposed in the paper. The conducted research shows that the application of the proposed approach makes it possible to increase the efficiency of multipath routing processes and the accuracy of determining routes in the SDN, as well as to reduce the transmission delay jitter in the network.

Maintaining Control Resiliency and Flow Programmability in Software-Defined WANs During Controller Failures

Providing resilient network control is a critical concern for deploying Software-Defined Networking (SDN) into Wide-Area Networks (WANs). For performance reasons, a Software-Defined WAN is divided into multiple domains controlled by multiple controllers with a logically centralized view. Under controller failures, we need to remap the control of offline switches from failed controllers to other active controllers. Existing solutions have three limitations: (1) the least flow programmability (e.g., the ability to change paths of flows) cannot be maintained; (2) active controllers could be overloaded, interrupting their normal operations; (3) network performance could be degraded because of the increasing controller-switch communication overhead. In this paper, we propose RetroFlow+ to recover the flow programmability and achieve low communication overhead during controller failures. By intelligently configuring a set of selected offline switches working under the legacy routing mode and several active controllers releasing a few control resources, RetroFlow+ enables active controllers to use the minimum control resource to sustain the flow programmability. RetroFlow+ also smartly transfers the control of offline switches with the SDN routing mode to active controllers to minimize the communication overhead from these offline switches to the active controllers. Simulation results show that RetroFlow+ realizes low communication overhead, recovers all offline flows under one and two controller failures, and improves the flow recovery percentage up to 70% under three controller failures, compared with the state-of-the-art solution.

Enabling Scalable Routing in Software-Defined Networks With Deep Reinforcement Learning on Critical Nodes

Traditional routing schemes usually use fixed models for routing policies and thus are not good at handling complicated and dynamic traffic, leading to performance degradation (e.g., poor quality of service). Emerging Deep Reinforcement Learning (DRL) coupled with Software-Defined Networking (SDN) provides new opportunities to improve network performance with automatic traffic analysis and policy generation. However, existing DRL-based routing solutions usually rely on all node information to make routing decisions for the network and hence are both hard to converge in large networks and vulnerable to topology changes. In this paper, we propose ScaleDeep, a scalable DRL-based routing scheme for SDN, which improves the routing performance and is resilient to topology changes. Essentially, ScaleDeep takes advantage of partial control on network nodes and DRL. We select a set of critical nodes from a network as driver nodes, which can simulate the entire network operation, based on the control theory. By observing the traffic variation on the driver nodes, DRL dynamically adjusts some link weights for a weighted shortest path algorithm to change the routing paths and improve the routing performance. Limiting the control on driver nodes improves the convergence ability of DRL and reduces the dependency of the DRL agent on the fixed network topology. To validate the performance of ScaleDeep, we conduct packet-level simulations on different topologies. The results show that ScaleDeep outperforms existing DRL-based schemes by reducing the average flow completion time by up to 36% and exhibiting better robustness against minor topology changes.

RLMR: Reinforcement Learning Based Multipath Routing for SDN

In recent years, as a new subject in the computer field, artificial intelligence has developed rapidly, especially in reinforcement learning (RL) and deep reinforcement learning. Combined with the characteristics of Software Defined Network (SDN) for centralized control and scheduling, resource scheduling based on artificial intelligence becomes possible. However, the current SDN routing algorithm has the problem of low link utilization and is unable to update and adjust according to the real-time network status. This paper aims to address these problems by proposing a reinforcement learning-based multipath routing for SDN (RLMR) scheme. RLMR uses Markov Decision Process (MDP) and Q-Learning for training. Based on the real-time information of network state and flow characteristics, RLMR performs routing for different flows. When there is no link that meets the bandwidth requirements, the remaining flows are redistributed according to the Quality of Service (QoS) priority to complete the multipath routing. In addition, this paper defines the forward efficiency (FE) to measure the link bandwidth utilization (LBU) under multipath routing. Simulation results show that compared with the current mainstream shortest path algorithm and ECMP algorithm, the routing algorithm in RLMR has advantages in FE, jitter, and packet loss rate. It can effectively improve the efficiency and quality of routing.

Reinforcement Learning-Based Service-Oriented Dynamic Multipath Routing in SDN

The increasing quality and various requirements of network services are guaranteed because of the advancement of the emerging network paradigm, software-defined networking (SDN), and benefits from the centralized and software-defined architecture. The SDN not only facilitates the configuration of the network policies for traffic engineering but also brings convenience for network state obtainment. The traffic of numerous services is transmitted within a network, whereas each service may demand different network metrics, such as low latency or low packet loss rate. Corresponding quality of service policies must be enforced to meet the requirements of different services, and the balance of link utilization is also indispensable. In this research, Reinforcement Discrete Learning-Based Service-Oriented Multipath Routing (RED-STAR) has been proposed to understand the policy of distributing an optimal path for each service. The RED-STAR takes the network state and service type as input values to dynamically select the path a service must be forwarded. Custom protocols are designed for network state obtainment, and a deep learning-based traffic classification model is also integrated to identify network services. With the differentiated reward scheme for every service type, the reinforcement learning model in RED-STAR gradually achieves high reward values in various scenarios. The experimental results show that RED-STAR can adopt the dynamic network environment, obtaining the highest average reward value of 1.8579 and the lowest average maximum bandwidth utilization of 0.3601 among all path distribution schemes in a real-case scenario.

Experimental Evaluation of Algorithms for Packet Routing in Software Defined Network

The packet routing problem in a software defined network is formulated and a greedy algorithm is proposed. An alternative approach to approximately solve this problem by means of reduction to a special case of the fractional length-bounded maximum multicommodity flow is suggested. Experimental comparison of the greedy algorithm and a fully polynomial-time approximation scheme (FPTAS) for fractional length-bounded maximum multicommodity flow is carried out on the testing instances representing prospective software defined satellite network.