Active Queue Management Algorithm Research Articles

TCP Congestion Management Using Deep Reinforcement Trained Agent for RED

ABSTRACTIncreasing data transmission volumes are causing more frequent and more severe network congestion. In order to handle spikes in network traffic, a substantially bigger buffer has been included into the system. Bufferbloat, which happens when a bigger buffer is implemented, exacerbates network congestion. Using the transfer control protocol (TCP) congestion management strategy with active queue management (AQM) can fix this issue. As congestion increases, it becomes increasingly difficult to forecast and fine‐tune dynamic AQM/TCP systems in order to achieve acceptable performance. To shed new light on the AQM system, we plan to use deep reinforcement learning (DRL) techniques. It is possible that AQM can learn about the appropriate drop policy the same way people do when using a model‐free technique like DRL‐AQM. After training in a simple network scenario, DRL‐AQM is able to recognize complex patterns in the data traffic model and apply them to improve performance in a wide variety of scenarios. Offline training precedes deployment in our approach. In many cases, the model does not require any further parameter tweaks after training. Even in the most complicated networks, AQM algorithms have proven to be effective, regardless of the network's complexity. Minimizing buffer capacity use is an important goal of DRL‐AQM. It automatically and continually adjusts to changes in network connectivity.

TCP BBR-n interplay with modern AQM in Wireless-N/AC networks: Quest for the golden pair.

Effective congestion control on the internet has been a problem since its inception. Transmission Control Protocol (TCP), being the most widely used transport layer protocol tries to mitigate it using a variety of congestion control algorithms. Cubic, Reno, and Bottleneck Bandwidth and Round-trip propagation time (BBR) are the most deployed congestion controls. BBR v2 is leading the congestion control race with its superior performance in terms of better throughput and lower latency. Furthermore, Active Queue Management (AQM) algorithms try to mitigate the congestion control at the network layer through active buffer control to avoid bufferbloat. The most efficient congestion control occurs when TCP and AQM work together. Indeed, it is the TCP-AQM algorithm "Golden pair" that can result in the most efficient performance. This paper proposes such a novel pair based on our previously tested and published BBR-n (BBR new) with the most effective of the modern AQMs, that completely gels together to provide lower latency in wireless networks based on Wireless N/AC. Real-time experiments were performed using Flent on our physical testbed with BBR-n and modern AQMs such as Fair Queuing (FQ), Constrained Delay (CoDel), Proportional Integral controller Enhanced (PIE), Common Applications Kept Enhanced (Cake) and Flow Queuing Controlled Delay (FQ_CoDel). Various tests done on our physical testbed helped us identify CAKE as the most optimum AQM that fits with our proposed BBR-n while providing optimum throughput and lower latency in 802.11N/AC-based wireless networks.

DESiRED — Dynamic, Enhanced, and Smart iRED: A P4-AQM with Deep Reinforcement Learning and In-band Network Telemetry

Active Queue Management (AQM) is a mechanism employed to alleviate transient congestion in network device buffers, such as routers and switches. Traditional AQM algorithms use fixed thresholds, like target delay or queue occupancy, to compute random packet drop probabilities. A very small target delay can increase packet losses and reduce link utilization, while a large target delay may increase queueing delays while lowering drop probability. Due to dynamic network traffic characteristics, where traffic fluctuations can lead to significant queue variations, maintaining a fixed threshold AQM may not suit all applications. Consequently, we explore the question: What is the ideal threshold (target delay) for AQMs? In this work, we introduce DESiRED (Dynamic, Enhanced, and Smart iRED), a P4-based AQM that leverages precise network feedback from In-band Network Telemetry (INT) to feed a Deep Reinforcement Learning (DRL) model. This model dynamically adjusts the target delay based on rewards that maximize application Quality of Service (QoS). We evaluate DESiRED in a realistic P4-based test environment running an MPEG-DASH service. Our findings demonstrate up to a 90x reduction in video stall and a 42x increase in high-resolution video playback quality when the target delay is adjusted dynamically by DESiRED.

Chronology of the development of Active Queue Management algorithms of RED family. Part 1: from 1993 up to 2005

This work is the first part of a large bibliographic review of active queue management algorithms of the Random Early Detection (RED) family, presented in the scientific press from 1993 to 2023. The first part will provide data on algorithms published from 1993 to 2005.

P4SQA: A P4 Switch-Based QoS Assurance Mechanism for SDN

Data center networks include a wide variety of service categories, which leads to a complex network topology. As a result, the QoS of different service flows cannot be guaranteed. In this paper, we propose a QoS assurance mechanism based on P4 switches (P4SQA) to make full use of network resources. P4SQA includes two functions: in-network traffic classification and in-network queue management. To reduce the communication overhead and computational pressure on the Software Defined Network (SDN) controller, we offload part of the neural computation from the control plane to the data plane. A fully connected neural network-based traffic classification method is deployed in the programmable data plane to classify different traffic types. In addition, the optimized active queue management algorithm CoDeL is integrated into the P4 switch (PCoDeL) to prevent network congestion while guaranteeing QoS more effectively. The evaluation results show that traffic classification accuracy has improved, especially in terms of the F1_score, which reached 0.97, significantly better than other state-of-the-art methods. Moreover, compared to traditional queue management algorithms, PCoDeL can improve network throughput and reduce transmission delay, which verifies the effectiveness and superiority of P4SQA in ensuring QoS.

Double Function Random Early Detection (DFRED): A Revised RED-Oriented Algorithm

Dropping packets with a linear function between two configured queue thresholds in Random Early Detection (RED) model is incapable of yielding satisfactory network performance. In this article, a new enhanced and effective active queue management algorithm, termed Double Function RED (DFRED in short) is developed to further curtail network delay. Specifically, DFRED algorithm amends the packet dropping probability approach of RED by dividing it into two sub-segments. The first and second partitions utilizes and implements a quadratic and linear increase respectively in the packet dropping probability computation to distinguish between two traffic loads: low and high. The ns-3 simulation performance evaluations clearly indicate that DFRED algorithm significantly controls the average queue occupancy and yields a reasonable gain in end-to-end-delay under different network conditions.

End-to-end active queue management with Named-Data Networking

The innovative information-based Named-Data Networking (NDN) architecture provides a good opportunity to rethink many of the design decisions that are taken for granted in the Internet today. For example, active queue management (AQM) tasks have been traditionally implemented in the routers to alleviate network congestion before their buffers fill up. However, AQM operations could be performed on an end-to-end basis by taking advantage of NDN features. In this paper, we provide an implementation of an AQM algorithm for the NDN architecture that we use to drive a classical AIMD-based congestion control protocol at the receivers. To accomplish this, we take advantage of the 64-bit Congestion Mark field present in the link layer of NDN packets to encode both rate and delay information about each transmission queue along a network path. In order to make the solution scalable, this information is delivered stochastically, guaranteeing that receivers get accurate and updated information about every pertinent queue. This information is enough to implement the well-known controlled delay (CoDel) AQM algorithm. Simulation results show that our client-located CoDel implementation is able to react to congestion when the bottleneck queuing delay surpasses the 5ms target set by the usual in-network CoDel implementation and, at the same time, get a fair and efficient share of the available transmission capacity.

Deadline-Constrained Multi-Agent Collaborative Transmission for Delay-Sensitive Applications

Delay-sensitive applications impose the strict deadline-constrained latency requirement for end-to-end transmission. However, due to the high dynamics of link status and the limited network resources, it is still challenging to ensure deterministic end-to-end latency for deadline-constrained flows. To this end, we propose a deadline-constrained multi-agent collaborative transmission mechanism (DMCT). Specially, DMCT is featured by combining dynamic routing and packet active dropping control to avoid additional end-to-end latency. Firstly, we design a specific packet structure to detect network status and aware the deadline of each flow. Secondly, to enhance the precision of routing decision, we further formulate deadline-constrained flow scheduling as a joint minimization problem that considers the deadline of flows, queuing delay, and network status. Then, we propose a multi-agent collaborative routing algorithm to find the solution to the joint minimization problem. Finally, we propose a local-agent deadline-constrained active queue management algorithm for packet active dropping control and routing adjustment. Simulation results show that the proposed DMCT outperforms the state-of-the-art solutions, e.g., graph convolutional network-based deep reinforcement learning (GCNL) and delay-based shortest path first (D-SPF), in terms of one-way round-trip time, drop rate, and queue length.

A Sketch-Based Fine-Grained Proportional Integral Queue Management Method

The phenomenon “bufferbloat” occurs when the buffers of the network intermediary nodes fill up, causing long queuing delays. This has a significant negative impact on the quality of service of network applications, particularly those that are sensitive to time delay. Many active queue management (AQM) algorithms have been proposed to overcome this problem. Those AQMs attempt to maintain minimal queuing delays and good throughput by purposefully dropping packets at network intermediary nodes. However, the existing AQM algorithms mostly drop packets randomly based on a certain metric such as queue length or queuing delay, which fails to achieve fine-grained differentiation of data streams. In this paper, we propose a fine-grained sketch-based proportional integral queue management algorithm S-PIE, which uses an additional measurement structure Sketch for packet frequency share judgment based on the existing PIE algorithm for the fine-grained differentiation between data streams and adjust the drop policy for a differentiated packet drop. Experimental results on the NS-3 simulation platform show that the S-PIE algorithm achieves lower average queue length and RTT and higher fairness than PIE, RED, and CoDel algorithms while maintaining a similar throughput performance, maintaining network availability and stability, and improving network quality of service.

A Triplex Region - Random Early Detection (TR-RED) Algorithm for Active Queue Management in Internet Routers

A Triplex Region - Random Early Detection (TR-RED) Algorithm for Active Queue Management in Internet Routers

QoS Implementation with Triple-Metric-Based Active Queue Management for Military Networks

For supporting Quality of Service (QoS) in a military network, applications of the triple-metric priority of performance, importance, and urgency as well as autonomous and lightweight implementation are required. In a previous study, we analyzed a Korean military network’s QoS implementation in the perspective of the triple-metric and presented some improvements in the simplification of the service classes of Differentiated Services (DiffServ). To extend the simplified DiffServ from the previous research, this paper proposes Active Queue Management (AQM) algorithms to process the traffic of each service class differently based on importance and urgency and shows the feasibility through some experiments.

Improved random early detection congestion control algorithm for internet routers

In the internet, router plays a strategic role in the transmission of data packets. Active queue management (AQM) aimed at managing congestion by keeping a reduced average buffer occupancy and hence a minimal delay. The novel random early detection (RED) algorithm suffers from large average buffer occupancy and delay shortcomings. This problem is due in part to the existence of a distinctive linear packet drop function it deploys. In this paper, we present a new version of RED, called improved RED (IMRED). An important strategy of IM-RED is to deploy two dropping functions: i) nonlinear (i.e. quadratic) to deal with both light-and moderatenetwork traffic load conditions, and ii) linear to deal with heavy traffic load condition. Simulation experiments conducted using open-source ns-3 software to evaluate and compare the functionality of the proposed IM-RED with other two previous AQM algorithms confirmed that IM-RED reduces the average buffer occupancy and obtained an improved delay performance especially at heavy network traffic load scenario. Very fortunately, since RED algorithm is known to appear as a built-in model in ns-3 and even Linux kernel, its implementation can therefore be leveraged to obtain IMRED while only adjusting the packet dropping probability profile and holding on to its other attributes.

Random early detection-quadratic linear: an enhanced active queue management algorithm

This paper identifies the lone linear drop function for computing the dropping probability between certain queue threshold values as a major weakness for the random early detection (RED) algorithm as it leads to large delay and queue instability. To address this concern, we propose an enhanced RED-based algorithm called random early detection-quadratic linear (referred to as “RED-QL”) active queue management (AQM) which leveraged the benefit of a quadratic packet drop function for a light-to moderate traffic load conditions together with a linear packet drop function for a heavy traffic load condition respectively. Results from ns-3 network simulator using different experimental scenarios clearly reveals that the proposed RED-QL algorithm yields a substantial reduction in delay performance and indeed a reduced average queue size than other three representative AQM algorithms. RED-QL is robust, easy to implement and deploy in routers (both in software and hardware) as no more than the packet drop probability profile of the classic RED’s algorithm implementation needs modest alteration.

RED-LE: A Revised Algorithm for Active Queue Management

The random early detection (RED) algorithm was developed in 1993. Nearly three decades later, several improved variants have been proposed by scientists. The use of a (pure) linear function for computing packet drop probability has turned out to be a disadvantage, leading to the problem of large delays. Such a problem may be addressed by using linear and non-linear (i.e. as exponential) packet drop probability functions. This paper proposes a revised RED active queue management algorithm named RED-linear exponential (RED-LE). This variant involves an interplay of linear and exponential drop functions, in order to improve the performance of the original RED algorithm. More importantly, at low and moderate network traffic loads, the RED-LE algorithm employs the linear drop action. However, for high traffic loads, RED-LE employs the exponential function for computing the packet drop probability rate. Experimental results have shown that RED-LE effectively controls congestion and offers an improved network performance under different traffic loads.

Applying reinforcement learning for random early detaction algorithm in adaptive queue management systems

Recently, <span>the use of internet has been increased all around the hose, the companies, government departments and the video games and so on. Thus, this increased the traffic used in the networks, which generated congestion issues and sent packet drop in the nodes. To solve this problem, certain algorithms are used. The Active queue management is one of the most important algorithms that helps with this issue. For an effective network management, the RL was used, and it will adapt with the parameters of algorithms. Where the suggested algorithm deep Q-networks (DQN) depends on the reinforcement learning (RL) to reduce the drop and delay. Also, the random early detection (RED) (an active queue management (AQM) algorithm) was adopted based on the NS3 situation.</span>

A survey on TCP enhancements using P4-programmable devices

The increasing performance requirements of today’s Internet applications demand a reliable mechanism to transfer data. Many applications rely on the Transmission Control Protocol (TCP) as the transport protocol, due to its ability to adapt to properties of the network and to be robust in the face of many kinds of failures. However, improving the performance of applications that rely on TCP has been limited by the closed nature of legacy switches, which do not provide accurate visibility of network events. With the emergence of P4-programmable devices, developers can rapidly implement and test customized solutions that use fine-grained telemetry, provide sub round-trip time feedback to end devices to enhance congestion control, precisely isolate traffic to offer better Quality of Service (QoS), quickly detect congestion and re-route traffic via alternate paths, and optimize server resources by offloading protocols.This paper first surveys recent works on P4-programmable devices, focusing on schemes aimed at enhancing TCP performance. It provides a taxonomy classifying the aspects that impact TCP’s behavior, such as congestion control, Active Queue Management (AQM) algorithms, TCP offloading, and network measurement schemes. Then, it compares the P4-based solutions, and contrasts those solutions with legacy implementations. Lastly, the paper presents challenges and future trends.

Adaptive Hurst-Sensitive Active Queue Management.

An Active Queue Management (AQM) mechanism, recommended by the Internet Engineering Task Force (IETF), increases the efficiency of network transmission. An example of this type of algorithm can be the Random Early Detection (RED) algorithm. The behavior of the RED algorithm strictly depends on the correct selection of its parameters. This selection may be performed automatically depending on the network conditions. The mechanisms that adjust their parameters to the network conditions are called the adaptive ones. The example can be the Adaptive RED (ARED) mechanism, which adjusts its parameters taking into consideration the traffic intensity. In our paper, we propose to use an additional traffic parameter to adjust the AQM parameters—degree of self-similarity—expressed using the Hurst parameter. In our study, we propose the modifications of the well-known AQM algorithms: ARED and fractional order and the algorithms based on neural networks that are used to automatically adjust the AQM parameters using the traffic intensity and its degree of self-similarity. We use the Fluid Flow approximation and the discrete event simulation to evaluate the behavior of queues controlled by the proposed adaptive AQM mechanisms and compare the results with those obtained with their basic counterparts. In our experiments, we analyzed the average queue occupancies and packet delays in the communication node. The obtained results show that considering the degree of self-similarity of network traffic in the process of AQM parameters determination enabled us to decrease the average queue occupancy and the number of rejected packets, as well as to reduce the transmission latency.

EVALUATING QOS IN NETWORKS WITH ACTIVE QUEUE MANAGEMENT ALGORITHMS

Packets queuing and scheduling in network routers is a key point of overall network performance. Many applications, especially applications require Quality of Services (QoS), need techniques to pass their packets throughout routers and control and/or avoid congestions in highly congested routes. Therefore, many Active Queue Management (AQM) algorithms have been developed to avoid or control congestion in routers and provide fairness among traffic flows. This paper provides an extensive evaluation performance analysis of three well-known AQM algorithms including RED, REM and traditional Drop-Tail with QoS application requirements. The evaluation performance is conducted by employing network simulator version 2 (NS2). The network performance is measured with Voice over Internet Protocol (VoIP) traffic and three performance metrics including, throughput, latency, and PSD (Probability of Sequential Drop). The analysis shows no AQM algorithm achieve all the VoIP QoS requirements, A new AQM is needed to fulfil QoS requirements and manage queue to handle unresponsive flows.

IRED: An Improved Active Queue Management Algorithm

IRED: An Improved Active Queue Management Algorithm

AQM based on the queue length: A real-network study.

Active Queue Management (AQM) is recommended by Internet Engineering Task Force to mitigate the bufferbloat phenomenon in the Internet. In this paper, we show the results of comprehensive measurements carried out in our university network, in which a device with an AQM algorithm, designed and programmed for this purpose, was running. The implemented AQM algorithm was based on the dropping function, i.e. arriving packets were dropped randomly, with the probability being a function of the queue length. Several different dropping function forms, proposed in the networking literature, were used, in addition to the classic FIFO queue (no AQM). The experiment lasted over a month, during which the state of the network was measured and recorded several thousand times. This made the results independent of the natural fluctuations of the users’ behavior and the network load. Conclusions on the general performance improvement offered by the implemented AQM, as well as the differences in the performance between particular forms of the dropping function, were reached. Some of these conclusions differ from those drawn previously from simulations. This underlines the need for carrying measurements of new AQMs in real, operating networks, with complex, natural traffic.