Explicit Congestion Notification Marking Research Articles

Micro-Burst Aware ECN in Multi-Queue Data Centers: Algorithm and Implementation

With the development of multi-queue data centers, various efforts have leveraged Explicit Congestion Notification (ECN) to achieve high throughput and low latency for data communication. However, one of the deep-seated problems is that the micro-burst traffic in networks could cause the instantaneous queue length to exceed the ECN threshold, leading to significant mismarkings of ECN. Such mismarkings could further lead to severe network performance degradation. In this paper, we propose Micro-Burst aware ECN (MBECN+) to mitigate this issue. MBECN+ is essentially a novel scheme that decouples ECN threshold setting and ECN marking. First, it finds a more appropriate ECN threshold on a per-queue basis to eliminate the spurious congestion signals caused by micro burst traffic. Second, MBECN+ consists of a double-threshold ECN marking scheme that it marks the packets with ECN in a finer grain. Considering the queue backlog caused by micro-burst traffic, it marks the packets when they are dequeuing, instead of enqueuing. Furthermore, we analyze the feasibility of implementing MBECN+ on commodity layer-3 multi-queue switches. Through testbed experiments and large scale simulations, we demonstrate that MBECN+ can improve the throughput by up to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\sim$</tex-math></inline-formula> 20% and reduce FCT (flow completion time) by up to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\sim$</tex-math></inline-formula> 40%. The throughput under MBECN+ improves by 1.5 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\sim 2.4\times$</tex-math></inline-formula> than DCTCP and 1.26 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\sim 1.35\times$</tex-math></inline-formula> than ECN*.

Choose a correct marking position: ECN should be freed from tail mark

Explicit Congestion Notification (ECN) has been widely employed in production datacenter networks (DCNs) to deliver high throughput and low latency communications. Despite being successful, the drawback of ECN-based transports is that they always mark the packets in the tail of the queue. We call this ECN marking scheme as TM-ECN (tail mark ECN). TM-ECN leads to long ECN feedback delay, significant queue oscillation, and poor short-flow friendliness. In this paper, we reveal these problems by leveraging extensive simulations to explore the intrinsic drawback of TM-ECN. Using the guidelines learned from the exploration, we design FM-ECN (front mark ECN) and RM-ECN (random mark ECN), two simple yet effective ECN marking schemes to mitigate the drawbacks. We theoretically analyze the TM-ECN, FM-ECN, and RM-ECN in the ECN feedback delay, queue stability, and short-flow friendliness. Finally, the conclusions from the analysis inspire us to propose a mixed mark ECN (MM-ECN) marking scheme. With the advantages of FM-ECN and RM-ECN, MM-ECN can maximize the effect of ECN. We further implement these ECN marking schemes in Linux kernel and ns-3 to evaluate them through testbed experiments and large-scale simulations. Our experimental results show that DCTCP with an appropriate ECN marking scheme achieves up to 55% higher concurrency tolerance of incast, and achieves up to 16% (25%) lower average (99th percentile) flow completion time (FCT) of short flows while delivering similar FCT for mid/large flows under production workloads.

A-ECN Minimizing Queue Length for Datacenter Networks

Recently, Explicit Congestion Notification (ECN) has been leveraged by DCTCP and its successive protocols to reduce the latency in datacenters. However, we find that DCTCP may induce the queue underflow and high latency in multi-queue scenarios and even single-queue environments. Our analysis concludes that the problem is caused by improper ECN marking thresholds of DCTCP. In this paper, an Adaptive ECN (A-ECN) marking scheme is proposed to enhance the performance of the ECN marking scheme of DCTCP.A-ECN is designed to minimize queue length as well as keeping high link utilization. More importantly, A-ECN can adaptively adjust ECN marking thresholds in different scenarios to achieve good generality. Therefore, network operators can directly deploy A-ECN in various environments regardless of underlying bandwidth, packet schedulers and the number of concurrent flows. At last, the testbed evaluation results show that A-ECN can minimize queue length and maintain high throughput. The flow completion times (FCT) for short flows could be reduced by up to 40% compared to the ECN marking scheme of DCTCP.

ECN+: A marking-aware optimization for ECN threshold via per-Port in Data Center Networks

Explicit Congestion Notification (ECN) is a powerful tool for quality of service in single queue scenarios. However, the industry trends to have Multi-Queue (MQ) per-port in Data Center Networks (DCNs). Having an MQ per-port is harmful at least for a scheduling policy of ECN. For instance, all the packets could get ECN marking due to the buffer pressure of the other queues belonging to the same shared port. By seeking solutions to the problems mentioned above, we propose a marking-aware optimization for ECN threshold via per-port marking in DCNs (ECN+). The intuition is that if any packet gets marked in the MQ buffer, ECN+ will not consider them in the marking decision of the Output Port Buffer (OPB). Furthermore, we show how this ignoring keeps the queue occupancy and ECN marking threshold (K) lower to achieve low latency. We do so through theoretical analysis of marked packets ratio in MQ and subtracting it from OPB. Besides, we calculate the ratio of the marked packet in OPB to achieve the minimum and maximum queue oscillations so that we can find an optimal K. To the best of our knowledge, we are the first to discuss the problem of faulty ECN marking via OPB through the fraction of marked packets. Steady-state analysis and simulation results demonstrate that ECN+ utilizes the buffer capacity, exactly 30% to absorb micro-burst traffic without sacrificing the latency.

DC-ECN: A machine-learning based dynamic threshold control scheme for ECN marking in DCN

Explicit Congestion Notification (ECN) is a deployment to IP and TCP also playing a crucial role in the congestion control of the Data Center Networks (DCNs). Most DCNs use a single queue scenario in each switch port. However, in the production of DCNs, the industry trend is moving towards one farther queue per port. Therefore, Multi-Service Multi-Queue Data Centers (MQ-ECN) have been proposed to ignite this service; afterward, DemePro improved the MQ-ECN. Yet, the fact that overhead and imprecise measurement are non-negligible for both models should be borne in mind. Also, MQ-ECN works for a round base scheduler while the overflow problem could be contributed by DemePro. Moreover, ECN was designed for single queue scenarios and having MQ-ECN is harmful at least for scheduling of flows. To solve the problem, we could take advantage of the lack of MQ-ECN and propose a machine-learning based dynamic threshold control scheme for ECN marking in DCN, which we named it DC-ECN (Data Center-Explicit Congestion Notification)–a first systematic solution to the problems which have already been mentioned earlier. The main point of DC-ECN is a separation of the mice and elephant flows in dual couple queues using machine learning. Then, to locate them into the requested queue with demand ECN marking threshold independently to achieve low latency and high throughput. Also, by dynamically increase and decrease the ECN marking threshold in elephant buffer, DC-ECN will never mark mice flows and succeed to absorb micro-burst mice traffic to have lowest latency without having sacrifice the throughput. Our mathematical analysis and simulation demonstrate that a steady state behavior of DC-ECN achieves 21.8% and 16.5% less flow completion time compared with MQ-ECN and DemoPro, respectively.

Flow-aware explicit congestion notification for datacenter networks

Explicit congestion notification (ECN) has been widely adopted by recent proposals to build up high-throughput and low-latency datacenter network transport. In these ECN-based proposals, when the queue length of a switch exceeds a pre-defined threshold, the switch would mark all arriving packets with ECN to explicitly notify their senders to slow down the rates. Such a design enables the network to eliminate congestions quickly. However, it marks packets without considering the flow state, which may overkill flows, especially those only send a few packets, thus resulting in significant throughput loss and long flow completion times. In this paper, we propose a novel flow-aware ECN marking approach (FECN), which can improve the throughput and flow completion time by taking flow states into consideration. By selectively marking packets respecting to their flow rates, FECN enables the network to precisely slow down the high-speed flows to avoid congestions without killing low-speed short flows. Moreover, FECN does not require switches to maintain per-flow state, which yields low overhead and thus makes FECN to be easily implemented and deployed in commodity switches. Simulations show that FECN can shorten the flow completion time by up to 44.7% and reduce the throughput loss by up to 40.3%, compared with prior flow-agnostic ECN marking approach.

Dynamic ECN marking threshold algorithm for TCP congestion control in data center networks

Recent years have witnessed the rapid development in data center, which hosts a wide variety of applications and services with diverse requirements for network performance. Data center networks (DCNs) have the characteristics of high bandwidth and low latency and TCP incast usually happens when multiple senders simultaneously communicate with a single receiver. In this paper, we first investigate the DCTCP, which is widely deployed in DCNs and uses Explicit Congestion Notification (ECN) as a congestion signal when queue length is larger than a fixed threshold. We demonstrate that the queuing delay may be high or the bottleneck link utilization can be insufficient due to inappropriate fixed threshold. Then we propose a dynamic ECN marking threshold algorithm based on the number of concurrent flows, referred to as DEMT, to improve the network performance. After that, an enhanced DEMT algorithm, referred to as EDEMT, is proposed by using the ratio of the queue length over the marking threshold to perceive the degree of network congestion. We also integrate DEMT with DCTCP to observe its behavior and theoretically analyze the throughput. Finally, simulation results demonstrate that the DEMT and EDEMT algorithms can effectively improve the network throughput while they can maintain a small queue length as well as the short flow completion time.

Slack-aware Congestion Control TCP for Data Center Networks

Aiming at minimizing average flow completion time of short flows and reducing flow deadline missed numbers for Online Data-Intensive (OLDI) applications in data center, this paper proposes LSTCP, a dynamic priority-based slack-aware transport control protocol for data center network that adopts the Least Slack First (LSF) scheduling strategy to prioritize flows. LSTCP implements the priority scheduling for emergency flows in the deadline constraint by dynamically adjusting congestion window according to the precedence of the flow and network congestion levels via Explicit Congestion Notification marking. Compared with the traditional deadline-aware TCP, LSTCP reduces the short flow’ AFCT and guarantee the throughput characteristics of long flows. Experiment results show that compared with L2DCT, LSTCP reduces the percentage of the flow missed deadline of 15%, and the AFCT for flows by 25%.

Adaptive marking threshold method for delay-sensitive TCP in data center network

Due to the highly dynamic traffic load, providing delay-sensitive services is a challenge in data centers. Many recent transport protocols, such as DCTCP, D2TCP, and L2DCT, use Explicit Congestion Notification (ECN) to control the switch queue length for achieving low-latency transmission. However, the original ECN marking threshold fails to make real-time tradeoff between bottleneck link utilization and queueing delay, thereby increasing the flow completion time. In this paper, we propose Adaptive Marking Threshold (AMT), which proactively tunes the marking threshold to eliminate the unnecessary queueing delay, and maintains high link utilization at the same time. The experimental results of NS2 simulation and physical testbed show that AMT significantly reduces the flow completion time of typical applications across different network scenarios with negligible change in the data center switch.

TCP flow aware adaptive path switching in diffserv enabled MPLS networks

We propose an adaptive flow-level multi-path routing-based traffic engineering solution for an IP backbone network carrying TCP/IP traffic. Incoming TCP flows are switched between two explicitly routed paths, namely the primary and secondary paths (PP and SP), for resilience and potential goodput improvement at the TCP layer. In the proposed architecture, PPs receive a preferential treatment over SPs using differentiated services mechanisms. The reason for this choice is not for service differentiation but for coping with the detrimental knock-on effect stemming from theuse of longer SP that is well known for conventional network load balancing algorithms. Moreover, both paths are congestion-controlled using Explicit Congestion Notification marking at the core and Additive Increase Multiplicative Decrease rate adjustment at the ingress nodes. The delay difference between PP and SP is estimated using two per-egress rate-controlling buffers maintained at the ingress nodes for each path, and this delay difference is used to determine the path over which a new TCP flow will be routed. We perform extensive simulations using ns-2 in order to demonstrate the viability of the proposed distributed adaptive multi-path routing method in terms of per-flow TCP goodput. The proposed solution consistently outperforms the single-path routing policy and provides substantial per-flow goodput gains under poor PP conditions. Moreover, highest goodput improvements under the proposed scheme are achieved by flows that receive the lowest goodputs with single-path routing, while the performance of the flows with high goodputs with single-path routing does not deteriorate with the proposed path switching technique. Copyright # 2011 John Wiley & Sons, Ltd.

Adding Randomness to Modeling Internet TCP-RED Systems With Interactive Gateways

In Internet gateways with active queue management, effective algorithms are often used to enhance the traffic flows through the gateway. The random early detection (RED) algorithm is a widely used algorithm for controlling the transmission control protocol (TCP) flows. Specifically, the RED algorithm has deliberately introduced randomness into the explicit congestion notification (ECN) marking mechanism in order to enhance the fairness and to avoid synchronization of TCP flows through the gateway. As randomness plays a key role in providing better performance for RED gateways than the DropTail gateways, knowledge of the effect of the random mechanism is important to the understanding the behavior of the system. The fluid flow model (FFM), ignoring the randomness of ECN markings, has provided a fast alternative to the slow ns-2 network simulation tool in studying the TCP flow in the RED gateway. In this brief, the random ECN marking mechanism in RED gateways has been studied. The randomness of the RED ECN marking algorithm is implemented into the FFM. Verified using ns-2 simulations, the proposed model shows better dynamical performance than FFM in both the single-bottleneck dumbbell network and in networks with interactive bottlenecks.

TCP-friendly flow control of wireless multimedia using ECN marking

In a wireless network packet losses can be caused not only by network congestion but also by unreliable error-prone wireless links. Therefore, flow control schemes which use packet loss as a congestion measure cannot be directly applicable to a wireless network because there is no way to distinguish congestion losses from wireless losses. In this paper, we extend the so-called TCP-friendly flow control scheme, which was originally developed for the flow control of multimedia flows in a wired IP network environment, to a wireless environment. The main idea behind our scheme is that by using explicit congestion notification (ECN) marking in conjunction with random early detection (RED) queue management scheme intelligently, it is possible that not only the degree of network congestion is notified to multimedia sources explicitly in the form of ECN-marked packet probability but also wireless losses are hidden from multimedia sources. We calculate TCP-friendly rate based on ECN-marked packet probability instead of packet loss probability, thereby effectively eliminating the effect of wireless losses in flow control and thus preventing throughput degradation of multimedia flows travelling through wireless links. In addition, we refine the well-known TCP throughput model which establishes TCP-friendliness of multimedia flows in a way that the refined model provides more accurate throughput estimate of a TCP flow particularly when the number of TCP flows sharing a bottleneck link increases. Through extensive simulations, we show that the proposed scheme indeed improves the quality of the delivered video significantly while maintaining TCP-friendliness in a wireless environment for the case of wireless MPEG-4 video.

End-to-end congestion control schemes: utility functions, random losses and ecn marks

We present a framework for designing end-to-end congestion control schemes in a network where each user may have a different utility function and may experience noncongestion-related losses. We first show that there exists an additive-increase-multiplicative-decrease scheme using only end-to-end measurable losses such that a socially optimal solution can be reached. We incorporate round-trip delay in this model, and show that one can generalize observations regarding TCP-type congestion avoidance to more general window flow control schemes. We then consider explicit congestion notification (ECN) as an alternate mechanism (instead of losses) for signaling congestion and show that ECN marking levels can be designed to nearly eliminate losses in the network by choosing the marking level independently for each node in the network. While the ECN marking level at each node may depend on the number of flows through the node, the appropriate marking level can be estimated using only aggregate flow measurements, i.e., per-flow measurements are not required.