TCP Incast Research Articles

The Effectiveness of Random Early Detection in Data Center Transmission Control Protocol-Based Cloud Computing Networks

Wireless cloud computing environment becomes an important element of Internet. The underlying transport layer protocol in such environment should efficiently deal with high bursty traffic tolerance, presents low delays, and offers high throughput. DCTCP is the common cloud transport protocol that is capable of tackling TCP incast and queue build-up problems. However, its performance is considerably degraded when there are tens of servers responding to one aggregator simultaneously. In this paper, the performance of DCTCP with the utilization of RED queue management mechanism is evaluated. The evaluation results showed that there is a tradeoff in the use of RED. On one hand, RED is not good for short-lived and medium-lived connections as it presents higher completion time delay compared to the normal DCTCP mechanism. On the other hand, it presentsacceptable delay for longlived connections but at the cost of throughput.

Staggered approach for alleviating TCP Incast in simultaneous Multi-VM migration

In cloud computing, virtual machine (VM) migration plays a key role in maximizing the server efficiency, saving energy, and facilitating system maintenance. However, while virtualization systems such as vSphere and Hyper-V enable the overall migration time to be reduced by moving more than one VM at the same time, the resulting rapid growth in the volume of data temporarily stored at the buffer can lead to significant packet losses and a catastrophic drop in the system throughput. The present study proposes a method for mitigating this so-called TCP Incast effect by sorting the VMs in order of descending load and then interleaving the migrations of the more lightly-loaded VMs with those of the more heavily-loaded VMs. By doing so, the overlapping time of the VM migrations is reduced, and hence the risk of overflow-induced packet losses is decreased. A server consolidation ratio is derived to estimate the energy saving achieved via server shutdown given pre-defined constraints on the CPU and memory resources of the destination servers. The simulation results show that the proposed staggered VM migration scheme achieves a similar server consolidation performance as existing sequential VM migration methods, but results in both a lower data storage requirement at the buffer and fewer vMotions.

SDTCP: Towards Datacenter TCP Congestion Control with SDN for IoT Applications.

The Internet of Things (IoT) has gained popularity in recent years. Today’s IoT applications are now increasingly deployed in cloud platforms to perform Big Data analytics. In cloud data center networks (DCN), TCP incast usually happens when multiple senders simultaneously communicate with a single receiver. However, when TCP incast happens, DCN may suffer from both throughput collapse for TCP burst flows and temporary starvation for TCP background flows. In this paper, we propose a software defined network (SDN)-based TCP congestion control mechanism, referred to as SDTCP, to leverage the features, e.g., centralized control methods and the global view of the network, in order to solve the TCP incast problems. When we detect network congestion on an OpenFlow switch, our controller can select the background flows and reduce their bandwidth by adjusting the advertised window of TCP ACK packets of the corresponding background flows so as to reserve more bandwidth for burst flows. SDTCP is transparent to the end systems and can accurately decelerate the rate of background flows by leveraging the global view of the network gained via SDN. The experiments demonstrate that our SDTCP can provide high tolerance for burst flows and achieve better flow completion time for short flows. Therefore, SDTCP is an effective and scalable solution for the TCP incast problem.

AAIC: Adaptive-Sliding-Connection-Window Solution to TCP Incast from Application Layer

In datacenters, TCP Incast causes catastrophic goodput degradation to applications with many-to-one traffic pattern. This letter intends to tame Incast from application layer. For this goal, we first develop an analytical model to understand how to minimize Incast probability by tuning connection variables related to application. Then, enlightened by the model, we propose an adaptive application-layer solution to TCP Incast (AAIC). The solution adopts an innovative sliding-connection-window mechanism that dynamically adjusts the number of concurrent connections and the advertised window of each connection. Simulations prove the validity of our model and the effectiveness of AAIC to avert Incast in changing datacenter environments.

TCP Incast Avoidance Based on Connection Serialization in Data Center Networks

TCP Incast Avoidance Based on Connection Serialization in Data Center Networks

Optimizing the data-collection time of a large-scale data-acquisition system through a simulation framework

The ATLAS detector at CERN records particle collision “events” delivered by the Large Hadron Collider. Its data-acquisition system identifies, selects, and stores interesting events in near real-time, with an aggregate throughput of several 10 GB/s. It is a distributed software system executed on a farm of roughly 2000 commodity worker nodes communicating via TCP/IP on an Ethernet network. Event data fragments are received from the many detector readout channels and are buffered, collected together, analyzed and either stored permanently or discarded. This system, and data-acquisition systems in general, are sensitive to the latency of the data transfer from the readout buffers to the worker nodes. Challenges affecting this transfer include the many-to-one communication pattern and the inherently bursty nature of the traffic. The main performance issues brought about by this workload are addressed in this paper, focusing in particular on the so-called TCP incast pathology. Since performing systematic studies of these issues is often impeded by operational constraints related to the mission-critical nature of these systems, we developed a simulation model of the ATLAS data-acquisition system. The resulting simulation tool is based on the well-established, widely-used OMNeT++ framework. This tool was successfully validated by comparing the obtained simulation results with existing measurements of the system’s behavior. Furthermore, the simulation tool enables the study of the theoretical behavior of the system in numerous what-if scenarios and with modifications that are not immediately applicable to the real system. In this paper, we take advantage of this to analyze the behavior of the system using different traffic shaping and scheduling policies, and with network hardware modifications. This analysis leads to conclusions that could be used to devise future system enhancements.

A simple and efficient approach for reducing TCP timeouts due to lack of duplicate acknowledgments in data center networks

The problem of TCP incast in data centers attracts a lot of attention in our research community. TCP incast is a catastrophic throughput collapse that occurs when multiple senders transmitting TCP data simultaneously to a single aggregator. Based on several experiments, researchers found that TCP timeouts are the primary cause of incast problem. Particularly, timeouts due to insufficient duplicate acknowledgments is unavoidable when at least one of the last three segments is lost from the tail of a window. As a result, this type of timeouts should be avoided to improve the goodput of TCP in data center networks. A few attempts have been made to reduce timeouts, but still the problem is not completely solved especially in the case of timeouts due to insufficient duplicate acknowledgments. In this paper, we present an efficient TCP fast retransmission approach, called TCP-EFR, which is capable to reduce TCP timeouts due to lack of duplicate acknowledgments which is caused by the loss of packets from the tail of a window in data center networks. TCP-EFR makes changes in the fast retransmission and recovery algorithm of TCP by using the congestion signal mechanism of DCTCP based on instantaneous queue length. In addition, TCP-EFR controls the sending rate for avoiding the overflow of switch buffer in order to reduce the loss of packets. The results of a series of simulations in single as well as multiple bottleneck topologies using qualnet 4.5 demonstrates that TCP-EFR can significantly reduce the timeouts due to inadequate duplicate acknowledgments and noticeably improves the performance compared to DCTCP, ICTCP and TCP in terms of goodput, accuracy and stability under various network conditions.

An early congestion feedback and rate adjustment schemes for many-to-one communication in cloud-based data center networks

Cloud data centers are playing an important role for providing many online services such as web search, cloud computing and back-end computations such as MapReduce and BigTable. In data center network, there are three basic requirements for the data center transport protocol such as high throughput, low latency and high burst tolerance. Unfortunately, conventional TCP protocols are unable to meet the requirements of data center transport protocol. One of the main practical issues of great importance is TCP Incast to occur many-to-one communication sessions in data centers, in which TCP experiences sharp degradation of throughput and higher delay. This important issue in data center networks has already attracted the researchers because of the development of cloud computing. Recently, few solutions have been proposed for improving the performance of TCP in data center networks. Among that, DCTCP is the most popular protocol in academic as well as industry areas due to its better performance in terms of throughput and latency. Although DCTCP provides significant performance improvements, there are still some defects in maintaining the queue length and throughput when the number of servers is too large. To address this problem, we propose a simple and efficient TCP protocol, namely NewDCTCP as an improvement of DCTCP in data center networks. NewDCTCP modified the congestion feedback and window adjusting schemes of DCTCP to mitigate the TCP Incast problem. Through detailed QualNet experiments, we show that NewDCTCP significantly outperforms DCTCP and TCP in terms of goodput and latency. The experimental results also demonstrate that NewDCTCP flows provide better link efficiency and fairness with respect to DCTCP.

DIABLO

Motivated by rapid software and hardware innovation in warehouse-scale computing (WSC), we visit the problem of warehouse-scale network design evaluation. A WSC is composed of about 30 arrays or clusters, each of which contains about 3000 servers, leading to a total of about 100,000 servers per WSC. We found many prior experiments have been conducted on relatively small physical testbeds, and they often assume the workload is static and that computations are only loosely coupled with the adaptive networking stack. We present a novel and cost-efficient FPGAbased evaluation methodology, called Datacenter-In-A-Box at LOw cost (DIABLO), which treats arrays as whole computers with tightly integrated hardware and software. We have built a 3,000-node prototype running the full WSC software stack. Using our prototype, we have successfully reproduced a few WSC phenomena, such as TCP Incast and memcached request latency long tail, and found that results do indeed change with both scale and with version of the full software stack.

DIABLO

Motivated by rapid software and hardware innovation in warehouse-scale computing (WSC), we visit the problem of warehouse-scale network design evaluation. A WSC is composed of about 30 arrays or clusters, each of which contains about 3000 servers, leading to a total of about 100,000 servers per WSC. We found many prior experiments have been conducted on relatively small physical testbeds, and they often assume the workload is static and that computations are only loosely coupled with the adaptive networking stack. We present a novel and cost-efficient FPGAbased evaluation methodology, called Datacenter-In-A-Box at LOw cost (DIABLO), which treats arrays as whole computers with tightly integrated hardware and software. We have built a 3,000-node prototype running the full WSC software stack. Using our prototype, we have successfully reproduced a few WSC phenomena, such as TCP Incast and memcached request latency long tail, and found that results do indeed change with both scale and with version of the full software stack.

Modeling and Solving TCP Incast Problem in Data Center Networks

TCP Incast problem attracts much attention due to the catastrophic goodput drop. In this paper, a goodput model of the problem is built to understand why goodput collapse occurs and a solution to the problem based on the theoretical analysis is proposed. We found that the TCP Incast goodput deterioration is mainly caused by two types of timeouts, one happens at the tail of data blocks and dominates the goodput when the number of senders is small, while the other one at the head of data blocks and governs the goodput when the number of senders is large. The proposed model describes the relationship between these two types of timeouts and the Incast communication pattern, block size, bottleneck buffer size, and so on. The simulation results indicate that the model well characterizes the features of the TCP Incast problem. Enlightened by the analysis, a PRiority-based solution to the TCP INcast problem (PRIN) is proposed, which avoids timeouts at the head of blocks by reducing TCP send window and prevents timeouts at the tail of blocks by leveraging priority technology. The experimental results show that PRIN solves the TCP Incast problem.

English

Many applications (e.g., cluster based storage and Map Reduce) in modern data centers require a high fan-in, manytoone type of data communication (known as TCP in cast), which could cause severe in cast congestion in switches and result in TCP good put collapse, substantially degrading the application performance. The root cause of such a collapse is the long idle period of the Retransmission Timeout (RTO) that is triggered at one or more senders by packet losses in congested switches. In this paper We Study a packet labeling scheme PLATO, which improves the loss detection capabilities of New Reno using an innovative packet labeling system. Packets carrying this special label are preferentially en queued, at the switch. This allows TCP to detect packet loss using three duplicate acknowledgements, instead of the time expensive RTO; thus avoiding the good put collapse. PLATO makes minor modifications to New Reno and does not alter its congestion control mechanism. The implementation and simulations have been done in Network Simulator 3 (NS3). PLATO’s performance is significantly better than New Reno as well as state-of-art in cast solutions In cast Control TCP (ICTCP) and Data Center TCP (DCTCP). We also show that TCP PLATO can be implemented using commodity switches with Weighted Random Early Detection (WRED) function.

TCP PLATO: Packet Labelling to Alleviate Time-Out

Many applications (e.g., cluster based storage and MapReduce) in modern data centers require a high fan-in, many-to-one type of data communication (known as TCP incast), which could cause severe incast congestion in switches and result in TCP goodput collapse, substantially degrading the application performance. The root cause of such a collapse is the long idle period of the Retransmission Timeout (RTO) that is triggered at one or more senders by packet losses in congested switches. In this paper we develop a packet labelling scheme PLATO, which improves the loss detection capabilities of NewReno using an innovative packet labelling system. Packets carrying this special label are preferentially enqueued, at the switch. This allows TCP to detect packet loss using three duplicate acknowledgements, instead of the time expensive RTO; thus avoiding the goodput collapse. PLATO makes minor modifications to NewReno and does not alter its congestion control mechanism. The implementation and simulations have been done in Network Simulator 3 (NS3). PLATO's performance is significantly better than NewReno as well as state-of-art incast solutions Incast Control TCP (ICTCP) and Data Center TCP (DCTCP). We also show that TCP PLATO can be implemented using commodity switches with Weighted Random Early Detection (WRED) function.

LTTP: An LT-Code Based Transport Protocol for Many-to-One Communication in Data Centers

TCP has been widely adopted in current data centers to ensure reliable data delivery. However, recently TCP Incast was found to occur in many-to-one communications with barrier-synchronized requirement, where the TCP goodput drops dramatically. Previous solutions to TCP Incast either require updating the OS/hardware to support fine-grained timers, or smartly control utilization of the switch buffer to reduce the probability of buffer overflow and packet loss. In this paper we explore a different approach to support many-to-one communication in data center networks, which we call LTTP (LT-code based Transport Protocol). LTTP improves LT (Luby Transform) code to achieve reliable UDP-based transmission by exploiting data redundancy, and employs TFRC (TCP Friendly Rate Control) to adjust the traffic sending rates at servers. NS-2 based simulation shows that the goodput of LTTP never degrades with the increase of the number of servers in many-to-one communications, and LTTP significantly outperforms DCTCP when the number of servers is large. Simulation results also demonstrate that LTTP flows can fairly share bandwidth with TCP flows.

IDTCP: An effective approach to mitigating the TCP incast problem in data center networks

Recently, TCP incast problem in data center networks has attracted a wide range of industrial and academic attention. Lots of attempts have been made to address this problem through experiments and simulations. This paper analyzes the TCP incast problem in data centers by focusing on the relationships between the TCP throughput and the congestion control window size of TCP. The root cause of the TCP incast problem is explored and the essence of the current methods to mitigate the TCP incast is well explained. The rationality of our analysis is verified by simulations. The analysis as well as the simulation results provides significant implications to the TCP incast problem. Based on these implications, an effective approach named IDTCP (Incast Decrease TCP) is proposed to mitigate the TCP incast problem. Analysis and simulation results verify that our approach effectively mitigates the TCP incast problem and noticeably improves the TCP throughput.

Congestion Control in Data Center Networks: A Survey and New Perspectives

Nowadays data centers have been becoming increasingly important for various web applications, huge amount of cost is invested to maintain good performance of data center. Whereas some studies indicated that TCP Incast phenomenon was widely observed in most of data centers, which results in congestion in data centers and damages the performance of data center greatly. Thus some congestion control mechanisms for data center have been proposed to solve the problems. These mechanisms are categorized and described in this paper, and the advantages and disadvantages of these mechanisms are analyzed. Subsequently, some new interesting topics which may be worthy of further study in congestion control mechanism on data center are presented.

Fair Quantized Congestion Notification in Data Center Networks

Quantized Congestion Notification (QCN) has been developed for IEEE 802.1Qau to provide congestion control at the Ethernet Layer or Layer 2 in data center networks (DCNs) by the IEEE Data Center Bridging Task Group. One drawback of QCN is the rate unfairness of different flows when sharing one bottleneck link. In this paper, we propose an enhanced QCN congestion notification algorithm, called fair QCN (FQCN), to improve rate allocation fairness of multiple flows sharing one bottleneck link in DCNs. FQCN identifies congestion culprits through joint queue and per flow monitoring, feedbacks individual congestion information to each culprit through multi-casting, and ensures convergence to statistical fairness. We analyze the stability and fairness of FQCN via Lyapunov functions and evaluate the performance of FQCN through simulations in terms of the queue length stability, link throughput and rate allocations to traffic flows with different traffic dynamics under three network topologies. Simulation results confirm the rate allocation unfairness of QCN, and validate that FQCN maintains the queue length stability, successfully allocates the fair share rate to each traffic source sharing the link capacity, and enhances TCP throughput performance in the TCP Incast setting.

On Architecture Design, Congestion Notification, TCP Incast and Power Consumption in Data Centers

With rapid deployment of modern data center networks (DCNs), many problems with respect to DCN architecture, congestion notification, TCP Incast, and power consumption have been observed. In this paper, we provide a comprehensive survey of the most recent research activities in DCNs, with emphasis on the network architecture design, congestion notification algorithms, TCP Incast, and power consumption. We describe in detail the architecture of a typical modern DCN. Furthermore, we investigate the challenges of the typical tree-based hierarchical DCN architecture encountered today and the requirements for the optimal DCN architecture, and we classify the proposed DCN architectures into switch-centric topology and server-centric topology. A section is also devoted to describe some newly proposed DCN architectures. We present a brief overview of the TCP Incast problem along with previously proposed solutions. We also review the recent energy-efficient solutions to minimize the power consumption in DCNs. Finally, we outline possible future research topics in DCNs.

Traffic Throttling to Improve TCP Incast Throughput

Cluster-based storage systems are widely used by many large data centers because of the manageability, low cost and other advantages. However, typical cluster-based storage systems rely on the standard TCP/IP Ethernet for clients to access data. Clients would experience the TCP effective throughput (called goodput) collapse which is termed as Incast problem. The main reason of Incast is TCP retransmit timeout which is caused by unfairness between the competing flows. In this paper, we analyze the TCP Incast dynamics from the fairness aspect and propose a method to make the competing flows share the network resource fairly by throttling the traffic. The simulation results show that our method is able to improve the TCP Incast goodput by 10% comparing to the high-timer-resolution

Safe and effective fine-grained TCP retransmissions for datacenter communication

This paper presents a practical solution to a problem facing high-fan-in, high-bandwidth synchronized TCP workloads in datacenter Ethernets---the TCP incast problem. In these networks, receivers can experience a drastic reduction in application throughput when simultaneously requesting data from many servers using TCP. Inbound data overfills small switch buffers, leading to TCP timeouts lasting hundreds of milliseconds. For many datacenter workloads that have a barrier synchronization requirement (e.g., filesystem reads and parallel data-intensive queries), throughput is reduced by up to 90%. For latency-sensitive applications, TCP timeouts in the datacenter impose delays of hundreds of milliseconds in networks with round-trip-times in microseconds. Our practical solution uses high-resolution timers to enable microsecond-granularity TCP timeouts. We demonstrate that this technique is effective in avoiding TCP incast collapse in simulation and in real-world experiments. We show that eliminating the minimum retransmission timeout bound is safe for all environments, including the wide-area.