Receive Side Scaling Research Articles

Making stateless and stateful network performance measurements unbiased

The Benchmarking Working Group (BMWG) of the Internet Engineering Task Force (IETF) has defined a series of Requests for Comments (RFC) to standardize the benchmarking of network interconnect devices (e.g., bridges, routers, different IPv6 transition solutions). The paper points out that there are cases where the performance results are significantly different when a single IP address pair or multiple IP addresses are used. The cause of this phenomenon is rooted in the recent hardware and software advancements: Receive Side Scaling (RSS) makes it possible to distribute packet processing workload over multiple CPU cores. However, this may be implemented in two ways: the first way only includes the IP addresses into the hash function used to distribute the workload among the CPU cores, whereas the second one also includes the port numbers. RFC 4814 proposed an excellent solution for the second case by recommending the usage of pseudorandom port numbers during benchmarking; however, the first case was not handled properly, because no explicit recommendation was given regarding the usage of multiple IP addresses. This paper attempts to bridge this methodological gap; a practical solution is proposed for using pseudorandom IP addresses in various scenarios including the benchmarking of IPv4 and IPv6 routers and Network Address Translation from IPv6 Clients to IPv4 Servers (stateful NAT64) gateways. Its feasibility is shown by disclosing the details of its implementation in siitperf. Then the proposed solution is validated by both stateless and stateful tests. It is shown that the measurement results of the tests following the proposed solution can better characterize the true performance of the network interconnect devices that follow the first type of RSS implementation than the results of the tests using a single IP address pair.

Speeding up IP lookup procedure in software routers by means of parallelization

Performance of software routers is limited by the speed of operating system network protocol stack. A faster network protocol stack can be implemented in user space by utilizing parallelization and different optimization techniques. In this work we will demonstrate an efficient implementation of the IP lookup algorithm in user space with multibit trie structures. Afterwards, we will demonstrate the improvements achieved through parallelization based on the receive side scaling mechanism.

Performance Enhancement for Network I/O Virtualization with Efficient Interrupt Coalescing and Virtual Receive-Side Scaling

Virtualization is a key technology in cloud computing; it can accommodate numerous guest VMs to provide transparent services, such as live migration, high availability, and rapid checkpointing. Cloud computing using virtualization allows workloads to be deployed and scaled quickly through the rapid provisioning of virtual machines on physical machines. However, I/O virtualization, particularly for networking, suffers from significant performance degradation in the presence of high-speed networking connections. In this paper, we first analyze performance challenges in network I/O virtualization and identify two problems—conventional network I/O virtualization suffers from excessive virtual interrupts to guest VMs, and the back-end driver does not efficiently use the computing resources of underlying multicore processors. To address these challenges, we propose optimization methods for enhancing the networking performance: 1) Efficient interrupt coalescing for network I/O virtualization and 2) virtual receive-side scaling to effectively leverage multicore processors. These methods are implemented and evaluated with extensive performance tests on a Xen virtualization platform. Our experimental results confirm that the proposed optimizations can significantly improve network I/O virtualization performance and effectively solve the performance challenges.

A Transport-Friendly NIC for Multicore/Multiprocessor Systems

Receive side scaling (RSS) is an NIC technology that provides the benefits of parallel receive processing in multiprocessing environments. However, RSS lacks a critical data steering mechanism that would automatically steer incoming network data to the same core on which its application thread resides. This absence causes inefficient cache usage if an application thread is not running on the core on which RSS has scheduled the received traffic to be processed and results in degraded performance. To remedy the RSS limitation, Intel's Ethernet Flow Director technology has been introduced. However, our analysis shows that Flow Director can cause significant packet reordering. Packet reordering causes various negative impacts in high-speed networks. We propose an NIC data steering mechanism to remedy the RSS and Flow Director limitations. This data steering mechanism is mainly targeted at TCP. We term an NIC with such a data steering mechanism “A Transport-Friendly NIC” (A-TFN). Experimental results have proven the effectiveness of A-TFN in accelerating TCP/IP performance.