Impact of Page Size on Communication Performance

Abstract

In this paper, the impact of page size on the communication performance is studied. In the interconnection communication of cluster system, the address translation table (ATT), which is located in the memory of the network interface card (NIC) and can in a way be seen as the translation look-aside buffer (TLB) used by the NIC processor, is usually used to translate virtual address to physical address by NIC. The page size of operating system not only affects the compulsory and capacity miss rate, but also the hit time and the miss penalty of ATT in some implementations. With a large page size, we can get lower ATT miss rate, shorter hit time and miss penalty to improve the communication performance. To test the impact of the page size, a Linux module based on AMD Opteron/spl trade/ processor is implemented to allocate both normal pages and super pages and the address translation mechanism in Myrinet GM is also extended to support either normal pages or super pages. With super pages, the latency of Ping-pong test can be reduced 4.3 us and the bandwidth can improve 55.3 MB/s in some case. The Linpack test results of 11 TFLOPS Dawning 4000A show that the Linpack efficiency can be increased from 0.66% to 2.86% for different number of processors.