Enhancing Inter-Node Process Migration for Load Balancing on Linux-Based NUMA Multicore Systems

Abstract

Multicore systems with Non-Uniform Memory Access (NUMA) architecture are getting popular in computer systems. Processor cores take longer time to access memories on remote NUMA nodes than on local nodes. Because of operating system kernel's memory allocation and load balancing activities, a process may be migrated across nodes and its allocated memory may scatter on several nodes. Remote memory access and resource contention cause significant performance degradation in multicore NUMA systems. In this study, to reduce contention for inter-node interconnect links and decrease remote memory access in NUMA systems, we enhance the kernel's inter-node load balancing by migrating suitable processes or light-weight processes (i.e. threads) between NUMA nodes. We further propose various selection policies that select processes for migration according to their memory usages, so that the selected process uses the least amount of page frames in the system and/or shares the least amount of memory with other processes. These processes are expected to incur less remote memory access and have least influence caused by migration. The Linux kernel is modified to incorporate the proposed policies in the enhanced inter-node load balancing procedure. Experimental results demonstrate that the system performance is successfully improved by effectively reducing remote memory access and resource contention.