A Dynamic Multi-Threaded Queuing Mechanism for Reducing the Inter-Process Communication Latency on Multi-Core Chips

Abstract

Reducing latency in inter-process/inter-thread communication is one of the key challenges in parallel and distributed computing. This is because as the number of threads in an application increases, the communication overhead also increases. Moreover, the presence of background load further increases the latency. Reducing communication latency can have a significant impact on multi-threaded application performance in multi-core environments. In a wide-range of applications that utilize queueing mechanism, inter-process/ inter-thread communication typically involves enqueuing and dequeuing. This paper presents a queueing techniques called eLCRQ, which is a lock-free block-when-necessary multi-producer multi-consumer (MPMC) FIFO queue. It is designed for scenarios where the queue can randomly and frequently become empty during runtime. By combining lock-free performance with blocking resource efficiency, it delivers improved performance. Specifically, it results in a 1.7X reduction in latency and a 2.3X reduction in CPU usage when compared to existing message-passing mechanisms including PIPE and Sockets while running on multi-core Linux based systems. The proposed scheme also provides a 3.4X decrease in CPU usage while maintaining comparable latency when compared to other (MPMC) lock-free queues in low load scenarios. Our work is based on open-source Linux and support libraries.