Efficient Handling of Arrays in Dataflow Process Networks

Abstract

Dataflow process networks (DPN) have been proposed as a programming model for distributed parallel systems that have communication paths with unpredictable latencies. The purely data-driven execution of DPNs does not require a global coordination and therefore allows one to easily map DPNs to many parallel hardware and software architectures with distributed memories. Problems due to shared memory communication like data races do not exist since the communication is done via point-to-point FIFO buffers between the process nodes. On the other hand, this distributed programming model leads to high communication costs if large data structures like arrays have to be communicated between nodes. This paper addresses the reduction of the communication costs due to arrays that are processed by -- and therefore sent between -- DPN nodes. The presented methods are given at a high level of abstraction and do not impose specific constraints on the used target architectures. Our experimental results show clearly the advantage of our approach compared to the standard message passing between the nodes.