SilkRoad II: mixed paradigm cluster computing with RC_dag consistency

Abstract

A parallel programming paradigm indicates the way to express applications. It also restricts the algorithms that may be used in the applications. Unfortunately, runtime systems for parallel computing often impose a particular programming paradigm. For a wider choice of algorithms, it is therefore desirable to support more than one paradigm. In this paper, we propose a formalism for modeling parallel programming paradigms from a graph–theoretic view of their execution instance dag and the memory consistency assumptions. This model allows us to formally reason about the properties of parallel programming paradigms that are hitherto only known informally and intuitively. We propose the concept of general paradigm and show that the single program multiple data, the divide and conquer, and the master/slave paradigms are all sub-sets of this general paradigm. We will also propose a super set of these three paradigms which we called the mixed paradigm and introduce the RC_dag memory consistency model. We also present our work on SilkRoad II, a variant of the Cilk runtime system for cluster computing. What is unique about SilkRoad II is its memory model which supports multiple paradigms with the underlying software distributed shared memory. Our experimental results show that the stronger RC_dag can achieve performance comparable to LC of Cilk while supporting a bigger set of paradigms with rather good performance.