Supporting coarse and fine grain parallelism in an extension of ML

Abstract

We have built an extension of Standard ML aimed at multicomputer platforms with distributed memories. The resulting language, paraML, differs from other extensions by including and differentiating both coarse-grained and fine-grained parallelism.The basis for coarse-grained parallelism in paraML is process creation where there is no sharing of data, with communication between processes via asynchronous message passing to typed ports. Fine-grained parallelism may be introduced in a number of ways; we have investigated data parallelism and light-weight threads as per Concurrent ML. Our experiments with data parallelism and algorithmic skeletons illustrate that libraries can be constructed to hide process creation and message passing from the user. Furthermore, since processes may be arbitrarily nested, such libraries can be linked together in program hierarchies, allowing the integration of a range of parallel granularities to be used within a program in a modular and efficient manner.Performance analysis of the system is encouraging, with good speedup for a range of problems. Optimisation of process creation and message passing have lowered the size at which paraML becomes cost-effective against sequential implementations of ML for scientific problems.Keywordsparallel granularityprocessesMLalgorithmic skeletonsdata parallelism