Data-parallel Language Research Articles

Plurals: A SIMD extension to EuLisp

There are now several versions of Lisp for massively parallel SIMD architectures like the Thinking Machines Connection Machine. We describe here the extensions made to EuLisp for data-parallel programming and their implementation on a specific platform, a MasPar MP-1. Plural EuLisp, in keeping with the rest of the language, presents a collection of simple orthogonal operators which capture the essence of data parallel processing. In support of this, we demonstrate how to implement a number of higher-level abstraction from other data-parallel languages.

Implementation of a portable nested data-parallel language

This paper gives an overview of the implementation of NESL, a portable nested data-parallel language. This language and its implementation are the first to fully support nested data structures as well as nested data-parallel function calls. These features allow the concise description of parallel algorithms on irregular data, such as sparse matrices and graphs. In addition, they maintain the advantages of data-parallel languages: a simple programming model and portability. The current NESL implementation is based on an intermediate language called VCODE and a library of vector routines called CVL. It runs on the Connection Machine CM-2, the Cray Y-MP C90, and serial machines. We compare initial benchmark results of NESL with those of machine-specific code on these machines for three algorithms: least-squares line-fitting, median finding, and a sparse-matrix vector product. These results show that NESL's performance is competitive with that of machine-specific codes for regular dense data, and is often superior for irregular data.

Compiling nested data-parallel programs for shared-memory multiprocessors

While data parallelism is well suited from algorithmic, architectural, and linguistic considerations to serve as a basis for portable parallel programming, its characteristic fine-grained parallelism makes the efficient implementation of data-parallel languages on MIMD machines a challenging task. The design, implementation, and evaluation of an optimizing compiler are presented for an applicative nested data-parallel language called Vcode targeted at the Encore Multimax, a shared-memory multiprocessor. The source language supports nested aggregate data types; aggregate operations including elementwise forms, scans, reductions, and permutations; and conditionals and recursion for control flow

An equational language for data-parallelism

Data-parallelism provides a clean conceptual framework for parallel programming. We are developing two programming languages: a high level equational language, called EL * , and a low-level implementation language. Both languages exploit data-parallelism instead of control-parallelism. EL * is a declarative data-parallel language. EL * programs are high-level equational specifications that use extensive pattern-matching and recursion. The language's syntax and semantics are intended to be clear and simple. Recursive forms are restricted to enable translation to efficient data-parallel operations. EL * programs are compiled into FP * , a variant of Backus's FP, where parallel operations are more explicit and low-level. The target language has a rich set of functions for performing communication, and computation. It also has a powerful set of combining forms that generate large highly-parallel functions from smaller program units. Prototype compilers have been implemented for both languages, and they demonstrate good performance. Several linear algebra and non-numeric problems have been programmed with relative ease using EL * . We are currently developing compilation techniques for a wider range of scientific problems that have more complex parallel solutions, and are continuing to expand the language's scope.

Generating local addresses and communication sets for data-parallel programs

Generating local addresses and communication sets is an important issue in distributed-memory implementations of data-parallel languages such as High Performance Fortran. We show that for an array A affinely aligned to a template that is distributed across p processors with a cyclic(k) distribution, and a computation involving the regular section A(l:h:s) , the local memory access sequence for any processor is characterized by a finite state machine of at most k states. We present fast algorithms for computing the essential information about these state machines, and extend the framework to handle multidimensional arrays. We also show how to generate communication sets using the state machine approach. Performance results show that this solution requires very little runtime overhead and acceptable preprocessing time.

Transforming high-level data-parallel programs into vector operations

Efficient parallel execution of a high-level data-parallel language based on nested sequences, higher order functions and generalized iterators can be realized in the vector model using a suitable representation of nested sequences and a small set of transformational rules to distribute iterators through the constructs of the language.

Compiling C* programs for a hypercube multicomputer

A data parallel language such as C * has a number of advantages over conventional hypercube programming languages. The algorithm design process is simpler, because (1) message passing is invisible, (2) race conditions are nonexistent, and (3) the data can be put into a one-to-one correspondence with the virtual processors. Since data are mapped to virtual processors, rather than physical processors, it is easier to move algorithms implemented on one size hypercube to a larger or smaller system. We outline the design of a C * compiler for a hypercube multicomputer. Our design goals are to minimize the amount of time spent synchronizing, limit the number of interprocessor communications, and make each physical processor's emulation of a set of virtual processors as efficient as possible. We have hand translated three benchmark programs and compared their performance with that of ordinary C programs. All three programs—matrix multiplication, LU decomposition, and hyperquicksort—achieve reasonable speedup on a commercial hypercube, even when solving problems of modest size. On a 64-processor NCUBE/7, the C * matrix multiplication program achieves a speedup of 27 when multiplying two 64 × 64 matrices, the hyperquicksort program achieves a speedup of 10 when sorting 16,384 integers, and LU decomposition attains a speedup of 7 when decomposing a 256 × 256 system of linear equations. We believe the degradation in machine performance resulting from the use of a data parallel language will be more than compensated for by the increase in programmer productivity.