High-level abstractions for efficient concurrent systems

Abstract

Parallel symbolic algorithms exhibit characteristics that make their efficient implementation on current multiprocessor platforms difficult: data is generated dynamically and often have irregular shape and density, data sets typically consist of o bjects of many different types and structure, the natural unit of concurrency is often mu ch smaller than can be efficiently supported on stock hardware, efficient scheduling, migr ation and load-balancing strategies vary widely among different algorithms, and sen sible decomposition of the program into parallel threads of control often cannot be achieved by mere examination of the source text. Many of these challenges are also faced by implementors of multi-threaded operating systems and kernels[2, 23, 29]. In both cases, the utility of an implemen tation is determined by how well it supports a diverse range of applications in terms of performance and programmability. Implementations that hard-wire decisions about process granularity, sch eduling, and data allocation often perform poorly when executing parallel symbolic alg orithms. An alternative approach to building concurrent systems would permit a syst em’s internals to be customized by users without sacrificing performance or security. The generality and expressivity of the abstractions used by an implementation dictate the e xtent to which it may be customized. Because different applications will exercise different operations in diff erent ways, systems which permit customization of these operations on a per-application basis offer the promise of greater flexibility and programmability. First-class procedures[1, 21] and continuations[13, 28] are two abstractions we have found to be extremely useful in implementing a variety of environment and control operations. Languages such as Scheme[10] or ML[25] have demonstrated that these abstractions are effective building blocks for expressing a number of interesting data, program and control structures. It is often claimed, however, that such high-level program constructs are too inefficient to serve as a foundation for building efficient high-performance concurrent systems. In this paper, we present evidence to the contrary. Sting is a highly efficient and flexible operating system implemented in Scheme. It provides mechanisms to (a) create Appeared in the Proceedings of the International Conference on Programming Languages and Systems Architecture, March 1994.