Software Threads Research Articles

Asynchronous software thread integration for efficient software

The overhead of context-switching limits efficient scheduling of multiple concurrent threads on a uniprocessor when real-time requirements exist. Existing software thread integration (STI) methods reduce context switches, but only provide synchronous thread progress within integrated functions. For the remaining, non-integrated portions of the secondary threads to run and avoid starvation, the primary thread must have adequate amounts of coarse-grain idle time (longer than two context-switches). We have developed asynchronous software thread integration (ASTI) methods which address starvation through the efficient use of coroutine calls and integration. ASTI allows threads to make independent progress efficiently and reduces the number of context switches needed through integration.Software-implemented protocol controllers are crippled by this problem; the primary thread "bit-bangs" each bit of a message onto or off of the bus, leaving only fragments of idle time shorter than a bit time. This fragmented time may be too short to recover through context switching, so only the primary thread can execute during message transmission or reception, slowing the secondary threads and potentially making them miss their deadlines. ASTI simplifies the implementation of embedded communication protocols on low-cost, moderate speed (1 - 100 MHz, 8- and 16-bit) microcontrollers. We demonstrate ASTI by replacing a standard automotive communication protocol controller (J1850) with software and generic hardware. Secondary thread performance improves significantly when compared with a traditional interrupt-based software approach.

The symmetric tridiagonal eigenproblem on a shared memory multiprocessor: Part II

The article presents an experimental evaluation of the effect of deflation on accuracy. A number of test matrices were used to test Cuppen's algorithm since the amount of deflation in this method depends on the test matrix. The motivation for a non-recursive version was to compare this with the recursive version and to investigate the relationship between the recursive and non-recursive implementations of Cuppen's divide-and-conquer method. Empirical comparison was carried out using C++ and the THREADS package on a shared memory Encore Multimax multiprocessor. The algorithms illustrate the advantages of parallel algorithms using dynamic allocation of tasks to THREADs on this shared memory machine.

Capriccio

This paper presents Capriccio, a scalable thread package for use with high-concurrency servers. While recent work has advocated event-based systems, we believe that thread-based systems can provide a simpler programming model that achieves equivalent or superior performance.By implementing Capriccio as a user-level thread package, we have decoupled the thread package implementation from the underlying operating system. As a result, we can take advantage of cooperative threading, new asynchronous I/O mechanisms, and compiler support. Using this approach, we are able to provide three key features: (1) scalability to 100,000 threads, (2) efficient stack management, and (3) resource-aware scheduling.We introduce linked stack management , which minimizes the amount of wasted stack space by providing safe, small, and non-contiguous stacks that can grow or shrink at run time. A compiler analysis makes our stack implementation efficient and sound. We also present resource-aware scheduling , which allows thread scheduling and admission control to adapt to the system's current resource usage. This technique uses a blocking graph that is automatically derived from the application to describe the flow of control between blocking points in a cooperative thread package. We have applied our techniques to the Apache 2.0.44 web server, demonstrating that we can achieve high performance and scalability despite using a simple threaded programming model.

A comparative analysis of fine-grain threads packages

The rising availability of multiprocessing platforms has increased the importance of providing programming models that allow users to express parallelism simply, portably, and efficiently. One popular way to write parallel programs is to use threads for concurrent sections of code. User-level threads packages allow programmers to implement multithreaded programs in which thread creation, thread management, and thread synchronization are relatively inexpensive. Fine-grain programs are multithreaded programs in which the work is divided into a large number of threads, where each thread contains a relatively small amount of work. The potential benefit of large numbers of threads include easier load balancing, better scalability, greater potential for overlapping communication and computation, and improved platform-independence. However, fine-grain programs are largely considered inefficient due to the overheads involved in managing numerous threads. In this paper, we survey several thread packages that take different approaches to the problem of efficiently supporting the creation and management of large numbers of fine-grain threads. Each package is compared based on its level of support of the general thread model as well as its performance on a set of fine-grain parallel programs. We find that while the thread packages we tested may support medium-grain parallelism efficiently, they do not always support fine-grain parallelism. Although no package supports fine-grain parallelism and a general thread model, we believe that it can potentially be done with help from the compiler.

Pointer and escape analysis for multithreaded programs

This paper presents a new combined pointer and escape analysis for multithreaded programs. The algorithm uses a new abstraction called parallel interaction graphs to analyze the interactions between threads and extract precise points-to, escape, and action ordering information for objects accessed by multiple threads. The analysis is compositional, analyzing each method or thread once to extract a parameterized analysis result that can be specialized for use in any context. It is also capable of analyzing programs that use the unstructured form of multithreading present in languages such as Java and standard threads packages such as POSIX threads. We have implemented the analysis in the MIT Flex compiler for Java and used the extracted information to 1) verify that programs correctly use region-based allocation constructs, 2) eliminate dynamic checks associated with the use of regions, and 3) eliminate unnecessary synchronization. Our experimental results show that analyzing the interactions between threads significantly increases the effectiveness of the region analysis and region check elimination, but has little effect for synchronization elimination.

Constructing generic data-farm templates

The data-farm template is a software component for the pipelined processor farm (PPF) software architecture. The PPF data-farm template is a flexible, semi-manual method of prototyping continuous-flow applications. The template design originates in a set of abstract design principles. The PPF template is instrumented to provide either support for logic debugging at an intermediate development level on a network of workstations (NOW) or performance tuning on a target multicomputer. General implementational problems arise. The paper demonstrates the feasibility of implementing the design on a Unix-based NOW. Sufficient detail is included to guide other implementations and to make comparison between thread packages. Descriptions of two target system implementations follow. Copyright © 1999 John Wiley & Sons, Ltd.

An experimental testbed for embedded real time Ada 95

The modifications made to Ada during the 9X process have resulted in a language that is ideally suited to programming real-time systems. In this paper we investigate the difficulties in realising this potential. In particular, we consider the issues raised when porting the public GNAT system on to a bare processor and producing a predictable and effective run-time system. As we were also concerned with issues of distribution via a CAN broadcast bus, support for the CAN protocol was included in the run-time system. In addition to investigating the performance of real-time Ada 95 applications we were also interested in more general issues associated with embedded kernel support for Ada 95. To facilitate these investigations and obtain the required level of performance, the thread package via which GNAT implement various aspects of the Ada language has been re-implemented. We discuss the architecture selected for this embedded kernel implementation and its relation to the architecture of the GNAT compiler.

Predictable threads for dynamic, hard real-time environments

Next-generation, hard real-time systems will require new, flexible functionality and guaranteed, predictable performance. This paper describes the UMass Spring threads package, designed specifically for multiprocessing in dynamic, hard real-time environments. This package is unique because of its support for new thread semantics for real-time processing. Predictable creation and execution of threads is achieved because of an underlying predictable kernel, the UMass Spring kernel. Design decisions and lessons learned while implementing the threads package are presented. Measurements affirm the predictability of this implementation on a representative multiprocessor platform. The adoption of the threads package in the UMass Spring kernel results in additional performance improvements, which include reduced context switching overhead and reduced average-case memory access durations.

High Performance Thread Migration on Clusters of SMPs

Thread migration is established as a mechanism for achieving dynamic load sharing and data locality. However, migration has not been used with fine-grained parallelism on workstation clusters due to the relatively high overheads associated with thread and messaging packages. This paper describes a portable, high performance thread migration system, implemented with user level threads and active messages. The thread system supports an extensible event mechanism which permits an efficient interface between the thread and active message system. Migration is supported by user level primitives; applications may implement different migration policies on top of the migration interface provided. The system runs on a cluster of SMPs and the performance obtained is orders of magnitude better than other reported measurements.

Reverse engineering of software threads: A design recovery technique for large multi-process systems

Many software systems currently being deployed in both military and industrial domains serve their users through a large number of concurrent processes. Such systems are expensive and time consuming to build and thus cannot be lightly discarded when circumstances change. Instead, the systems must be able to evolve to meet new challenges. But a software system cannot evolve without an understanding of its design, and such understanding is easily lost when the developers disperse. A particularly difficult problem may be to understand how the many processes cooperate to produce each functionality that the user needs. This paper presents a novel, but relatively simple method for using traces of inter-process messages to recover functional software design threads from a large, multi-process system. The method is illustrated with a preliminary case study of the Joint STARS battle management software. The method seems to be effective in identifying the “as built” threads, and in providing unexpected insights even to Software Engineers who are relatively familiar with the system. Design thread recovery for Joint STARS is feasible because instrumentation to capture inter-process messages was designed into the original system. Our results suggest that the designers of new systems should give thought to including such instrumentation since the cost is small and the future economic benefits in easier evolution could be very large.

Towards High-Performance Multithreaded CORBA Servers

Parallel platforms have become widely available. Moderately priced commodity SMPs are now manufactured by most major hardware vendors. Platform independent software environments, emphasizing a transparent programming model for building distributed applications, are rapidly emerging. In this paper we demonstrate how to combine the transparency characteristics of these environments with the high-performance features of the affordable server technology. We integrate the thread-per-request concurrency model into CORBA servers while providing high-performance. We demonstrate that thread creation overhead can be minimal and is merely attribute to the thread package used. We introduce and evaluate optimization techniques for increasing overall server performance. These techniques are based on increasing locality of reference for the client-server interaction.

Support for software interrupts in log-based rollback-recovery

The piecewise deterministic execution model is a fundamental assumption in many log-based rollback-recovery protocols. Process execution in this model consists of intervals, each starting with the receipt of a message at an application-defined execution point. Execution within each interval is deterministic and messages are the only source of nondeterminism that affects the computation. This simple model excludes the nondeterminism that results when asynchronous signals or interrupts occur at arbitrary execution points. As a result, a wide range of applications cannot use log-based rollback-recovery in practice. We present a solution that removes this restriction and allows applications to replay interrupts at the same execution points during recovery. The solution relies on using a software counter to compute the number of instructions between the asynchronous signals during normal operation. Should a failure occur, the instruction counts are used to force the replay of these signals at the same execution points. The execution of the application thus can be replayed to recreate the prefailure state while accommodating nondeterminism due to asynchronous signals. We then use the deterministic replay of interrupts to solve another problem, namely tracking nondeterminism due to interleaved shared memory access in multithreaded applications on a single processor. We use the instruction counter solution to implement a user-level thread package in which thread scheduling decisions can be replayed if a failure occurs. By repeating the scheduling decisions during an execution replay, threads access the shared memory in the same order and the execution to be reconstructed. This technique allows multithreaded applications to use log-based rollback-recovery with low overhead, which was not previously possible. We carried out two prototype implementations that have shown the overhead is no more than a 6 percent slowdown in application execution on the DEC Alpha, and from 6 percent to 18 percent on the Inter Pentium. Thus, restrictions of the piecewise deterministic execution model can be lifted at a reasonable cost.

An open implementation analysis and design for lightweight threads

Open Implementation Analysis and Design (OIA/D) has been introduced as a design methodology for object-oriented software systems, and in particular for substrate software. In this paper we detail our experiences with using OIA/D to design and implement a common substrate component for parallel language runtime systems: a lightweight threads package. We show how existing thread packages employ a "black-box" design, hiding crucial design decisions that drastically reduce their ability to be re-used. We detail these design decisions (dilemmas) and show how an implementation based on OIA/D principles results in a thread package that is flexible, efficient, portable, and re-usable.

Reflection of developing user-level real-time thread packages

In multimedia environments, applications use continuous-media data, such as video data and audio data, to provide users with a wide variety of interaction schemes with computers. Continuous-media applications require more efficient and flexible support from real-time threads than such research real-time systems. To fulfill such requirements, we have been developing user-level real-time threads, which are real-time threads implemented at the user level.We have implemented two user-level real-time thread packages on different operating system platforms, ARTS and RT-Mach. There were several issues, which we considered to design their architectures, such as compatibility, performance, correctness and accuracy. This paper describes the design issues, the lessons we have learned, and the discussions on their development.

Optimistic active messages

Low-overhead message passing is critical to the performance of many applications. Active Messages reduce the software overhead for message handling: messages are run as handlers instead of as threads, which avoids the overhead of thread management and the unnecessary data copying of other communication models. Scheduling the execution of Active Messages is typically done by disabling and enabling interrupts, or by polling the network. This primitive scheduling control, combined with the fact that handlers are not schedulable entities, puts severe restrictions on the code that can be run in a message handler. This paper describes a new software mechanism, Optimistic Active Messages (OAM), that eliminates these restrictions; OAMs allow arbitrary user code to execute in handlers, and also allow handlers to block. Despite this gain in expressiveness, OAMs perform as well as Active Messages. We used OAM as the base for an RPC system, Optimistic RPC (ORPC), for the Thinking Machines CM-5 multiprocessor; it consists of an optimized thread package and a stub compiler that hides communication details from the programmer. ORPC is 1.5 to 5 times faster than traditional RPC (TRPC) for small messages and performs as well as Active Messages (AM). Applications that primarily communicate using large data transfers or are fairly coarse-grained perform equally well, independent of whether AMs, ORPCs, or TRPCs are used. For applications that send many short messages, however, the ORPC and AM implementations are up to three times faster than the TRPC implementations. Using ORPC, programmers obtain the benefits of well-proven programming abstractions such as threads, mutexes, and condition variables, do not have to be concerned with communication details, and yet obtain nearly the performance of hand-coded Active Message programs.

A machine independent interface for lightweight threads

Recently, lightweight thread libraries have become a common entity to support concurrent programming on shared memory multiprocessors. However, the disparity between primitives offered by operating systems creates a challenge for those who wish to create portable lightweight thread packages. What should be the interface between the machine-independent and machine-dependent parts of the thread library? We have implemented a portable lightweight thread library on top of Unix on a KSR-1 supercomputer, BBN Butterfly multiprocessor, SGI multiprocessor, Sequent multiprocessor and Sun 3/4 family of uniprocessors. This paper first compares the nature and performance of the OS primitives offered by these machines. We then present procedure-level abstraction that is efficiently implementable on all the architectures and is a sufficient base upon which a user-level thread package can be built.

Procs and locks

We have built a portable platform for running Standard ML of New Jersey programs on multiprocessors. It can be used to implement user-level thread packages for multiprocessors within the ML language with first-class continuations. The platform supports experimentation with different thread scheduling policies and synchronization constructs. it has been used to construct a Modula-3 style thread package and a version of Concurrent ML, and has been ported to three different multiprocessors running variants of Unix. This paper describes the platform's design, implementation, and performance.

Parallelising a financial system

This paper reports on the parallelising of a financial system that calculates actuarial liabilities. The purpose of such a system is to evaluate the liabilities of a pension scheme by calculating the expected cost of each member to the system in the future. An analysis of the system has been carried out and two levels of parallelism have been detected. From the more coarse-grained of these levels a parallel model has been proposed. A parallel simulation of this model has been developed both on a Meiko Computing Surface distributed store machine populated with Inmos transputers and Intel i860 microprocessors, and on a network of workstations to enable detailed evaluation of the potential speedup. Figures are given and various metrics are used to analyse the speedup and efficiency factors. The potential for making use of virtual shared memory and lightweight thread packages at this coarse-grained level of parallelism is considered. The more fine-grained level of parallelism identified is discussed with a view to massively parallel architectures.

Toward Dynamic. Multiprocessor Real-Time Threads

Publisher Summary This chapter describes some of the research issues and solutions surrounding the development of a real-time threads package for multiprocessor systems. The package involved is based on Mach threads and has been implemented on standard UNIX platforms and on 32-node BBN Butterfly multiprocessor. It has also been used as a portable basis for the construction of parallel real-time applications and multiprocessor operating system kernels. The essential idea underlying real-time threads is that the schedulability of a thread should be guaranteed before it is actually run, not only at the time of program compilation but also at the time of thread creation. Specifically, the real-time threads package guarantees the schedulability of threads being created prior to their execution, assuming that thread deadlines, start times, and maximum execution times are known. The second main idea underlying real-time threads is that minimum functionality should be offered by the threads package itself. Specifically, when scheduling, guarantees cannot be made by the package's underlying algorithms.

Scheduler activations

Threadsare the vehicle for concurrency in many approaches to parallel programming. Threads can be supported either by the operating system kernel or by user-level library code in the application address space, but neither approach has been fully satisfactory.This paper addresses this dilemma. First, we argue that the performance of kernel threads isinherentlyworse than that of user-level threads, rather than this being an artifact of existing implementations; managing parallelism at the user level is essential to high-performance parallel computing. Next, we argue that the problems encountered in integrating user-level threads with other system services is a consequence of the lack of kernel support for user-level threads provided by contemporary multiprocessor operating systems; kernel threads are thewrong abstractionon which to support user-level management of parallelism. Finally, we describe the design, implementation, and performance of a new kernel interface and user-level thread package that together provide the same functionality as kernel threads without compromising the performance and flexibility advantages of user-level management of parallelism.