Scaling shared-bus multiprocessors with multiple buses and shared caches: a performance study

Abstract

The main limitation of shared-bus multiprocessors is that the common bus tends to be the primary source for contention, and thus imposes a limit on the number of processors in the system. Alternative architectural features are necessary to reduce the memory bandwidth demands and to increase the bus bandwidth. In this paper, we investigate the cost-performance effects of two enhancements: higher bus transaction rates, e.g., through the use of multiple buses, and shared two-level caches. The performance figures are obtained via simulation with loads derived from traces of real applications, some of which show a significant skew in the distribution of memory bank access. A new multiple bus scheme, called multiple interleaved buses, is described and analysed. This scheme is a generalization of previous approaches, and attempts to balance performance and cost trade-offs in a snoopy-cache multiprocessor environment. The results from simulation show that multiple interleaved buses perform almost as well as multiple independent buses, but with simpler and less costly implementation. Furthermore, multiple interleaved buses are shown to deliver much better performance than interleaved buses when the skew of accesses across the interleaves is large. Shared second-level caches have been shown to be very effective in the design space under consideration. Such systems might offer considerable implementation economies with relatively small design cost. We show that depending on the design point in question, bus operation buffers might be useful in shared second level caches by reducing the effects of high skew and greater multiprocessing level. With the presence of these buffers, the uses of shared caches resulted in only a small throughput degradation.