Interconnection network organization and its impact on performance and cost in shared memory multiprocessors

Abstract

Interconnection network organization has a significant impact on multiprocessor system performance. However, detailed and comprehensive studies of the performance and the effect of different organizations are virtually non-existent. In this study, three interconnection network organizations are evaluated as part of a cache-based shared memory multiprocessor system: tori, multistage and single-stage shuffle-exchange networks. A system size is limited to 256 or fewer processors, a range where wiring constraints can be largely ignored. The performance impact of topology choice and switch size and channel width are studied under three different constraints: fixed switch size and channel width, constant number of switch pins, and constant network cost. The cost model reflects switch size and channel width. We find that after a certain point the performance advantage of wider channels becomes small, and the network topology and switch size become the determining parameters. Our results show that the multistage network is the best network topology if cost is not the main limiting factor. Otherwise, the single-stage network is the most cost-effective network topology. 2-D torus networks are seriously limited in terms of performance and cost. The dimension of torus networks needs to be larger than 2 in order to have a reasonable relative performance vis-à-vis networks of other topologies.