Hiding shared memory reference latency on the Galactica Net distributed shared memory architecture

Abstract

In order to provide shared memory in large-scale multiprocessors, techniques to hide the latency of shared memory accesses must be developed. In this paper, we describe the latency hiding mechanisms employed by the Galactica Net scalable distributed shared memory architecture being developed at the Center for High Performance Computing. We introduce our novel technique for maintaining the coherence of shared data caches, based on a flexible hardware-supported but software-controlled mechanism supporting both update and invalidate based protocols. We also consider the use of alternative memory consistency models, and find that the use of weaker consistency models is an effective way to hide memory reference latency in the Galactica Net architecture. Preliminary performance evaluations indicate that together these mechanisms are able to hide a significant amount of the memory reference latency, thus increasing the scalability of the architecture.