Clock Synchronization on a Multicomputer

Abstract

We describe hardware and software schemes for achieving precise clock synchronization on SP2 parallel system nodes. The SP2 multistage interconnection network has an unusual hardware feature, a set of distributed counters that the processor nodes may utilize for synchronizing their time–of–day clocks. We describe an algorithm for synchronizing the counters to within less than 200 nanoseconds of each other in a network of up to 512 processor nodes. This is 4–5 orders of magnitude better than what can be achieved by existing software schemes. We also describe experimental system software, calledsptimed, for synchronizing the node clocks to the Internet time of day, utilizing the synchronous counters in the SP2 network.Sptimedsynchronizes the node clocks typically within 5 μs of each other, which is up to 2–3 orders of magnitude better than could be achieved by previous methods on the SP2 system. Synchronized clocks are useful in parallel and distributed environments, for example for performance measurement, tuning, tracing, debugging, gang scheduling of parallel processes, and timestamping of transactions. We also measure the performance of a widely used time synchronization utility, the Network Time Protocol, using the synchronous counters of the SP2 interconnection network.