Sustained Petaflop and Beyond: Can Parallel Computing Systems Meet The Challenges?

Abstract

Summary form is only given. Breakthrough-quality scientific discoveries in the new millennium (such as those expected in computation biology and others), along with optimal engineering designs, have created a demand for High-End Computing (HEC) systems with sustained performance requirements at a petaflop scale and beyond. Despite the very pessimistic (if not negative) views on parallel computing systems that have prevailed in 1990s, there seems to be no other viable alternatives for such HEC systems. In this talk, we present a fresh look at the problems facing the design of petascale parallel computing systems. We review several fundamental issues that such HEC parallel computing systems must resolve. These issues include: execution models that support dynamic and adaptive multithreading, fine-grain synchronization, and global name-space and memory consistency. Related issues in parallel programming, dynamic compilation models, and system software design will also be discussed. Present solutions and future direction will be discussed based on (1) application demand (e.g. computation biology and others), (2) the recent trend as demonstrated by the HTMT, HPCS, and the Blue-Gene Cyclops (e.g. Cyclops-64) architectures, and (3) a historical perspective on influential models such as dataflow, along with concepts learned from these models.