Thoughts on high-performance computing

Abstract

Parallel computing is the main technical approach for achieving very high performance computing. In the history of parallel computing, there have been threephases, i.e.moderateparallelismdescribed by Amdahl’s law [1], large-scale parallelism described by Gustafson’s law [2], andhigh-productivity parallelismdescribed by the productivity evaluation model [3]. In April 2010, IBM Inc. in their report ‘Some Challenges on Road fromPetascale to Exascale’ presented five challenges in an exascale system; these stem from power consumption, memory access, communication, reliability, and programming [4], respectively referred to as the energy wall, memory wall, communication wall, reliability wall, and programming wall. Faced with the challenges of ‘walls’, we investigate wall measurement models at the scientific level. For example, existing reliability theories, such as probability theory, do not consider the effect of reliability on performance, while the classic speedup model does not reflect the relation between performance and reliability. To incorporate reliability and performance into a unified measurement model, we measure reliability based on fault-tolerant overhead. As current faulttolerant techniques include a certain time overhead,we created a reliability speedup model with fault tolerance to measure the effect of fault-tolerant overhead on speedup: