Load Balancing and Performance Issues for Data Parallel

Abstract

A data parallel program is presented that solves the reactive Euler equations for stiff chemical nonequilibrium ows on Connection Machines CM-2/200 and CM-5/5E. The program is written in CM Fortran and uses direction and time-step splitting to couple representations of the chemical and uid dynamicprocesses on a structured Cartesian grid.An explicit high-ordermonotonealgorithmwith nonlineardamping is used to integrate the convection terms, and a hybrid asymptotic/modi® ed-Euler approach is used to solve the system of ordinary differential equations from the chemical source terms. Integration of the uid dynamics was conservatively determined to be 9.4 and 12.0 G ops on a 512-node CM-5 and CM-5E, respectively. The uid dynamics solver scaled well for large problems; however, both the performance and the scaling are signi® cantly affected by the nearest-neighbor communications, which accounted for at least 24% of the execution time on the CM-5. For the integration of the chemistry source terms, poor load balancing signi® cantly affected performance of the program. Therefore, a new load-balancing algorithmwas developed that reduces the chemistry integration time by a factor of six for the test problem, a detonation propagating in a hydrogen± oxygen± argon mixture. Moreover, the chemistry integration time, with the load balancing, is slightly less than the time required to integrate the uid dynamics. As a result, an ef® cient data parallel program for solving stiff chemical nonequilibrium ows is available for problems that were too expensive to solve in the past.