Abstract
Many applications in high performance computing are designed based on underlying performance and execution models. While these models could successfully be employed in the past for balancing load within and between compute nodes, modern software and hardware increasingly make performance predictability difficult if not impossible. Consequently, balancing computational load becomes much more difficult. Aiming to tackle these challenges in search for a general solution, we present a novel library for fine-granular task-based reactive load balancing in distributed memory based on MPI and OpenMP. With our approach, individual migratable tasks can be executed on any MPI rank. The actual executing rank is determined at run time based on online performance data. We evaluate our approach under an enforced power cap and under enforced clock frequency changes for a synthetic benchmark and show its robustness for work-induced imbalances for a realistic application. Our experiments demonstrate speedups of up to 1.31X.
Highlights
Over the past decades, most scientific applications have been developed under the assumption of a homogeneous execution environment where every compute node – and even every single core – in a larger cloud or High Performance Computing (HPC) system has a constant equal speed
We present our library implementation based on MPI+OpenMP that allows an incremental integration into existing task-based applications with minimal programming efforts
All tests are conducted on the HPC production system of RWTH Aachen University CLAIX that is equipped with an Intel Omni-Path interconnect and dual-socket Intel Xeon E5-2650v4 processor nodes with a TDP of 105 W
Summary
Most scientific applications have been developed under the assumption of a homogeneous execution environment where every compute node – and even every single core – in a larger cloud or High Performance Computing (HPC) system has a constant equal speed. Executing the same work on every node should require the same computation time In the past, this execution model was shown to be highly accurate and efficient for balancing computational load. This execution model was shown to be highly accurate and efficient for balancing computational load As both hardware and software become increasingly complex, this model might no longer be sufficient on current and future systems. CPU power efficiency variations arising from the manufacturing process can lead to performance variations in presence of an enforced power cap [12] Another source of dynamic variability stems from modern numerics in simulation applications such as particle simulations or iterative codes employing adaptive mesh refinement. Traditional approaches like global re-partitioning of work were an effective technique to ensure proper balance in the past They are based on a cost model to predict future execution time. This paper makes the following contributions: 1. We present the first conceptual generalization of reactive load balancing to arbitrary MPI-parallel task-based applications, detailing both requirements and limitations associated with it
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
