Locality-Aware Memory Association for Multi-Target Worksharing in OpenMP

Abstract

Heterogeneity is an ever-growing challenge in computing. The clearest example is the increasing popularity of GPUs, and purpose-designed coprocessors such as Intel Xeon Phi. Even disregarding coprocessors, heterogeneity continues to increase with the rise in CPU core counts, adaptive per-core frequencies, and increasingly hierarchical and complex memory systems. Take a system with four memory nodes, associated with four cores each, and four GPUs, each with a distinct address space and tens to hundreds of cores pro­grammed like a bulk-synchronous parallel cluster. In this case, we are effectively programming clusters of miniature constellations in every node.