Linkage of XcalableMP and Python languages for high productivity on HPC cluster system

Abstract

When developing applications on high-performance computing (HPC) cluster systems, Partitioned Global Address Space (PGAS) languages are used due to their high productivity and performance. However, in order to more efficiently develop such applications, it is also important to be able to combine a PGAS language with other languages instead of using a single PGAS language alone. We have designed an XcalableMP (XMP) PGAS language, and developed Omni Compiler as an XMP compiler. In this paper, we report on the development of linkage functions between XMP and {C, Fortran, or Python} for Omni Compiler. Furthermore, as a functional example of interworking between XMP and Python, we discuss the development of an application for the Graph Order/degree problem. Specifically, we paralleled all of the shortest paths among the vertices searches of the application using XMP. When the results of the application in XMP and the original Python were compared, we found that the performance of XMP was 21% faster than that of the original Python on a single CPU core. Moreover, when applying the application on an HPC cluster system with 1,280 CPU cores of 64 compute nodes, we could achieve a 921 times better performance than that on a single CPU core.