Segmenting large traces of inter-process communication with a focus on high performance computing systems

Abstract

The understanding of the interactions among processes of a High Performance Computing (HPC) system can be made easier if trace analysis is used. Traces, however, can be quite large, making it difficult to analyze their content unless some abstraction is provided. This paper presents a novel trace abstraction approach that aims to facilitate the analysis of large execution traces generated from HPC applications. Our approach allows automatic segmentation of large traces into smaller and meaningful clusters that reflect the various execution phases of the traced scenarios. Our approach is based on the application of information theory principles to the analysis of sequences of communication patterns extracted from traces of HPC systems. This work is inspired by recent studies in the field of bioinformatics where several techniques have been proposed to segment DNA sequences into homogeneous sub-domains, where each sub-domain exhibits a certain degree of internal homogeneity. Trace segments can be used in a number of applications such as recovering high-level views of the system behavior and program understanding. We demonstrate the usefulness of our approach by applying it to different traces of hundreds of millions of events, generated from two HPC systems.