Workload Characteristics of a Multi-cluster Supercomputer

Abstract

AbstractThis paper presents a comprehensive characterization of a multi-cluster supercomputer workload using twelve-month scientific research traces. Metrics that we characterize include system utilization, job arrival rate and interarrival time, job cancellation rate, job size (degree of parallelism), job runtime, memory usage, and user/group behavior. Correlations between metrics (job runtime and memory usage, requested and actual runtime, etc) are identified and extensively studied. Differences with previously reported workloads are recognized and statistical distributions are fitted for generating synthetic workloads with the same characteristics. This study provides a realistic basis for experiments in resource management and evaluations of different scheduling strategies in a multi-cluster research environment.KeywordsMemory UsageInterarrival TimeSystem UtilizationAverage Arrival RateWorkload CharacteristicThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.