Parallel implementation of evolutionary strategies on heterogeneous clusters with load balancing

Abstract

This paper presents a load balancing algorithm for a parallel implementation of an evolutionary strategy on heterogeneous clusters. Evolutionary strategies can efficiently solve a diverse set of optimization problems. Due to cluster heterogeneity and in order to improve the speedup of the parallel implementation a load balancing algorithm has been implemented. This load balancing algorithm takes into account cluster heterogeneity and it is based on an optimal initial distribution. This initial distribution is determined based on the cluster nodes' computational powers that are dynamically measured in each slave node by an ad hoc load-benchmark. The implementation presents very satisfactory parallelization results, both in performance and scalability and super-linear speedup is reached for several tests configurations. Experimental results show excellent performance, increasing the improvements with the load balancing algorithm.