An evolutionary algorithm for parallel mapping of backpropagation learning on heterogeneous processor networks

Abstract

This paper presents an evolutionary algorithm for optimal mapping of the backpropagation learning algorithm onto a parallel heterogeneous processor network. Training-set parallelism is used as the paradigm for parallelizing the backpropagation algorithm, and the processor network is a heterogeneous array of transputers connected in a pipelined ring topology. It is known from earlier studies that finding the optimal mapping (i.e. optimal allocation of training patterns among the processors to minimize the time for a training epoch) involves solving a linear Mixed Integer Programming (MIP) problem. Solving the MIP using the traditional Branch and Bound (B&B) method takes a large amount of computing time. Approaches based on evolutionary algorithms are then investigated as alternatives to the branch and bound method to solve the pattern allocation problem. It is found that a conventional genetic algorithms (GAs) search does not perform as well as the B&B in terms of the quality of solution and the search time taken. However when the crossover and mutation probabilities in the GA are varied over a wide range, the best solution is obtained by an evolutionary algorithm even though the studies were begun with a conventional GA. A new stopping criterion to detect convergence to stop the search is also incorporated in the final algorithm.