Encoding feedforward networks for topology optimization by simulated evolution

Abstract

While in the past years enhanced training algorithms for multilayer networks were devised, the construction of application specific network topologies, by comparison, remains something like a black art. Construction rules for determining an application specific network topology, e.g. the number of hidden units, the number of layers and the interconnection patterns between the network components, do not exist. Usually, the topology of the networks is optimized manually by trial and error and it takes up a lot of time and effort from the human designer. Due to the tremendous complexity of the problem, it is useful to look for automatic search procedures that scan the space of possible topologies in order to find application specific network graphs. Simulated evolution is well suited to this task: it replicates intermediate solutions by evolutionary operators (mutation, crossover) before evaluating the newly created solution. The fitness of a solution controls the chance of becoming selected as the parent of more enhanced solutions found later on. Because of its relationship to the evolutionary operators the network encoding (chromosomes) is essential for the simulated evolution. In order to restrict the search space to useful topologies one has to choose compact and complete coding schemes. In the talk, I present a general framework for such efficient encoding schemes, developed in the GANNAS (Genetic Artificial Neural Network Synthesis) project. Based on GANNAS, several application specific networks were bred on a big cluster computing machine providing the necessary computing power.