Biasing the evolution of modular neural networks

Abstract

Modularity is known to have benefits for neural systems and their evolution, and this paper aims to improve the evolutionary neural network algorithm EPNet to take advantage of those benefits. Neural networks exist with varying degrees of modularity ranging from pure modular networks characterized by disjoint partitions of hidden nodes with no communication between modules, to pure homogeneous networks with significant connections throughout. In between are apparently homogeneous networks that can be seen to have some degree of modularity if the hidden nodes are reorganized appropriately. In this paper, a modularity measure is presented and extended that can be applied to any neuron at any level in the network to provide a fine analysis of node partitioning. It also allows the rearrangement of nodes to create modules in homogeneous networks, and that is used to improve the EPNet algorithm to evolve modular neural networks. Experimental results on a simple classification task confirm that the new modular EPNet algorithm does indeed lead to more modular networks than the classical EPNet algorithm, without compromising the performance on the given task.