Use of methodological diversity to improve neural network generalisation

Abstract

Littlewood and Miller [4] present a statistical framework for dealing with coincident failures in multiversion software systems. They develop a theoretical model that holds the promise of high system reliability through the use of multiple, diverse sets of alternative versions. In this paper, we adapt their framework to investigate the feasibility of exploiting the diversity observable in multiple populations of neural networks developed using diverse methodologies. We evaluate the generalisation improvements achieved by a range of methodologically diverse network generation processes. We attempt to order the constituent methodological features with respect to their potential for use in the engineering of useful diversity. We also define and explore the use of relative measures of the diversity between version sets as a guide to the potential for exploiting interset diversity.