Auxilliary computations for perceptron networks

Abstract

In this study we consider a multilayer perceptron network with sigmoidal activation and trained via the backpropagation algorithm. The output of all neurons is collected and a simple linear regression is performed. It is shown that untrained networks with randomly chosen coefficients perform comparably with fully trained networks. This result casts a new light on the role of activation functions, the impact of dimensionality, and the efficacy of training algorithms such as backpropagation.