High throughput computing for neural network simulation

Abstract

Publisher Summary Artificial neural networks (ANNs) are densely interconnected networks of processing nodes that provide robust methods for learning real, discrete, and vectored valued functions from example training sets and have been widely applied to problems in control, prediction and pattern recognition. Processing nodes are organized into layers of input units Ni, layers of hidden units Nh and a layer of output units No. Interconnections between processing units are represented by a matrix of adjustable weight parameters W, which can be tuned by a gradient decent algorithm to learn functions presented by the training set data. However, neural network learning is computationally expensive and training can take the order of days or weeks for large training sets. Training sets for applications, such as character recognition and speech recognition can require the order of 106 training samples and 106 network parameters. This chapter discusses these computational requirements by exploiting neural network parallelism in a high throughput meta computing environment.