Learning in Single Hidden‐Layer Feedforward Network Models: Backpropagation in a Spatial Interaction Modeling Context

Abstract

Learning in neural networks has attracted considerable interest in recent years. Our focus is on learning in single hidden‐layer feedforward networks which is posed as a search in the network parameter space for a network that minimizes an additive error function of statistically independent examples. We review first the class of single hidden‐layer feedforward networks and characterize the learning process in such networks from a statistical point of view. Then we describe the backpropagation procedure, the leading case of gradient descent learning algorithms for the class of networks considered here, as well as an efficient heuristic modification. Finally, we analyze the applicability of these learning methods to the problem of predicting interregional telecommunication flows. Particular emphasis is laid on the engineering judgment, first, in choosing appropriate values for the tunable parameters, second, on the decision whether to train the network by epoch or by pattern (random approximation), and, third, on the overfitting problem. In addition, the analysis shows that the neural network model whether using either epoch‐based or pattern‐based stochastic approximation outperforms the classical regression approach to modeling telecommunication flows.