A modified backpropagation algorithm for training neural networks on data with error bars

Abstract

A method is proposed for training multilayer feedforward neural networks on data contaminated with noise. Specifically, we consider the case that the artificial neural system is required to learn a physical mapping when the available target values for the output variable are subject to experimental uncertainties, but are characterized by error bars. The proposed method, based on a maximum-likelihood criterion for parameter estimation that allows for nonzero model error, introduces two simple modifications of the on-line backpropagation learning algorithm: 1. (i) The differential reliability of individual training patterns is taken into account by multiplying the learning-rate constant with a factor [1 + ( σ r σ ) 2] −1 , where σ r is the experimental uncertainty associated with data point r and σ is the current estimate of the model error parameter. 2. (ii) After each full pass through the training sample, the model error measure σ is updated by solution of a nonlinear algebraic equation, at the current values of the connection weights. The extended backpropagation algorithm is tested on two problems related to the modeling of atomic-mass systematics by neural networks, the most extensive study being performed for data generated by the liquid-drop mass formula and deliberately obscured by additive Gaussian noise. Provided the underlying mapping is reasonably smooth, neural nets trained with the new procedure are able to learn the true function to a good approximation even in the presence of high levels of noise.