Theory of Time Domain Ensemble On-Line Learning of Perceptron under the Existence of External Noise

Abstract

We analyze the time domain ensemble on-line learning of a Perceptron under the existence of external noise. We adopt three typical learning rules, Hebbian, Perceptron, and AdaTron rules. We treat the input and output noises. In order to improve the learning when it does not succeed in the sense that the student vector does not converge to the teacher vector, we use an averaging method and give theoretical analysis of the method. We obtain the precise formula for the overlap between the teacher vector and the time averaged student vector for t !1 limit as a function of the number of student vectors to be averaged. We compare the theoretical results with numerical simulations and find that the theoretical results agree quite well with the numerical simulations.