Online Learning through Moving Ensemble Teachers — An Exact Solution of a Linear Model —

Abstract

Listen

Translate article

Since a model in which a student learns from two or more teachers who themselves are learning has a certain similarity with actual human society, the analysis of such a model is interesting. In this paper, a model composed of a true teacher, multiple moving ensemble teachers existing around the true teacher, and a student, which are all linear perceptions, is analyzed using the statistical–mechanical method in the framework of on-line learning. The dependences of the generalization performance on the ensemble teachers’ learning rate, the student’s learning rate, and the number of ensemble teachers are clarified. Furthermore, it is shown that the generalization error can be reduced to the lower bound in the case of moving ensemble teachers, while there are unattainable generalization errors in the case of stationary ensemble teachers. These results show that it is important for teachers to continue learning in order to educate students.