Structural learning with forgetting

Abstract

It is widely known that, despite its popularity, back propagation learning suffers from various difficulties. There have been many studies aiming at the solution of these. Among them there are a class of learning algorithms, which I call structural learning, aiming at small-sized networks requiring less computational cost. Still more important is the discovery of regularities in or the extraction of rules from training data. For this purpose I propose a learning method called structural learning with forgetting. It is applied to various examples: the discovery of Boolean functions, classification of irises, discovery of recurrent networks, prediction of time series and rule extraction from mushroom data. These results demonstrate the effectiveness of structural learning with forgetting. A comparative study on various structural learning methods also supports its effectiveness.