Meta-neural networks that learn by learning

Abstract

A novel method for training neural networks is introduced. The method uses an additional observing neural network called a meta-neural network (MNN) to direct the training of the basic neural network. The MNN provides the basic neural network with a step size and a direction vector which is optimal based on successful training strategies learned from problems solved previously. The combination of the MNN with the basic neural network is shown to improve learning rates for several problems when the MNN is trained on a similar problem. The MNN is shown to help solve the problem of sensitivity to initial weight vectors. In addition, computer simulations demonstrate the improvement in the learning rate of the enhanced neural network on a 4-b parity problem, when it has been trained on a different nonlinear Boolean function. >