A Conjugate Gradient-Based Efficient Algorithm for Training Single-Hidden-Layer Neural Networks

Abstract

A single hidden layer neural networks (SHLNNs) learning algorithm has been proposed which is called Extreme Learning Machine (ELM). It shows extremely faster than typical back propagation (BP) neural networks based on gradient descent method. However, it requires many more hidden neurons than BP neural networks to achieve assortive classification accuracy. This then leads more test time which plays an important role in practice. A novel learning algorithm (USA) for SHLNNs has been presented which updates the weights by using gradient method in the ELM framework. In this paper, we employ the conjugate gradient method to train the SHLNNs on the MNIST digit recognition problem. The simulated experiment demonstrates the better generalization and less required hidden neurons than the common ELM and USA.