A study of English word category prediction based on neural networks

Abstract

This paper describes word category prediction based on neural network models for constructing an accurate word recognition system. It is difficult to represent hidden linguistic structure and make an N-gram word prediction model using traditional stochastic approaches. In this paper, two neural network models that can learn hidden linguistic structure are proposed. These models can easily be expanded from Bigram to N-gram networks. They were tested by training experiments with an open English text database. The Trigram word category prediction rates show that neural network models are comparable to stochastic models. Trigram neural network models compress information about 150 times, which is the ratio of the Trigram stochastic model free parameters (893 = 704 969) to the neural network model link weights (4649). In addition, this paper proposes a new method that dynamically controls the training parameters, updating step size and momentum. These techniques are effective for calculating the efficiency of this system.