Training recurrent neural networks to learn lexical encoding and thematic role assignment in parsing Mandarin Chinese sentences

Abstract

In this paper, we used the extended backpropagation learning method on the recurrent networks to learn lexical encodings and thematic role assignment tasks in parsing Mandarin Chinese sentences. In order to objectively evaluate the learning performance, the training and test sentences are automatically generated from tentatively designed sentence templates. Three learning experiments were carried out. In the first two experiments, we used a small lexicon of 33 words to observe the learned encodings and learning performances. 75 training sentences and 6503 test sentences were used for the supervised learning on a simple recurrent network (SRN) and a modified SRN (MSRN), respectively, in the first experiment. The experimental results showed the MSRN performed better than the SRN. In the second experiment, we focused on the investigation of the learning of thematic role assignment of sentences with embedded clause. 100 training sentences and 6609 test sentences were used. MSRN performed well on compound sentences of POST type, but not so satisfactorily on MIDDLE type. To analyze the learning of lexical encodings, a merge clustering algorithm and Kohonen's feature map methods were used. The results showed the words belonging to the same category tended to have similar representations. In the third experiment, we attempted to scale up the experiment. The lexicon size increases from 33 to 82. 900 training sentences and 54 194 test sentences were used. We observe quite consistent performance in experiment 2 after scaling up.