Learning Long-Term Dependencies in Segmented Memory Recurrent Neural Networks

Jinmiao Chen,Narendra S Chaudhari

doi:10.1007/978-3-540-28647-9_61

Learning Long-Term Dependencies in Segmented Memory Recurrent Neural Networks

Jinmiao Chen, Narendra S Chaudhari

https://doi.org/10.1007/978-3-540-28647-9_61

Copy DOI

Publication Date: Jan 1, 2004

Citations: 11

Affiliation: Nanyang Technological University

#Real Time Recurrent Learning Algorithm #Algorithms For Recurrent Neural Networks + Show 8 more

Abstract
Full-Text PDF
Similar Papers

Abstract

AbstractGradient descent learning algorithms for recurrent neural networks (RNNs) perform poorly on long-term dependency problems. In this paper, we propose a novel architecture called Segmented-Memory Recurrent Neural Network (SMRNN). The SMRNN is trained using an extended real time recurrent learning algorithm, which is gradient-based. We tested the SMRNN on the standard problem of information latching. Our implementation results indicate that gradient descent learning is more effective in SMRNN than in standard RNNs.

Full Text