An Improved Differential Neural Computer Model Using Multiplicative LSTM

Abstract

Artificial neural networks excel at doing specific tasks like image recognition, sequence learning, machine translation. But, the direction of research has moved towards the creation of more general purpose neural network architectures. Recently, DeepMind introduced Differentiable Neural Computer (DNC), with an external memory system that is capable of working on complex data structures. DNC can infer from graph problem, solve block puzzle using reinforcement learning and so on. DNC uses LSTM as controller network that manipulates the memory matrix. In this paper, we introduce a change to DNC architecture by replacing LSTM network with multiplicative LSTM and measure the performance of the improved model by training it on three different tasks; namely question answering task using bAbI dataset, character level modelling using harry potter text and planning search using air cargo problem. We compared the performance of the previous model to determine its behavior.