A Deep Learning Based Multi-task Ensemble Model for Intent Detection and Slot Filling in Spoken Language Understanding

Abstract

An important component of every dialog system is understanding the language popularly known as Spoken Language Understanding (SLU). Intent detection (ID) and slot filling (SF) are the two very important and inter-related tasks of SLU. In this paper, we propose a deep learning based multi-task ensemble model that can perform both intent detection and slot filling tasks together. We use a deep bi-directional recurrent neural network (RNN) with long short term memory (LSTM) and gated recurrent unit (GRU) as the base-level classifiers. A multi-layer perceptron (MLP) framework is used to combine the outputs together. A combined word embedding representation is used to train the model obtained from both Glove and word2vec. This is further augmented with the syntactic Part-of-Speech (PoS) information. On the benchmark ATIS dataset, our experiments show that the proposed ensemble multi-task model (MTM) achieves better results than the individual models and the existing state-of-the-art systems. Experiments on the another dataset, TRAINS also proves that the proposed multi-task ensemble model is more effective compared to the individual models.