Evaluation of Stochastic Gradient Descent and Relevance Vector Classifier for Improvement in Spoken Language Understanding in Terms of Performance and Data Sparsity

Abstract

Spoken Language Understanding (SLU) is a part of spoken dialogue system (SDS) which represents speech data into semantic form required to understand user's intention. The human-computer interaction system is most commonly placed in noisy environment. This introduces error in Automatic Speech Recognizer (ASR). SLU takes into account ASR errors. Thus robustness of SLU plays important role in efficient working of human computer interaction system. The SLU consist of semantic decoder and belief tracker. Recent line of research models semantic decoder as a classification task or sequence to sequence learning. The SLU models incorporate learning models for classification. The learning model requires lot of labelled data for training. The process of obtaining labelled data is expensive and time consuming. Thus it is desirable that SLU must be able to maintain its accuracy even if sparse training data is available. The objective of this research is to evaluate robustness in terms of accuracy and data sparsity of SLU incorporating Relevance Vector Classifier (RVC) and Stochastic Gradient Descent (SGD) classifier. The framework of SLU incorporates semantic tuple classifier (STC). The STC performs dialogue act classification and slot filling by using classifiers. The classifiers that were incorporated in semantic decoder are Support Vector Machine (SVM), SGD and RVC. The dataset used is DSTC II. The SVM classifier showed maximum robustness and better accuracy as compared to other classifiers. For SVM, classifier the Inter Cross Entropy (ICE) and accuracy were 1.0 and 0.91, respectively. For RVC, ICE and accuracy were 4.055 and 0.52.