Confidence scoring for ANN-based spoken language understanding

Abstract

We present an efficient way for confidence scoring in a new spoken language understanding (SLU) approach. The SLU system is based on a combination of weighted finite state automata and an artificial neural network (ANN). Given an input sentence, the system extracts a set of semantic frames, called concepts, and a user intention, called a goal. The confidence scoring is applied for detecting the goals misclassified by the neural network. A set of confidence features is derived from the outcomes of the SLU system, and is automatically selected for confidence scoring. Two classifiers, Fisher linear discriminant analysis and support vector machines (SVM), are compared. Experiments show that when we evaluate on a speech-recognized sentence set that contains about 23% word errors, the SVM achieves over 70% correct rejection of misunderstood sentences at 93% correct acceptance rate.