Human-system turn taking analysis for the let’s go bus information system

Abstract

We examined turn-taking in the Let’s Go Bus spoken dialog data from two aspects: study the consequences of system barge-in when users are not finished speaking (false system barge-in, FSB); determine whether using a partial recognition result other than the final one produces better results. The consequence of FSBs is a less user-friendly system coupled with poor recognition caused by barge-ins, which divide one user turn into several fragments (UFs). We observed that UFs result in longer dialogs because the dialog manager has to recover from misrecognized utterances. Dialogs with UFs have 34 turns on average those without have 27. Poor recognition and long dialogs together cause lower task success rate. Dialogs with UFs have a success rate of 62% versus 84% for dialogs without. Moreover, we annotated the number of correct and incorrect slots for all partial recognitions. For 51% of the utterances, there exists a partial that contains more correct slots than the final recognition result. This will lead us to develop an algorithm to find the best partial. We conclude that systems that avoid FSB will have more efficient dialogs. They will also have better recognition by using the best partial instead of only the final one.