Using speech recognition for just-in-time language learning

Abstract

While many language learning systems rely on the student to be a dedicated learner, the opportunity to teach language is present outside the classic teaching situation, in all everyday oral and text-related activities. Teaching language in real situations, where the new knowledge is needed and used immediately, has great potential for robust learning: items are retained over a long period of time, make learning other items easier, and are generalized to other situations. Giving immediate and effective error detection and corrective feedback is a technical challenge for language modeling, spoken dialogue structure, and targeted speech synthesis. We will discuss just-in-time language learning and then present one type of just-in-time learning, lexical entrainment, where the automatic system detects an oral error and then respeaks the utterance that contained the error in corrected form, emphasizing the corrected portion. To test the effectiveness of this approach, the Lets Go spoken dialogue system that furnishes bus information to Pittsburghers was modified to predict errors from a variety of non-native speakers of English. A study of the effectiveness of lexical entrainment for a group of non-native users will be described. [This work is sponsored by NSF Grants 0208835 and IIS0096139.]