Customization of an example-based dialog system with user data and distributed word representations

Abstract

There is a need to develop spoken dialog systems which are capable of engaging in natural conversations with people, for example, the socially-isolated elderly. We propose an example-based dialog system featuring an adaptation method which customizes the dialog for each user. After retrieving user profile-related information from the Web, named entity extraction is applied to the retrieval results. Words with a high term frequency-inverse document frequency (TF-IDF) score are adopted as user related words. We then calculate the similarity between the selected user-related words and words in the existing example phrases of the dialog system. Cosine similarity between the distributed representations of the nouns is calculated using word2vec embedding. We then generate phrases adapted to the user by substituting user-related words for highly similar words in the original example phrases. Word2vec also has a special property which allows arithmetic operations such as plus and minus to be applied to distributed word representations. By applying operations to the words used in the original phrases, we were able to derive replacement words related to the user and insert them in the example phrases. We evaluated the naturalness of the generated phrases and found that the system could generate natural phrases.