Designing a Hungarian Multimodal Database – Speech Recording and Annotation

Abstract

AbstractThe Hungarian spontaneous speech recording and annotation subproject is being carried out by our Computational Linguistics research group and my PhD work at the University of Debrecen and is a part of a comprehensive multimodal human-machine interaction development project and multimodal (audio and video) database collection. The efficiency of speech recognition systems can be increased by proper acoustic preprocessing and by investigation of the suprasegmental characteristics of spontaneous speech. The research aims to contribute to the exact knowledge of prosody through the examination of spontaneous speech, with special regard to syntactic embeddings, insertions, iterations, hesitations and restarts, various kinds of emotions and discourse markers regarding Hungarian, the lack of a prosodically labelled, representative spontaneous speech database makes the development more difficult. The spontaneous multimodal database is being recorded via guided formal and informal conversations. During the conversation, several points are to be discussed in order to provoke longer monologues, including those phenomena of spontaneous speech, which are to be examined within our research. Designing a continuous spontaneous speech recognition system that is speaker-independent and is able to contribute to our theoretical assumptions, requires the construction of a speech database for which we need to take several personnel and technical aspects into account. The visual channel also needs to be annotated, which will enable us to examine and implement multimodal features as well.Keywordsdatabase planningspontaneous speechprosody researchmultimodality