Mean Opinion Score performance in classifying voice-enabled emergency communication systems

Abstract

Freedom Fone (FF) is an easy to use Interactive Voice Response (IVR) System that integrates with Global System for Mobile (GSM) telecommunications [1]. Sahana is a disaster management expert system [2]. Project intent was to interconnect the FF and Sahana free and open source software systems. The research adopted Emergency Data Exchange Language (EDXL) interoperable content standard [3] for data interchange between the two platforms. It was an initiative to enable Sarvodaya, Sri Lanka's largest humanitarian organization, with voice-enabled services for exchanging disaster information. An early automation challenge was introducing Sinhala and Tamil language Automatic Speech Recognition (ASR) and Text-To-Speech (TTS) software algorithms for interchanging information between the two disparate software systems [4]. Experiments with human substitution for ASR and TTS with decoupled less streamlined systems revealed inefficiencies [5]. Voice quality was a key factor affecting the Mean Time To Completion (MTTC). The research applied the International Telecommunication Union (ITU) recommended R800 Mean Opinion Score (MOS) and Difficulty Score (DS) voice quality evaluation methods [4]. The overall system was classified with a 3.52 MOS and predicted with a 29.44% DS [4]. This paper justifies the MOS classification accuracy in setting a 4.0 MOS threshold for differentiating good emergency communication IVRs from bad ones.