AcListant with Continuous Learning: Speech Recognition in Air Traffic Control

Abstract

AbstractIncreasing air traffic creates many challenges for air traffic management (ATM). A general answer to these challenges is to increase automation. However, communication between air traffic controllers (ATCos) and pilots is still widely analog and far away from digital ATM components. As communication content is important for the ATM system, commands are still entered manually by ATCos to enable the ATM system to take the content of the communication into account. However, the disadvantage of this procedure is significant additional workload for the ATCos. To avoid this additional effort, automatic speech recognition (ASR) can automatically analyze the communication and extract the content of spoken commands. DLR together with Saarland University invented the AcListant® system, the first assistant based speech recognition (ABSR) with both a high command recognition rate and a low command recognition error rate. Beside the high recognition performance, AcListant® project revealed shortcomings with respect to costly adaptations of the speech recognizer to different air traffic control (ATC) environments. Machine learning algorithms for the automatic adaptation of ABSR to different airports were developed to counteract this disadvantage within the MALORCA project, funded by Single European Sky ATM Research Programme 2020 Exploratory Research (SESAR-ER). To support the standardization of speech recognition in ATM, an ontology for ATC command recognition on semantic level was developed to enable the reuse of expensively manually transcribed ATC communication in the SESAR Industrial Research project PJ.16-04. Finally, results and experiences are used in two further SESAR Wave-2 projects. For the first time, this paper presents the evolution from the idea of ABSR born in an academic environment, starting with the project AcListant®, to industrialization ready research prototype of technology reediness level (TRL) 4. In this course, relevant industrial needs such as costs and necessary standardizations supported by tailored European funding scheme are considered. The addressed SESAR projects are MALORCA, PJ.16-04, PJ.10-96 HMI Interaction modes for ATC centre, and PJ.05-97 HMI Interaction modes for Airport Tower.KeywordsAssistant based speech recognitionMachine learningAcListant®MALORCAPJ.16-04Ontology