Disordered voice recognition benchmark

Abstract

Over the past decade, the application of machine learning to voice disorder recognition has shown promising results. However, several areas of the discipline that impact recognition accuracy remain understudied. These areas include the impact of different vocal tasks, patient demographics, and symptom details. Additionally, the hyperparameters associated with voice features cannot always be easily explored in terms of recognition accuracy. Furthermore, preliminary research from an ongoing scoping review reveals differences in diagnostic terms used within the field. To address the issues raised above, we introduce the Disordered Voice Recognition (DiVR) Benchmark that includes a variety of vocal tasks, some patient history data, and a hierarchical organization of diagnostic terms derived from a consensus of multiple clinical experts. The DiVR Benchmark includes detailed spreadsheets outlining the variety of datasets, features, and machine learning algorithms described in the literature. In this work, we present an ensemble of baseline models and compare recognition results with some of the most commonly employed algorithms. We also describe how the classification accuracy varies with data availability, the granularity of the disorder classification label, and the vocal task employed.