Optimization of classifying accurate and misarticulated speech sounds for use in a gamified real-time ultrasound biofeedback system

Abstract

Integrating ultrasound biofeedback therapy (UBT) into a real-time, gamified interface to provide articulatory feedback for speech remediation promotes an external focus of attention, thereby reducing the complex cognitive demands required for standard UBT. Previous studies have shown that accuracy of American English rhotic /r/ can be predicted by a single parameter, the difference between tongue dorsum and blade displacements measured by ultrasound imaging during speech production. This parameter has classified speech productions of rhotic syllables as correct versus misarticulated with a classification accuracy up to 85%. However, implementation of this classification approach into real-time gamified UBT, including both measurement timing and establishment of difficulty levels for progressive therapy, would benefit from optimization using a larger dataset. 2,450 productions of 10 distinct rhotic syllables (including prevocalic and postvocalic contexts) from 50 children, with and without misarticulations, were analyzed. For each production, ultrasound image sequences were processed by TonguePART software to acquire tongue displacement trajectories, and accuracy was judged by trained listeners using a visual analog scale. Analyses were conducted to optimize selection of the image frame for classification, determine parameter thresholds appropriate for real-time prediction of /r/ production accuracy, and integrate these thresholds into a difficulty level design for gamified UBT.