Acoustic features of voice as a measure of cognitive load during performance of serial subtraction in a remote data collection context

Abstract

AbstractBackgroundCognitive load is the mental demand a task imposes for a specific person. Performance declines when demand exceed capacity; therefore, increase of mental effort may precede measurable cognitive decline. Physiological indices of load (e.g. heart rate, skin conductance etc.) are sensitive to task demand (e.g. subtracting three vs seven), show increased cognitive load with ageing, and in MCI compared to healthy ageing. Voice features have promise as non‐invasive and scalable indicators of mental effort. Here, we aim to classify serial subtraction at high and low cognitive load using voice recordings captured using an automated remote data collection system.MethodParticipants (aged 17‐86) completed serial subtraction via the Neurovocalix web‐app on their own devices. From a pool of 5,742 participants, 100 were randomly selected for manual review. Seven participants were excluded for audio or performance issues. Responses were transcribed and the start and end of each subtraction attempt marked, producing 3,254 attempts for analysis. Low‐level acoustic features were extracted and aggregated over each attempt, then normalized within participant. Random Forest classifiers were trained and evaluated using Leave‐One‐Subject‐Out‐Cross‐Validation (LOSOCV) to predict high vs low load. LOSOCV repeatedly splits the dataset by subject, with one participant at a time used for testing, and the remainder used for training the model. This produces model predictions for each participant and attempt.ResultAverage cross‐validation accuracy was 0.81 (95% CI 0.78 to 0.84), with an average area under the curve (AUC) of 0.87 (95% CI 0.85 to 0.89). We tested predictions for specific numbers which appeared in both subtraction by seven and by three. Accuracy was 0.78, suggesting that predictions were not driven by specific numeric responses. We observed a significant negative correlation between behavioural performance on the task (response rate), and utterance load probability metric for utterances (ρ=‐0.32, p<0.001), suggesting that participants who were more fluent in serial subtraction exhibited lower cognitive load.ConclusionAcoustic features of voice can distinguish between utterances generated under conditions of high and low cognitive load during serial subtraction, adding a novel, independent and sensitive outcome measure to a cognitive task with established utility in the context of neurodegeneration.