LARGE-SCALE REMOTE ASSESSMENT OF VERBAL COGNITIVE FUNCTION USING AUTOMATIC SPEECH RECOGNITION

Abstract

Listen

Translate article

Verbal neuropsychological tests are widely used to identify early signs of neurodegeneration in older adults. To date, employing these tasks remotely at scale, for example in large-scale screening and home-based monitoring programmes, has been limited by the dependence on skilled raters. Here, we report data from a large sample of participants tested in their own homes showing that these tasks can be reliably administered and scored via device-agnostic web-based technology (Cambridge Cognition's Neurovocalix platform). Analysis focused on a brief verbal test battery showing: 1) performance across different devices and platforms; 2) performance with respect to participant demographics, particularly age and native language; 3) the acceptability to participants of the automated testing modality. 3,264 participants aged 17–86 years (M=34.5, SD=12.32) completed a battery of three tasks modeled on traditional neuropsychological assessments, adapted for automated administration. These were digit span, serial subtraction and verbal paired associates. We recorded participant demographics, native language and information regarding the operating system, browser and platform on which the tasks were completed. Voice data was recorded and stored for analysis and quality control. Nearly half (47%) of participants completed the testing on a Microsoft Windows platform, followed by mobile phone (iPhone and Android)(36%), then by Apple Mac (12%). There was no significant difference in performance depending on platform, suggesting that testing is feasible across a range of different devices. Most participants did not report using special audio recording hardware. Background noise significantly affected automatic speech recognition performance. Behaviorally, we observed predicted differences in performance depending on task difficulty (e.g. easy vs hard word pairs, digits forward and back), and predicted relationships between demographic variables (e.g. age) and task performance, replicating established observations for in-person versions for these tasks. Qualitatively, participants reported that the automated instructions were clear and easy to understand, and that the tasks were enjoyable. These results demonstrate the feasibility and acceptability of remote automated cognitive testing at scale, conducted on participants’ own devices and in their own homes. Therefore, these automated tasks may be useful in remote screening or monitoring of older adults.