Quality comparison of remote vs. in‐person digital speech assessment for dementia

Abstract

Listen

Translate article

AbstractBackgroundThe COVID‐19 pandemic has brought the need for reliable, remote assessments for clinical trials into sharp focus. Remote, home‐based assessment reduces patient and caregiver burden, enabling more frequent monitoring. There are concerns, however, regarding the quality of in‐clinic vs. remote assessments. In the present study, we compared the quality of verbal responses to a tablet‐based speech assessment in patients with dementia across two settings: in‐person at a clinical research site, and remotely, at home.MethodThe in‐person sample consisted of individuals with Alzheimer’s disease, participating in a clinical trial. The remote sample consisted of individuals with variants of Frontotemporal dementia, participating in a longitudinal observational study. In‐person assessments were conducted at a clinical research site and administered by a trained rater. Remote assessments were conducted in the participant’s home, by a caregiver who had received assessment training and a tablet by mail. 575 in‐person speech samples and 574 remote speech samples were compared. All samples were manually transcribed and tagged for recording anomalies or task compliance issues by trained transcriptionists.ResultOverall incidence of recording anomalies was low and did not differ significantly (p = 0.63) between in‐person (10.1%; 58/574) and remote (11.1%, 64/575) recordings. Less than 1% of samples in either study were marked as “incomplete task”, “low audio quality” or “noisy background”. Incidences of clinician/caregiver interference were more frequent for remote recordings (1.2% in‐person, 4.5% remote, p = 0.001) and incidences of quiet participants were more frequent for in‐person recordings (4.5% in‐person, 1.0% remote, p < 0.001), though both were relatively rare overall. The mean duration of in‐person samples was significantly longer than remote samples, but mean speech rate did not differ.ConclusionThis study suggests that remote speech assessments yield recordings of comparable quality to in‐person assessments. We found higher, though still low, rates of caregiver interference for remote assessment, which should be monitored and mitigated in future remote assessment. Remote assessments yielded shorter recordings, but this may be due to the different dementia diagnoses across groups. Future work should compare the same participants across both assessment settings.