Computer‐administered neuropsychological assessment batteries: Validity, reliability, and user experience in an Australian sample of community‐living older adults in the CogSCAN Study

Abstract

AbstractBackgroundAccess to a “gold‐standard” in‐person neuropsychological assessment to detect cognitive decline in older adults using paper‐and‐pencil (PnP) tests is limited. Computerized neuropsychological assessments (CNAs) potentially provide lower cost, scalable, accessible alternatives but critical psychometric data on the validity, reliability and user‐experience of CNAs in older adults are lacking.Method263 community‐dwelling individuals aged 60 years and older without dementia were assigned to complete two CNA batteries and one standardised neuropsychological assessment with 14 PnP tests, one week apart (Construct Validity Group) or two CNAs on two occasions, one month apart (Reliability Group). CNAs were self‐administered under supervision. Orders of CNA batteries, and of CNA and PnP sessions were counterbalanced (Figure 1). Participants completed questionnaires evaluating their experience after each CNA battery. Four CNA batteries were investigated: Cambridge Brain Sciences (CBS), CANTAB, Cogstate, and NIH Toolbox, each containing 5—8 tests; key variables recommended by the test developers were examined (Table 1).ResultParticipant characteristics are summarised in Table 2. Concurrent Validity: Modest correlations (r’s of 0.3‐0.6) between CNA and PnP measures were observed for all CNA batteries, generally aligning with corresponding domains, though notably reflecting the multidimensionality of cognitive measures (Table 3); the strongest associations (r ≥0.5) were with attention and processing speed measures across three CNA batteries, also NIH Toolbox vocabulary, memory and executive measures.Reliability: One‐month test‐retest reliabilities of individual measures were variable (Table 4); across three batteries some, typically speed measures, had reliability coefficients ≥ 0.7. User Experience: Completion rates were high (96‐100%). Usability ratings of CNAs were high on average, although variability was large for CBS (Figure 2). Ratings of acceptability were lower than usability, though on average, participants endorsed CNAs positively.ConclusionA selection of computerized measures demonstrated adequate psychometric standards for concurrent validity with ‘gold standard’ in‐person neuropsychological assessments, alongside good usability and acceptability, though test‐retest reliability was low for many measures. CogSCAN’s comprehensive comparative data will guide researchers in selecting the most suitable CNA measures for older adults. Work in progress is investigating CNA validity, reliability and user experience in older adults with cognitive decline and culturally and linguistically diverse older adults.