Story Recall Task Research Articles

A comparison of different connected-speech tasks for detecting mild cognitive impairment using multivariate pattern analysis

ABSTRACT Background It is common for the elderly population to have age-associated cognitive decline and/or develop neurodegenerative diseases such as dementia. Several studies have suggested that classification algorithms based on linguistic features may be useful for the early detection of mild cognitive impairment (MCI). Aims The current study aimed to examine connected-speech performance in people with MCI and cognitively healthy controls (HC). It tests whether patterns of lexical-semantic features extracted from these tasks could distinguish participants with MCI from HC, using univariate and multivariate analyses. Methods & procedure We selected 16 English-speaking participants with MCI and 16 matched HC from the Delaware corpus. Four connected-speech tasks (a picture description, a story narrative, a story recall, and a procedural narrative). Eight lexical-semantic features were selected for analyses. Outcomes & results Univariate analyses showed inter-group differences in revision ratio, core lexicon, or open/closed class words ratio, depending on the task. Multivariate pattern analysis (MVPA) results demonstrated that the story recall task is the only task that can discriminate the two groups above chance. Conclusion To conclude, results showed that connected-speech tasks have the potential to detect subtle language changes in people with MCI. In particular, the story recall task had the potential to predict the group of a participant (MCI or HC).

Examining Children’s Memory Performance: The Role of Parents’ and Children’s Metacognitive Talk During Reminiscence and Play

ABSTRACT Recent research has highlighted the critical role in children’s cognitive development of the metacognitive support parents give their children during everyday interactions. Our main goal was to examine whether parents made consistent use of metacognitive talk across different parent – child interaction contexts and to document the effect of this metacognitive talk across contexts on preschoolers’ memory. The relation between children’s and parents’ metacognitive talk was also examined, along with the link between children’s metacognitive talk and their own memory performance. 64 Belgian preschooler – parent dyads (M age = 40.3 months) were invited to play memory games together and to reminisce about a standardized event. In both contexts, parents’ and children’s utterances were analyzed for their metacognitive content. Children’s episodic memory was assessed through a story-recall task. Results revealed that, in terms of frequency of use, parents exhibited a somewhat consistent metacognitive style across contexts. The nature of the metacognitive comments that were produced, however, varied depending on the context. Moreover, parents’ consistent production of higher rates of metacognitive comments in both contexts was found to be associated with better memory performance in children. Our data also revealed that children’s tendency to make metacognitive comments was related to their parents’ metacognitive production and to their own memory performance. By documenting the relations between parents’ and children’s metacognitive talk and the effect on children’s memory, this study provides promising avenues for a better understanding of how metacognitive processes might be involved in early memory development.

Automated lexical analysis of story recall in healthy controls

AbstractBackgroundImpaired episodic memory is one of the earliest and most prominent symptoms of Alzheimer’s disease (AD). Examining how patients produce words and phrases in story recall tasks is useful for tracking and understanding progression of their disease. Traditional approaches rely on manual assessments of recalled stories which can be limited, time‐consuming, and not applicable in large‐scale studies. Here, we implement an automated, computational approach to measure verbal episodic memory in a digitally recorded story recall task performed by young healthy speakers using natural language processing models.MethodWe analyzed digitized speech samples of pre‐recorded immediate and delayed Craft Story recall tasks performed by healthy speakers (n = 67, mean age = 20.31 years, SD = 2.17, 45 (67%) females). All transcripts were transcribed by trained annotators and automatically scored for number of verbatim and paraphrase recall, total recall score ( = verbatim+paraphrase), and semantic distance from the original story (degree of semantic similarity between the original craft story and the recalled story based on the Word2vec module in python). We also calculated total word count and number of unique words, and rated all recalled words for word familiarity, concreteness, and semantic ambiguity based on published norms. Number of pauses and total durations of speech and silent pauses were also measured.ResultWe found larger semantic distance between successive words in the delayed recall task compared with immediate recall within speaker (p<.001). Delayed recall also elicited lower total recall scores than immediate recall (p = .03). Comparing the effect sizes of semantic distance vs. total recall scores, our model comparison showed a smaller prediction error for semantic distance (AIC = 95.45) than total recall score (AIC = 736.18). In addition, speakers in the delayed recall task performed more paraphrase recall, had higher total word count, number of unique words, more ambiguous words, and produced more speech (all p‐values<.001).ConclusionOur findings suggest that automated speech analysis can provide informative and cost‐effective measures from a recorded story recall task. The study provides an automated and quantitative way to better measure memory abilities by including semantic distance. Future work will test procedures in patients with AD or other neurodegenerative diseases.

Optimising speech‐testing to predict prodromal Alzheimer’s disease: head‐to‐head comparison study of tasks and analysis methods

AbstractBackgroundChanges in speech occur in early‐stage Alzheimer’s disease. A range of approaches have been used for eliciting and automatically analysing speech, but limited research has directly compared these methods.MethodParticipants from the AMYPRED‐UK (NCT04828122) and AMYPRED‐US (NCT04928976) studies completed four speech‐elicitation tasks: the Automatic Story Recall Task (ASRT), the Logical Memory Test (LMT), Semantic (Animals, Vegetables, Fruit) and Phonemic Verbal Fluency (F, A, S) Tasks. Responses were recorded and automatically transcribed. Analyses were completed with Novoic’s speech analysis software, using four key approaches: (1) feature extraction, (2) representations from large language models (LLM), (3) text‐similarity evaluation, and (4) autoscoring. Together, these evaluated audio, linguistic, and temporal speech domains. Outputs were entered into logistic regression models predicting Mild Cognitive Impairment (MCI)/mild AD and Amyloid positivity in MCI/mild AD. Area under the ROC curves (AUC) were evaluated via 5‐fold cross‐validation.Result165 older adults (including N = 74 MCI and 9 mild AD; of which N = 38 MCI/mild AD and amyloid beta positive) provided data for all tasks. Strength of model predictions varied by task, analytic approach and domain (Fig 1). AUCs predicting MCI/mild AD were generally higher for ASRTs compared to other tasks (Fig 1A). Text‐similarity approaches (G‐match: similarity of word embeddings between the source text and retelling, and V‐match: string‐based matching), produced the highest AUC for ASRTs (G‐match AUC = 0.87‐0.88), and the LMT (V‐match AUC = 0.83‐85). Autoscoring tasks produced the strongest predictor for the semantic fluency (AUC = 0.85), and also one of the strongest for phonemic fluency (AUC = 0.72). Changes in linguistic and temporal characteristics were more sensitive to MCI/mild AD, compared to audio‐only metrics. Amyloid in MCI/mild AD predictions were overall less strong (Fig 1B), with the most consistent pattern being better performance of text‐similarity approaches on story recall tasks (up to AUC = 0.77 for ASRTs, 0.73 for LMT).ConclusionSelection of task and analytic approach is important for developing more sensitive speech‐based testing. The best performing tasks (ASRT, Semantic Fluency) and metrics (text‐similarity metrics and autoscoring) have been incorporated in Novoic’s Storyteller automated remote speech‐based test.

Optimising speech‐testing to predict prodromal Alzheimer’s disease: head‐to‐head comparison study of tasks and analysis methods

AbstractBackgroundChanges in speech occur in early‐stage Alzheimer’s disease. A range of approaches have been used for eliciting and automatically analysing speech, but limited research has directly compared these methods.MethodParticipants from the AMYPRED‐UK (NCT04828122) and AMYPRED‐US (NCT04928976) studies completed four speech‐elicitation tasks: the Automatic Story Recall Task (ASRT), the Logical Memory Test (LMT), Semantic (Animals, Vegetables, Fruit) and Phonemic Verbal Fluency (F, A, S) Tasks. Responses were recorded and automatically transcribed. Analyses were completed with Novoic’s speech analysis software, using four key approaches: (1) feature extraction, (2) representations from large language models (LLM), (3) text‐similarity evaluation, and (4) autoscoring. Together, these evaluated audio, linguistic, and temporal speech domains. Outputs were entered into logistic regression models predicting Mild Cognitive Impairment (MCI)/mild AD and Amyloid positivity in MCI/mild AD. Area under the ROC curves (AUC) were evaluated via 5‐fold cross‐validation.Result165 older adults (including N = 74 MCI and 9 mild AD; of which N = 38 MCI/mild AD and amyloid beta positive) provided data for all tasks. Strength of model predictions varied by task, analytic approach and domain (Fig 1). AUCs predicting MCI/mild AD were generally higher for ASRTs compared to other tasks (Fig 1A). Text‐similarity approaches (G‐match: similarity of word embeddings between the source text and retelling, and V‐match: string‐based matching), produced the highest AUC for ASRTs (G‐match AUC = 0.87‐0.88), and the LMT (V‐match AUC = 0.83‐85). Autoscoring tasks produced the strongest predictor for the semantic fluency (AUC = 0.85), and also one of the strongest for phonemic fluency (AUC = 0.72). Changes in linguistic and temporal characteristics were more sensitive to MCI/mild AD, compared to audio‐only metrics. Amyloid in MCI/mild AD predictions were overall less strong (Fig 1B), with the most consistent pattern being better performance of text‐similarity approaches on story recall tasks (up to AUC = 0.77 for ASRTs, 0.73 for LMT).ConclusionSelection of task and analytic approach is important for developing more sensitive speech‐based testing. The best performing tasks (ASRT, Semantic Fluency) and metrics (text‐similarity metrics and autoscoring) have been incorporated in Novoic’s Storyteller automated remote speech‐based test.

The impact of subjective cognitive decline on aging effects in the brain as measured by diffusion tensor imaging

AbstractBackgroundDiffusion tensor imaging (DTI) can separate subjective cognitive decline (SCD) participants who progress to mild cognitive impairment and those who do not 1. We explored the effect age and memory performance has on DTI measures in SCD and controls in the hippocampal cingulum bundle, which is typically affected early in the course of Alzheimer’s disease2.Method325 participants Cam‐CAN 3 (127 SCD) older than 55 were included in the analysis. Diffusion imaging data was collected on a 3T Siemens TIM Trio scanner with a 32‐channel head coil for 0, 1000 and 2000 s/mm2 b‐values and 30 gradient directions. Preprocessing and calculation of the DTI metrics fractional anisotropy (FA, mean diffusivity (MD), radial diffusivity (RD) and axial diffusivity (AxD) were completed using MRTrix3, FSL and MATLAB. The hippocampal cingulum bundle regions of interest (ROIs) were extracted from the ICBM‐DTI‐81 atlas. Memory was assessed using the delayed story recall (DSR) task from WMS‐III. Linear regression models were completed in R. Gender was included as a nuisance variable.ResultThere were no differences in gender or age distribution between cohorts, however participants with SCD had a 14% lower DSR score (Table 1). There was a significant interaction effect between age and cohort in MD, AxD, and RD (Figure 1), as well as between memory and age in AxD (Figure 2). In post‐hoc analyses, compared to controls, the SCD group showed a stronger correlation with age in MD, AxD, and RD (Figure 1) while participants with worse memory showed a stronger correlation with age in MD (Figure 2), indicating an accelerated aging effect.ConclusionParticipants with SCD and worse memory present with an accelerated aging effect, showing steeper variations in diffusion microstructure with age. Since SCD is likely to be a preclinical stage of Alzheimer’s disease, future work will analyze this effect with more advanced diffusion microstructure models to further elucidate the underlying pathophysiology. • Selnes P, Aarsland D, Bjornerud A, et al. J Alzheimers Dis. 2013;33(3):723‐36. • Jack Jr CR, Knopman DS, Jagust WJ, et al. The Lancet Neurology. 2013;12(2):207‐16. • Shafto MA, Tyler LK, Dixon M, et al. BMC neurology. 2014;14(1):1‐25.

The impact of subjective cognitive decline on aging effects in the brain as measured by diffusion tensor imaging

AbstractBackgroundDiffusion tensor imaging (DTI) can separate subjective cognitive decline (SCD) participants who progress to mild cognitive impairment and those who do not 1. We explored the effect age and memory performance has on DTI measures in SCD and controls in the hippocampal cingulum bundle, which is typically affected early in the course of Alzheimer’s disease2.Method325 participants Cam‐CAN 3 (127 SCD) older than 55 were included in the analysis. Diffusion imaging data was collected on a 3T Siemens TIM Trio scanner with a 32‐channel head coil for 0, 1000 and 2000 s/mm2 b‐values and 30 gradient directions. Preprocessing and calculation of the DTI metrics fractional anisotropy (FA, mean diffusivity (MD), radial diffusivity (RD) and axial diffusivity (AxD) were completed using MRTrix3, FSL and MATLAB. The hippocampal cingulum bundle regions of interest (ROIs) were extracted from the ICBM‐DTI‐81 atlas. Memory was assessed using the delayed story recall (DSR) task from WMS‐III. Linear regression models were completed in R. Gender was included as a nuisance variable.ResultThere were no differences in gender or age distribution between cohorts, however participants with SCD had a 14% lower DSR score (Table 1). There was a significant interaction effect between age and cohort in MD, AxD, and RD (Figure 1), as well as between memory and age in AxD (Figure 2). In post‐hoc analyses, compared to controls, the SCD group showed a stronger correlation with age in MD, AxD, and RD (Figure 1) while participants with worse memory showed a stronger correlation with age in MD (Figure 2), indicating an accelerated aging effect.ConclusionParticipants with SCD and worse memory present with an accelerated aging effect, showing steeper variations in diffusion microstructure with age. Since SCD is likely to be a preclinical stage of Alzheimer’s disease, future work will analyze this effect with more advanced diffusion microstructure models to further elucidate the underlying pathophysiology.1. Selnes P, Aarsland D, Bjornerud A, et al. J Alzheimers Dis. 2013;33(3):723‐36.2. Jack Jr CR, Knopman DS, Jagust WJ, et al. The Lancet Neurology. 2013;12(2):207‐16.3. Shafto MA, Tyler LK, Dixon M, et al. BMC neurology. 2014;14(1):1‐25.

A - 129 An Examination of the Relationship between Posttraumatic Stress Disorder Symptom Severity and Cognitive Test Performance in a Veteran Sample.

This study examined the relationship between Posttraumatic Stress Disorder (PTSD) symptom severity and cognitive test performance in veterans, hypothesizing that PTSD Checklist for DSM-5 (PCL-5) scores would be correlated with test performance on commonly used neuropsychological measures. Clinically-referred veterans without a neurocognitive disorder who passed the Test of Memory Malingering (TOMM), completed the PCL-5, and completed Weschler Adult Intelligence Scale-Fourth Edition (WAIS-IV) Digit Span or Coding, Weschler Memory Scale-Fourth Edition (WMS-IV) Logical Memory (LMI/LMII), letter fluency (FAS), or Trail Making Test (Parts A and B) were included. The sample included 38 veterans (81.6% male); mean age = 58.79 (SD = 14.71; range 22-80) and mean education level = 13.97years (SD = 2.42, range 6-20). Primary diagnoses included PTSD (47.4%), depression (21.1%), and no diagnosis (23.7%). Pearson, Spearman, and partial correlations were calculated. Moderate negative correlations were found between PCL-5 and FAS raw scores (r = -0.41, p = 0.014; 95%CI [-0.654,-0.089]) and LMI raw scores (r = -0.453, p = 0.02; 95%CI [-0.715,-0.080]). After controlling for age and education, moderate negative correlations remained for FAS (r = -0.46, p = 0.004) and LMI (r = -0.43, p = 0.018) and a weak, non-significant correlation with LMII (r = -0.297, p = 0.079). In veterans with valid cognitive test performance and no neurocognitive disorder, PTSD symptom severity negatively correlated with FAS and WMS-IV LM performance. This suggests PTSD symptoms may influence performance on phonemic fluency and story recall task. Assessing and addressing PTSD symptoms may result in more accurate diagnoses and improved cognitive functioning. Understanding the PTSD-cognition relationship can inform interventions and support strategies to enhance veterans' quality of life.

Rage at Strangers: Anger Elicitation and Regulation as a Function of Relationship Type

Abstract Anger can be the most socially debilitating of the basic emotions, and effective and simple techniques for managing anger are much needed. Autobiographical recall is a powerful method for emotion elicitation, with obvious clinical utility. However, the content of the material elicited, most notably the variable of relationship type, has not been systematically studied. The present study investigated the effectiveness of the Affective Story Recall task for anger elicitation, using 5 relationship categories (family, partner, friend, stranger, abstract) and the effectiveness of 2 emotion regulation (ER) techniques: reappraisal and distraction. Fifty-two neurologically healthy adults completed a series of anger elicitation and regulation tasks, several measures of cognitive ability, and questionnaires on attachment styles and the use of ER techniques. Most relationship categories selectively elicited anger, with some categories (e.g., stranger) eliciting especially high levels. Both reappraisal and distraction reduced anger intensity across all relationship categories. However, distraction was particularly effective for strangers. Distraction effectiveness was predicted by the regular use of this ER technique, attachment style, and age, but cognitive factors played no significant role. These findings have implications for rehabilitation, allowing clinicians to tailor interventions to patients’ strengths and weaknesses.

Semantic Speech Networks Linked to Formal Thought Disorder in Early Psychosis

Mapping a patient's speech as a network has proved to be a useful way of understanding formal thought disorder in psychosis. However, to date, graph theory tools have not explicitly modelled the semantic content of speech, which is altered in psychosis. We developed an algorithm, "netts," to map the semantic content of speech as a network, then applied netts to construct semantic speech networks for a general population sample (N = 436), and a clinical sample comprising patients with first episode psychosis (FEP), people at clinical high risk of psychosis (CHR-P), and healthy controls (total N = 53). Semantic speech networks from the general population were more connected than size-matched randomized networks, with fewer and larger connected components, reflecting the nonrandom nature of speech. Networks from FEP patients were smaller than from healthy participants, for a picture description task but not a story recall task. For the former task, FEP networks were also more fragmented than those from controls; showing more connected components, which tended to include fewer nodes on average. CHR-P networks showed fragmentation values in-between FEP patients and controls. A clustering analysis suggested that semantic speech networks captured novel signals not already described by existing NLP measures. Network features were also related to negative symptom scores and scores on the Thought and Language Index, although these relationships did not survive correcting for multiple comparisons. Overall, these data suggest that semantic networks can enable deeper phenotyping of formal thought disorder in psychosis. Whilst here we focus on network fragmentation, the semantic speech networks created by Netts also contain other, rich information which could be extracted to shed further light on formal thought disorder. We are releasing Netts as an open Python package alongside this manuscript.

Preschoolers’ memory recall of new information: relation to parental reminiscing

ABSTRACT The influence of parental reminiscing style (the way parents discuss past events with their children) on the development of children’s autobiographical memory has been well documented. The specific mechanisms involved in this effect, however, remain unknown. We explored the association between specific components of parental reminiscing and preschoolers’ episodic memory. Fifty-three parent–child dyads (M Child Age = 53.13 months, 29 females) from Belgium were tested twice within a delay of about nine months. At the first time point, parental reminiscing style was assessed via a parent–child discussion of a prior standardised event (a museum visit) focusing on both the structure of parental interactions and the addressed content. At each time point, children were administered with story–recall tasks in the form of true–false recognition about previously heard stories. Generalised linear mixed-effect models were conducted on an item-by-item basis. Results indicated that parental reminiscing is associated with preschoolers’ recognition memory performance at both time points, but not all reminiscing components equally influence children’s performance. Specifically, parents’ concretisations and metamemory talk were found to impact children’s memory. The identification of these components provided insights for exploring the processes underlying the reminiscing–memory influence (in-depth encoding of information, binding processes, and metacognition).

The recency ratio assessed by story recall is associated with cerebrospinal fluid levels of neurodegeneration biomarkers

Recency refers to the information learned at the end of a study list or task. Recency forgetting, as tracked by the ratio between recency recall in immediate and delayed conditions, i.e., the recency ratio (Rr), has been applied to list-learning tasks, demonstrating its efficacy in predicting cognitive decline, conversion to mild cognitive impairment (MCI), and cerebrospinal fluid (CSF) biomarkers of neurodegeneration. However, little is known as to whether Rr can be effectively applied to story recall tasks. To address this question, data were extracted from the database of the Alzheimer's Disease Research Center at the University of Wisconsin – Madison. A total of 212 participants were included in the study. CSF biomarkers were amyloid-beta (Aβ) 40 and 42, phosphorylated (p) and total (t) tau, neurofilament light (NFL), neurogranin (Ng), and α-synuclein (a-syn). Story Recall was measured with the Logical Memory Test (LMT). We carried out Bayesian regression analyses with Rr, and other LMT scores as predictors; and CSF biomarkers (including the Aβ42/40 and p-tau/Aβ42 ratios) as outcomes. Results showed that models including Rr consistently provided best fits with the data, with few exceptions. These findings demonstrate the applicability of Rr to story recall and its sensitivity to CSF biomarkers of neurodegeneration, and encourage its inclusion when evaluating risk of neurodegeneration with story recall.

EXPLORING LINGUISTIC PATTERNS ON A STORY RECALL TASK IN PEOPLE WITH MILD COGNITIVE IMPAIRMENT

Abstract Mild cognitive impairment (MCI) involves declines in language and episodic memory. Episodic memory is often assessed using language tasks. To prevent linguistic factors from confounding recall scores, memory and language should be jointly examined. We explored linguistic patterns on a story recall task among cognitively healthy adults aged 65+ (n=18) and people with amnestic MCI (n=18). Participants completed immediate and delayed (20-30min) recall on a set of novel story recall materials (6 pairs, i.e., 12 stories total). Stories were coded using a propositional coding scheme (where a proposition refers to the smallest unit of meaning), as well as a unit scoring scheme (i.e., individual words). Responses were coded as veridical (word-for-word), gist (general idea), and distortion (error). Linguistic features of the output were coded using the Linguistic Inquiry and Word Count (LIWC) program. Overall, people with MCI produced more verbs, fewer time-related words, and fewer total words than control participants. In the MCI group, delayed unit- and proposition-based veridical and gist recall scores were positively correlated with certainty and causation words, indicating that higher certainty about events and their causal links is associated with better memory. Total words were positively correlated with all immediate and delayed recall scores, indicating that amount of linguistic output is strongly linked to memory in MCI. Time-related words were positively correlated with immediate unit-based veridical recall, suggesting that, in MCI, more words denoting time signal better immediate recall of story details. Examining linguistic features of verbal output in memory tasks could improve detection of MCI.

A novel dual-task paradigm with story recall shows significant differences in the gait kinematics in older adults with cognitive impairment: A cross-sectional study.

Cognitive and motor dysfunctions in older people become more evident while dual-tasking. Several dual-task paradigms have been used to identify older individuals at the risk of developing Alzheimer's disease and dementia. This study evaluated gait kinematic parameters for dual-task (DT) conditions in older adults with mild cognitive impairment (MCI), subjective cognitive decline (SCD), and normal cognition (NC). This is a cross-sectional, clinical-based study carried out at the Zhongshan Rehabilitation Branch of First Affiliated Hospital of Nanjing Medical University, China. We recruited 83 community-dwelling participants and sorted them into MCI (n = 24), SCD (n = 33), and NC (n = 26) groups based on neuropsychological tests. Their mean age was 72.0 (5.55) years, and male-female ratio was 42/41 (p = 0.112). Each participant performed one single-task walk and four DT walks: DT calculation with subtracting serial sevens; DT naming animals; DT story recall; and DT words recall. Kinematic gait parameters of speed, knee peak extension angle, and dual-task cost (DTC) were obtained using the Vicon Nexus motion capture system and calculated by Visual 3D software. A mixed-effect linear regression model was used to analyze the data. The difference in gait speed under DT story recall and DT calculation was -0.099 m/s and - 0.119 m/s (p = 0.04, p = 0.013) between MCI and SCD, respectively. Knee peak extension angle under DT story recall, words recall, and single task was bigger in the MCI group compared to the NC group, respectively (p = 0.001, p = 0.001, p = 0.004). DTC was higher in the DT story recall test than all other DT conditions (p < 0.001). Kinematic gait parameters of knee peak extension angle for the DT story recall were found to be sensitive enough to discriminate MCI individuals from NC group. DTC under DT story recall was higher than the other DT conditions.

Associations between semantic memory for proper names in story recall and CSF amyloid and tau in a cognitively unimpaired sample

AbstractBackgroundDifficulty recalling proper names (PNs; e.g., names of people/places) is a common complaint of individuals at risk for Alzheimer’s disease (AD). We recently reported that cognitively unimpaired (CU), positron emission tomography amyloid‐beta (Aβ) positive (A+) individuals recalled fewer PNs than Aβ negative individuals on the Logical Memory (LM) delayed story recall test. Declines in delayed LM total scores (LMtot) have been associated with cerebrospinal fluid (CSF) A+ and tau (T); the relationship between PNs and AD‐CSF biomarkers is unknown. We investigated whether delayed recall of PNs at baseline, or most recent visit, were associated with biomarker positivity for two AD‐CSF biomarkers, and whether PNs were more strongly associated than traditionally used LMtot scores.MethodCU participants with no other neurological diagnoses and CSF data available were included from the Wisconsin Registry for Alzheimer’s Prevention (N = 219; mean baseline age [standard deviation] = 58.12 [6.31]). Item‐level LM data were used to create PNs (0–9) and LMtot (0–50) scores. A and T statuses (+/‐) were determined using published cutoffs for CSF Aβ42/40 and phosphorylated tau 181 (pTau181). Logistic regression models investigated whether LMtot or PNs were associated with most recent biomarker status, covarying CSF age, sex, Wide Range Achievement Test‐3 score, and apolipoprotein E4 status. Model fits were compared using Akaike information criterion (AIC). For descriptive purposes, effect sizes between adjusted means of PNs and LMtot were calculated for biomarker‐positive vs. negative groups.ResultOverall, 31 individuals (14%) were Aβ42/40+ and 29 (13.0%) were pTau181+ (Table 1). Baseline PNs nor LMtot were not significantly associated with AD‐CSF biomarkers (Table 2). Most recent PNs and LMtot were associated with AD‐CSF (Table 3). AICs indicated similar fits between PNs and LMtot at each timepoint, and better fits with most recent vs. baseline LM scores. Effect sizes were higher for most recent LM metrics compared to baseline and were slightly higher for PNs than LMTot for Aβ (Figure 1).ConclusionMetrics from story recall tasks are associated with increased risk of CSF‐AD biomarker positivity and suggest that PNs from story recall may be sensitive to early AD biomarker changes. Validation by replication in additional cohorts is required.

A simple, automated, speech‐based and AI‐enhanced screener for mild cognitive impairment and amyloid beta positivity

AbstractBackgroundVocal and linguistic changes in Alzheimer’s dementia have been documented. The current study assesses whether a fully automated speech‐based artificial intelligence (AI) system can detect early clinical impairment and amyloid positivity, which characterise the earliest stages of Alzheimer’s disease (AD).MethodTwo studies were completed in the UK and USA: AMYPRED‐UK (NCT04828122) and AMYPRED‐US (NCT04928976). 200 participants with established amyloid beta (Aβ) and clinical diagnostic status (97 Aβ+, 103 Aβ‐ from prior PET scan or CSF tests; 94 cognitively unimpaired (CU), 106 with mild cognitive impairment (MCI) or mild AD) were recruited and completed automated assessments with the Automatic Story Recall Task (ASRT) either in‐clinic or via telemedicine appointment. The AI text‐pair evaluation model ParaBLEU produced vector‐based representations of the abstract, generalized patterns that differed between the original story text and transcribed retellings. These were fed into logistic regression models trained with tournament leave‐pair‐out cross‐validation analysis to predict Aβ status and MCI/mild AD. Potential benefits of screening using the ASRT system were examined via simulation, including: (1) identifying MCI in primary care, and (2) reducing the number of PET scans required in clinical research studies and trials.ResultUsing an average of only 6.56 minutes of speech per participant, the ASRT system predicted Aβ positivity in the full sample (area under the receiver operating characteristic curve (AUC) = 0.77) and in diagnostic subsamples (MCI/mild AD: AUC = 0.82; CU: AUC = 0.71), as well as MCI/mild AD (AUC = 0.83) in the full sample. Simulation indicated that screening with the ASRT system could: (1) increase correct (+8.5%) and reduce incorrect referrals (‐59.1%) compared with the Mini‐Mental State Exam; and (2) enrich samples for Aβ positivity prior to PET scan (‐35.3% and ‐35.5% fewer scans required in MCI and CU individuals, respectively).ConclusionWith the first disease‐modifying treatment for AD now available, there is an urgent need for improved screening to identify individuals at risk for AD dementia. The ASRT system is a brief, effective, automated, speech‐based cognitive assessment offering scalable screening for MCI/mild AD and amyloid beta biomarker positivity.

Validation of a Remote and Fully Automated Story Recall Task to Assess for Early Cognitive Impairment in Older Adults: Longitudinal Case-Control Observational Study.

BackgroundStory recall is a simple and sensitive cognitive test that is commonly used to measure changes in episodic memory function in early Alzheimer disease (AD). Recent advances in digital technology and natural language processing methods make this test a candidate for automated administration and scoring. Multiple parallel test stimuli are required for higher-frequency disease monitoring.ObjectiveThis study aims to develop and validate a remote and fully automated story recall task, suitable for longitudinal assessment, in a population of older adults with and without mild cognitive impairment (MCI) or mild AD.MethodsThe “Amyloid Prediction in Early Stage Alzheimer’s disease” (AMYPRED) studies recruited participants in the United Kingdom (AMYPRED-UK: NCT04828122) and the United States (AMYPRED-US: NCT04928976). Participants were asked to complete optional daily self-administered assessments remotely on their smart devices over 7 to 8 days. Assessments included immediate and delayed recall of 3 stories from the Automatic Story Recall Task (ASRT), a test with multiple parallel stimuli (18 short stories and 18 long stories) balanced for key linguistic and discourse metrics. Verbal responses were recorded and securely transferred from participants’ personal devices and automatically transcribed and scored using text similarity metrics between the source text and retelling to derive a generalized match score. Group differences in adherence and task performance were examined using logistic and linear mixed models, respectively. Correlational analysis examined parallel-forms reliability of ASRTs and convergent validity with cognitive tests (Logical Memory Test and Preclinical Alzheimer’s Cognitive Composite with semantic processing). Acceptability and usability data were obtained using a remotely administered questionnaire.ResultsOf the 200 participants recruited in the AMYPRED studies, 151 (75.5%)—78 cognitively unimpaired (CU) and 73 MCI or mild AD—engaged in optional remote assessments. Adherence to daily assessment was moderate and did not decline over time but was higher in CU participants (ASRTs were completed each day by 73/106, 68.9% participants with MCI or mild AD and 78/94, 83% CU participants). Participants reported favorable task usability: infrequent technical problems, easy use of the app, and a broad interest in the tasks. Task performance improved modestly across the week and was better for immediate recall. The generalized match scores were lower in participants with MCI or mild AD (Cohen d=1.54). Parallel-forms reliability of ASRT stories was moderate to strong for immediate recall (mean rho 0.73, range 0.56-0.88) and delayed recall (mean rho=0.73, range=0.54-0.86). The ASRTs showed moderate convergent validity with established cognitive tests.ConclusionsThe unsupervised, self-administered ASRT task is sensitive to cognitive impairments in MCI and mild AD. The task showed good usability, high parallel-forms reliability, and high convergent validity with established cognitive tests. Remote, low-cost, low-burden, and automatically scored speech assessments could support diagnostic screening, health care, and treatment monitoring.

Leveraging speech and artificial intelligence to screen for early Alzheimer's disease and amyloid beta positivity.

Early detection of Alzheimer's disease is required to identify patients suitable for disease-modifying medications and to improve access to non-pharmacological preventative interventions. Prior research shows detectable changes in speech in Alzheimer's dementia and its clinical precursors. The current study assesses whether a fully automated speech-based artificial intelligence system can detect cognitive impairment and amyloid beta positivity, which characterize early stages of Alzheimer's disease. Two hundred participants (age 54-85, mean 70.6; 114 female, 86 male) from sister studies in the UK (NCT04828122) and the USA (NCT04928976), completed the same assessments and were combined in the current analyses. Participants were recruited from prior clinical trials where amyloid beta status (97 amyloid positive, 103 amyloid negative, as established via PET or CSF test) and clinical diagnostic status was known (94 cognitively unimpaired, 106 with mild cognitive impairment or mild Alzheimer's disease). The automatic story recall task was administered during supervised in-person or telemedicine assessments, where participants were asked to recall stories immediately and after a brief delay. An artificial intelligence text-pair evaluation model produced vector-based outputs from the original story text and recorded and transcribed participant recalls, quantifying differences between them. Vector-based representations were fed into logistic regression models, trained with tournament leave-pair-out cross-validation analysis to predict amyloid beta status (primary endpoint), mild cognitive impairment and amyloid beta status in diagnostic subgroups (secondary endpoints). Predictions were assessed by the area under the receiver operating characteristic curve for the test result in comparison with reference standards (diagnostic and amyloid status). Simulation analysis evaluated two potential benefits of speech-based screening: (i) mild cognitive impairment screening in primary care compared with the Mini-Mental State Exam, and (ii) pre-screening prior to PET scanning when identifying an amyloid positive sample. Speech-based screening predicted amyloid beta positivity (area under the curve = 0.77) and mild cognitive impairment or mild Alzheimer's disease (area under the curve = 0.83) in the full sample, and predicted amyloid beta in subsamples (mild cognitive impairment or mild Alzheimer's disease: area under the curve = 0.82; cognitively unimpaired: area under the curve = 0.71). Simulation analyses indicated that in primary care, speech-based screening could modestly improve detection of mild cognitive impairment (+8.5%), while reducing false positives (-59.1%). Furthermore, speech-based amyloid pre-screening was estimated to reduce the number of PET scans required by 35.3% and 35.5% in individuals with mild cognitive impairment and cognitively unimpaired individuals, respectively. Speech-based assessment offers accessible and scalable screening for mild cognitive impairment and amyloid beta positivity.

Do preschool children have modality specific recall abilities? - A cross-sectional pilot study

Little is explored regarding the modality-specific differences in recall abilities of preschool children. Understanding modality-specific differences in the recall at an early age might give an insight into age-linked trends, which can lay a foundation for later development. The current study used a cross-sectional design to investigate the modality-specific differences in recall abilities of preschool children using a story and story stimulus. A total of 20 typically developing children between the ages of 3.6 to 5.5 years were presented with a story followed by a story recall task across auditory, visual, and auditory-visual modalities to determine modality-specific recall abilities. The results were analyzed using mixed ANOVA, which revealed higher recall ability in the auditory-visual modality than in other modalities. The findings are explained based on Dual coding theory, Multimedia effect, Mental imagery skills, Visual superiority effect, Attentional behavior, and Comprehension skill. The study implies the importance of Auditory-Visual modality in facilitating better recall in story-based tasks among preschool children.

Improving the Applicability of AI for Psychiatric Applications through Human-in-the-loop Methodologies.

ObjectivesMachine learning (ML) and natural language processing have great potential to improve efficiency and accuracy in diagnosis, treatment recommendations, predictive interventions, and scarce resource allocation within psychiatry. Researchers often conceptualize such an approach as operating in isolation without much need for human involvement, yet it remains crucial to harness human-in-the-loop practices when developing and implementing such techniques as their absence may be catastrophic. We advocate for building ML-based technologies that collaborate with experts within psychiatry in all stages of implementation and use to increase model performance while simultaneously increasing the practicality, robustness, and reliability of the process.MethodsWe showcase pitfalls of the traditional ML framework and explain how it can be improved with human-in-the-loop techniques. Specifically, we applied active learning strategies to the automatic scoring of a story recall task and compared the results to a traditional approach.ResultsHuman-in-the-loop methodologies supplied a greater understanding of where the model was least confident or had knowledge gaps during training. As compared to the traditional framework, less than half of the training data were needed to reach a given accuracy.ConclusionsHuman-in-the-loop ML is an approach to data collection and model creation that harnesses active learning to select the most critical data needed to increase a model’s accuracy and generalizability more efficiently than classic random sampling would otherwise allow. Such techniques may additionally operate as safeguards from spurious predictions and can aid in decreasing disparities that artificial intelligence systems otherwise propagate.