Modeling cognitive processes with shuffled‐ and fixed‐order multi‐trial wordlist memory tests

Abstract

AbstractBackgroundMany commonly‐used multi‐trial wordlist memory (WLM) tests present list words in a shuffled order across learning trials, while others maintain a fixed order across trials. The ADAS‐Cog employs shuffling across trials in only one sequence for all assessed subjects. Therefore, to examine the impact of presentation order shuffling on the cognitive processes of learning and memory with a hierarchical Bayesian cognitive processing (HBCP) model, we assessed healthy subjects with a computerized version of the ADAS‐Cog WLM test that randomized wordlist presentation order on each of the three learning trials for each subject (randomized group). We then applied the model with resulting modifications to Alzheimer's Disease Neuroimaging Initiative (ADNI) subjects assessed with the ADAS‐Cog (shuffled group). Finally, we compared results to cognitive processes of patients of a cognitive disorders clinic assessed with the fixed‐order MCI Screen (MCIS; fixed group).MethodRandomized group (n = 89), shuffled group (n = 2,235), and fixed group (n = 3,635) subjects completed respective assessments for university course credit, according to ADNI protocol, or as part of clinical assessment, respectively. The HBCP model uses item response data to estimate cognitive processes of encoding, storage, and retrieval for each wordlist item for each subject (Figure 1), through the probabilities of transferring item episodic memory from one storage state to another on a given trial.ResultAnalysis of randomized group data enabled development of a model extension accounting for observed presentation order. Examination of posterior distributions shows high recall and no serial position curve in the healthy randomized group (Figure 2). Applied to the shuffled group, there is greater interpretability and descriptive adequacy (Figure 3) than there is without the model extension (Figure 4), as can be seen in comparison to fixed group posterior distributions (Figure 5).ConclusionThese findings reveal the impact of presentation order shuffling and demonstrate our model's ability to mitigate it. Shuffling results in loss of information fidelity and the inability to estimate cognitive processing parameters in terms of serial position curves. The greater precision and range of measurement for fixed order presentation tests remains an important consideration for clinical trial design.