Effectiveness of immediate vs. delayed recall in detecting invalid performance in coached and uncoached simulators: Results of two experimental studies

Florin Alin Sava,Laurențiu Paul Maricuțoiu,Iulia Crișan

doi:10.24913/rjap.23.2.02

Abstract

Objective: Two experimental studies were conducted to compare the ability of immediate and delayed recall indicators to discriminate between performances of simulators and full-effort clinical and nonclinical participants. Methods: Three groups of simulators (uncoached, symptom-coached, and testcoached), one group of community controls, and one group of cognitively impaired patients were assessed with four experimental memory tests, in which the immediate and delayed recall tasks were separated by three other tasks. Results: Across both studies, delayed recall demonstrated higher accuracy than immediate recall in classifying simulated performances as invalid, as compared to performances of bona fide clinical participants. ROC curve results showed sensitivities below 50% for both indicators at specificities of ≥ 90%. Computing performance curves across recall trials revealed descending trends for all three simulator groups indicating a suppressed learning effect as a marker of noncredible performances. Among types of coaching, test-coaching proved to decrease differences between simulators and patients. Discussion: The effectiveness of such indicators in clinical evaluations and their vulnerability to information about test-taking strategies are discussed.

Highlights

Types of recall: Immediate vs. delayed recallDecades of research show that standard performance validity tests (PVTs) assessing short-time memory are especially effective in detecting invalid performances (Bigler, 2014; Larrabee, 2003)
Manipulation checks – differences between groups As our participants came from three different populations, one-way ANCOVAs were conducted between scores of the three aggregated groups on the immediate and delayed recall indicators whilst adjusting for age, gender, and education, at a
We found significant differences between the three groups on the delayed recall indicator [F (2, 181) = 53.448, p = .001, Eta2 = .371], and on the immediate recall indicator [F (2, 181) = 38.547, p = .001, Eta2 = .299]

Summary

Introduction

Types of recall: Immediate vs. delayed recallDecades of research show that standard performance validity tests (PVTs) assessing short-time memory (e.g., recall, recognition) are especially effective in detecting invalid performances (Bigler, 2014; Larrabee, 2003). Forced-choice testing has been deemed the best method to discriminate invalid responses from genuine performances by numerous empirical studies (Denning, 2012; Gunner et al, 2012; Inman & Berry, 2002; Strauss et al, 2002), reviews (Leighton et al, 2014), and metaanalyses (Crişan et al, 2021; Sollman & Berry, 2011). Despite their effectiveness, forced-choice measures are not infallible, as some tests are vulnerable to coaching (i.e., coached simulators producing above-chance performances on recognition tests), and online information is available about them (Rüsseler et al, 2008; Strauss et al, 2002). We designed two experimental studies to test the effectiveness of recall tasks in discriminating simulated from genuine cognitive impairment

Methods

Results

Discussion

Conclusion