Criterion Performance Validity Tests Research Articles

The Word Memory Test genuine memory impairment profile discriminates genuine memory impairment from invalid performance in a mixed clinical sample with cognitive impairment

Objective: The Word Memory Test (WMT) is a memory-based performance validity test (PVT) with adjusted interpretive criteria (Genuine Memory Impairment Profile; GMIP) proposed for those with cognitive impairment (CI). The GMIP has been criticized for poor discriminability; thus, this study sought to validate the GMIP in a mixed clinical sample. Analyses aimed to demonstrate enhanced detection of invalid neuropsychological test performance while minimizing false positives in a sample including patients with mild CI.Method: Data included 116 Veterans who completed the WMT and four criterion PVTs during clinical evaluation. This sample was 37.1% valid-CI, 33.6% valid-unimpaired, and 29.3% invalid per criterion PVTs. Group differences in WMT performance and diagnostic accuracy were assessed.Results: WMT performance significantly differed between validity groups (Wilk’s Λ = .40, F[6, 109] = 27.62, p < .001, = .60) with invalid participants scoring significantly lower across all WMT indices, with larger effect sizes for the effort subtests ( = .44–.55) than memory subtests ( = .16–.32). CI also had a significant effect on WMT performance (Wilk’s Λ = .65, F[6, 75] = 6.66, p < .001, = .35). Those with CI scored lower across all indices, with larger effect sizes for the memory subtests ( = .29–.30) relative to effort subtests ( = .18–.20). Standard WMT pass/fail criteria were sensitive (.97), but had unacceptable specificity (.66). GMIP sensitivity (.94) and specificity (.91) were robust with a DOR of 171.43.Conclusions: WMT indices were more significantly affected by performance validity than memory. However, with CI, GMIP criteria is essential to improve diagnostic accuracy and reduce false positive errors when identifying invalid performance.

Cross-validation of three Advanced Clinical Solutions performance validity tests: Examining combinations of measures to maximize classification of invalid performance

Use of multiple performance validity tests (PVTs) may best identify invalid performance, though few studies have examined the utility and accuracy of combining PVTs. This study examined the following PVTs in the Advanced Clinical Solutions (ACS) package to determine their utility alone and in concert: Word Choice Test (WCT), Reliable Digit Span (RDS), and Logical Memory Recognition (LMR). Ninety-three veterans participated in clinical neuropsychological evaluations to determine presence of cognitive impairment; 25% of the performances were deemed invalid via criterion PVTs. Classification accuracy of the ACS measures was assessed via receiver operating characteristic curves, while logistic regressions determined utility of combining these PVTs. The WCT demonstrated superior classification accuracy compared to the two embedded measures of the ACS, even in veterans with cognitive impairment. The two embedded measures (even when used in concert) exhibited inadequate classification accuracy. A combined model with all three ACS PVTs similarly demonstrated little benefit of the embedded indicators over the WCT alone. Results suggest the ACS WCT has utility for detecting invalid performance in a clinical sample with likely cognitive impairment, though the embedded ACS measures (RDS and LMR) may have limited incremental utility, particularly in individuals with cognitive impairment.

Examination of optimal embedded PVTs within the BVMT-R in an outpatient clinical sample

Objective: This study investigated whether indices within the Brief Visuospatial Memory Test – Revised (BVMT-R) could function as embedded performance validity measures in an outpatient clinical sample.Method: A sample of 138 neuropsychological outpatients was utilized; approximately 45% had a known or suspected external incentive. Patients were determined to be valid performers if they passed all criterion performance validity tests (PVTs) and determined to be invalid performers if they failed two or more PVTs. BVMT-R indices met criteria for optimal embedded PVTs if they were not significantly correlated with genuine cognitive dysfunction and if they adequately differentiated the validly from invalidly performing patient groups. Classification accuracy statistics for the indices were then calculated. Supplementary analyses were also calculated for a separate dementia sample.Results: Recognition Hits and Recognition Discrimination were identified as two optimal embedded PVTs for patients without dementia. Recognition Hits showed a sensitivity rate of 41% and a specificity rate of 95%. Recognition Discrimination showed a sensitivity rate of 54% and a specificity rate of 93%.Conclusion: Embedded BVMT-R PVTs are discussed in relation to previous research findings, which were obtained from veteran samples. Recognition Hits and Recognition Discrimination are now validated in a non-veteran clinical sample.

One-Minute PVT: Further Evidence for the Utility of the California Verbal Learning Test—Children’s Version Forced Choice Recognition Trial

To replicate previous research on the forced choice recognition trial for the California Verbal Learning Test—Children’s Version (FCR-C). Classification accuracy of the FCR-C was computed in 178 children referred for neuropsychological assessment to a tertiary care hospital. FCR-C ≤ 14 produced the best combination of sensitivity (.12–.42) and specificity (.88–.96). Unlike some of the criterion performance validity tests (PVTs), base rate of failure on the FCR-C was evenly distributed across age ranges. The FCR-C is a quick and inexpensive PVT, unaffected by cognitive maturation, and highly specific to psychometrically defined invalid performance.

Convergent, Discriminant, and Concurrent Validity of Nonmemory-Based Performance Validity Tests.

To supplement memory-based Performance Validity Tests (PVTs) in identifying noncredible performance, we examined the validity of the two most commonly used nonmemory-based PVTs-Dot Counting Test (DCT) and Wechsler Adult Intelligence Scale-Fourth edition (WAIS-IV) Reliable Digit Span (RDS)-as well as two alternative WAIS-IV Digit Span (DS) subtest PVTs. Examinees completed DCT, WAIS-IV DS, and the following criterion PVTs: Test of Memory Malingering, Word Memory Test, and Word Choice Test. Validity groups were determined by passing 3 (valid; n = 69) or failing ⩾2 (noncredible; n = 30) criterion PVTs. DCT, RDS, RDS-Revised (RDS-R), and WAIS-IV DS Age-Corrected Scaled Score (ACSS) were significantly correlated (but uncorrelated with memory-based PVTs). Combining RDS, RDS-R, and ACSS with DCT improved classification accuracy (particularly for DCT/ACSS) for detecting noncredible performance among valid-unimpaired, but largely not valid-impaired examinees. Combining DCT with ACSS may uniquely assess and best supplement memory-based PVTs to identify noncredible neuropsychological test performance in cognitively unimpaired examinees.

The D-KEFS Trails as performance validity tests.

This study was designed to examine the potential of the Delis-Kaplan Executive System (D-KEFS) version of the Trail Making Test (TMT) as a performance validity test (PVT). Data were collected from a mixed clinical sample of 157 consecutively referred outpatients (49% male, MAge = 47.1, MEducation = 13.6) undergoing neuropsychological assessment at an academic medical center in the northeastern United States. Sensitivity and specificity of the D-KEFS Trails to psychometrically defined invalid responding was calculated across various cutoffs and criterion PVTs. The D-KEFS Trails produced classification accuracy comparable to the original version of the TMT, hovering around the "Larrabee limit" (.50 sensitivity at .90 specificity). Different cutoffs (age-corrected scaled score ≤5 on Trails 1-3, ≤4 on Trails 4 and ≤8 on Trails 5) were needed to achieve the same classification accuracy across the five trials. Combining multiple cutoffs improved the signal detection performance. The study provides preliminary evidence of the utility of D-KEFS Trails as a PVT. Embedded PVTs are valuable, because they make a multivariate approach to validity assessment feasible. Combining validity indicators is superior to relying on single cutoffs. (PsycINFO Database Record

The Boston Naming Test as a Measure of Performance Validity

This study was designed to evaluate the potential of the Boston Naming Test (BNT) as a performance validity test (PVT). The classification accuracy of the BNT was examined against several criterion PVTs in a mixed clinical sample of 214 adult outpatients physician referred for neuropsychological assessment. Mean age was 46.7 (SD = 12.5); mean education was 13.5 (SD = 2.5). All participants were native speakers of English. A BNT raw score ≤ 50 produced high specificity (.87–.95), but low and variable sensitivity (.15–.41). Similarly, a T score ≤ 37 was specific (.87–.95), but not very sensitive (.15–.35) to psychometrically defined non-credible responding. Ipsative analyses (i.e., case-by-case review of individual PVT profiles) suggest that failing these cutoffs was associated with zero false positives when all available PVTs were taken into account. Results are consistent with previous reports that the validity cutoffs on the BNT have high positive predictive power, but low negative predictive power. As such, they are useful in ruling in invalid performance, but they cannot be used to rule it out.

Evaluating the accuracy of the Wechsler Memory Scale-Fourth Edition (WMS-IV) logical memory embedded validity index for detecting invalid test performance

ABSTRACTEmbedded performance validity tests (PVTs) allow for continuous assessment of invalid performance throughout neuropsychological test batteries. This study evaluated the utility of the Wechsler Memory Scale-Fourth Edition (WMS-IV) Logical Memory (LM) Recognition score as an embedded PVT using the Advanced Clinical Solutions (ACS) for WAIS-IV/WMS-IV Effort System. This mixed clinical sample was comprised of 97 total participants, 71 of whom were classified as valid and 26 as invalid based on three well-validated, freestanding criterion PVTs. Overall, the LM embedded PVT demonstrated poor concordance with the criterion PVTs and unacceptable psychometric properties using ACS validity base rates (42% sensitivity/79% specificity). Moreover, 15–39% of participants obtained an invalid ACS base rate despite having a normatively-intact age-corrected LM Recognition total score. Receiving operating characteristic curve analysis revealed a Recognition total score cutoff of < 61% correct improved specificity (92%) while sensitivity remained weak (31%). Thus, results indicated the LM Recognition embedded PVT is not appropriate for use from an evidence-based perspective, and that clinicians may be faced with reconciling how a normatively intact cognitive performance on the Recognition subtest could simultaneously reflect invalid performance validity.