Embedded Performance Validity Tests Research Articles

Detecting malingered neurocognitive dysfunction: Comparative analysis of freestanding and embedded performance validity tests

Detecting malingered neurocognitive dysfunction is a major issue in forensic context, particularly in legal proceeding/insurance assessment after a traumatic brain injury (TBI), condition frequently associated with persistent cognitive impairments that may potentially be related to malingering. Consequently, research has devoted considerable efforts on developing tools to verify symptoms authenticity. This study compared two freestanding performance validity tests (PVTs) (Amsterdam Short-Term Memory Test—ASTM; Word Completion Memory Test—WCMT) and five embedded PVTs (Rey Complex Figure Test—RCFT—Copy and Recall trials; Reliable Digit Span—RDS; Rey Auditory Verbal Learning Test—RAVLT—Recognition and Total Learning trials) in a sample of 120 participants, including 15 patients with TBI (Mage = 44.40), 52 experimental simulators (Mage = 29.52) and 53 control (Mage = 29.77). Group performance was analyzed to assess tests’ discriminatory power, and Receiver Operating Characteristic (ROC) curves were used to examine tools’ sensitivity and specificity. Results indicated that experimental simulators performance on the ASTM, WCMT, and RAVLT differed significantly from TBI patients and controls. The RDS and RCFT did not discriminate experimental simulators from TBI group. ROC curves analysis reveals that the most accurate tests in this battery for detecting malingering were the ASTM and the RAVLT-Total Learning score. This study offers implications for identifying malingering in medico-legal settings, underscoring the importance of incorporating PVTs into clinical practice.

B - 131 Validation of the California Verbal Learning Test-II-Short Form Forced Choice Paradigm as an Indicator of Performance Validity

Abstract Objective Although the Forced Choice (FC) paradigm of the standard California Verbal Learning Test-Second Edition (CVLT-II) has been extensively validated as an embedded performance validity test (PVT), no studies have validated the CVLT-II Short Form (SF) against a standard battery of external PVTs. Method Participants included 98 veterans (Mage = 69.7 ± 9.0; 86.7% cisgender-male) who completed the CVLT-II-SF FC and the following independent criterion freestanding/embedded PVTS as part of comprehensive outpatient neuropsychological evaluations: the Advanced Clinical Solutions Word Choice Test, Test of Memory Malingering-Trial 1, and Reliable Digit Span from the Wechsler Adult Intelligence Test-Fourth Edition Digit Span subtest. Invalid performance was identified using two different criteria: (1) failing ≥2 PVTs in addition to other clinical indicators of poor engagement, or (2) failing one PVT. Receiver Operator Characteristic (ROC) curves tested the classification accuracy of identifying participants in the invalid group versus those with valid presentations and with or without cognitive impairment. Results Using both criteria, ROC analyses accurately identified invalidity (AUCs = 0.708–0.807) in the total sample, with cut-scores of ≤7 or ≤ 8 on the CVLT-II-SF FC exhibiting adequate specificities (0.900–0.986) and sensitivities of 0.407–0.500. Classification accuracy was acceptable for differentiating the invalid and cognitively intact groups (AUCs = 0.734–0.829) at a cut score of ≤8 (specificities = 0.923–0.958, sensitivities = 0.500–0.704), but was variably acceptable for differentiating the invalid and cognitively impaired groups (AUCs = 0.684–0.794; cut-score of ≤7 with specificities = 0.962–0.978 and sensitivities = 0.229–0.407). Conclusions Our findings support the CVLT-II-SF FC trial as an embedded PVT, particularly when differentiating cognitively intact examinees from those with noncredible presentations.

B - 142 Reason to Forget Use of (Most) Digit Span Validity Measures in Aging Clinical Samples

Abstract Objective The Digit Span (DS) subtest has survived every iteration of the Wechsler Adult Intelligence Scale (WAIS), and the reliable digit span (RDS forward+backward) is an established embedded performance validity test (PVT). The only known DS meta-analysis drew near equivalency between the WAIS-R and WAIS-III RDS and age-corrected scaled score (ACSS) validity cutoffs, recommending use of either. The WAIS-IV included a new DS sequencing subtest. Alternate and enhanced RDS measures, adding sequencing, were derived. However, aging samples often perseverate on sequencing. The aims of this study are to confirm and update the faulty use of the enhanced RDS in the aging, establish that use of the previously recommended DS ACSS cutoff (now including sequencing) is similarly faulty, and update all DS PVT cutoff specificities in a clinical aging sample. Method Memory Clinic data were taken from 350 individuals with neuropsychologist-neurologist consensus diagnoses including 111 individuals with early Alzheimer’s disease (ad), 181 with mild cognitive impairment (MCI; 99 amnestic; 82 non-amnestic) and 58 with subjective cognitive complaints (SCC). Results Most DS embedded PVTs (e.g., ACSS<6, Enhanced RDS < 11) present with poor specificity in aging clinical populations (e.g., Χ2 (3350) = 26.331,p = <0.001), while the traditional RDS (< 6) excludes sequencing and retains the best relative specificity (81–95%), as ad performs worst on sequencing (<0.001). Conclusions While prior research favored the DS ACSS in older individuals, this study provided support for the traditional RDS in a memory clinic population. Future directions include corroborating these findings using standalone PVTs and conducting an updated DS meta-analysis.

An Examination of WAIS-IV Digit Span Performance Inconsistency as a Novel Embedded Performance Validity Test Among Adults Clinically Referred for Attention-Deficit/Hyperactivity Disorder

Embedded performance validity tests (PVTs), like Digit Span PVTs from Wechsler Adult Intelligence Scale-Fourth Edition (WAIS-IV), offer a valuable means of evaluating validity without extending administration time. This study investigated the utility of novel indices of performance inconsistency for WAIS-IV Digit Span (DS IRs) in the detection of invalid performance among 705 adults referred for ADHD evaluation. Results showed DS IR indices were inadequate in classifying overall validity status (areas under the curve = 0.52–0.59). Predictably, four established Digit Span PVTs effectively distinguished between valid and invalid performance score groups (areas under the curve = 0.74–0.78) with 32–49% sensitivity and 86–93% specificity at optimal cut-scores. Overall, individuals with noncredible performance scores did not differ significantly from those with valid scores regarding performance inconsistency on WAIS-IV Digit Span.

The Impact of Increasing Material-Specific Verbal and Visual Memory Impairment Severity on Embedded Performance Validity Tests in the Rey Auditory Verbal Learning Test and Brief Visuospatial Memory Test-Revised

The Impact of Increasing Material-Specific Verbal and Visual Memory Impairment Severity on Embedded Performance Validity Tests in the Rey Auditory Verbal Learning Test and Brief Visuospatial Memory Test-Revised

Replicating the classification accuracy of the Verbal Paired Associates and Visual Reproduction recognition trials as embedded performance validity tests.

This study was designed to replicate previous research on the clinical utility of the Verbal Paired Associates (VPA) and Visual Reproduction (VR) subtests of the WMS-IV as embedded performance validity tests (PVTs) and perform a critical item (CR) analysis within the VPA recognition trial. Archival data were collected from a mixed clinical sample of 119 adults (MAge = 42.5, MEducation = 13.9). Classification accuracy was computed against psychometrically defined criterion groups based on the outcome of various free-standing and embedded PVTs. Age-corrected scaled scores ≤ 6 were specific (.89-.98) but had variable sensitivity (.36-.64). A VPA recognition cutoff of ≤ 34 produced a good combination of sensitivity (.46-.56) and specificity (.92-.93), as did a VR recognition cutoff of ≤ 4 (.48-.53 sensitivity at .86-.94 specificity). Critical item analysis expanded the VPA's sensitivity by 3.5%-7.0% and specificity by 5%-8%. Negative learning curves (declining output on subsequent encoding trials) were rare but highly specific (.99-1.00) to noncredible responding. Results largely support previous reports on the clinical utility of the VPA and VR as embedded PVTs. Sample-specific fluctuations in their classification accuracy warrant further research into the generalizability of the findings. Critical item analysis offers a cost-effective method for increasing confidence in the interpretation of the VPA recognition trial as a PVT. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

The impact of race and other demographic factors on the false positive rates of five embedded Performance Validity Tests (PVTs) in a Veteran sample

ABSTRACT Introduction It is common to use normative adjustments based on race to maintain accuracy when interpreting cognitive test results during neuropsychological assessment. However, embedded performance validity tests (PVTs) do not adjust for these racial differences and may result in elevated rates of false positives in African American/Black (AA) samples compared to European American/White (EA) samples. Methods Veterans without Major Neurocognitive Disorder completed an outpatient neuropsychological assessment and were deemed to be performing in a valid manner (e.g., passing both the Test of Memory Malingering Trial 1 (TOMM1) and the Medical Symptom Validity Test (MSVT), (n = 531, EA = 473, AA = 58). Five embedded PVTs were administered to all patients: WAIS-III/IV Processing Speed Index (PSI), Brief Visuospatial Memory Test-Revised: Discrimination Index (BVMT-R), TMT-A (secs), California Verbal Learning Test-II (CVLT-II) Forced Choice, and WAIS-III/IV Digit Span Scaled Score. Individual PVT false positive rates, as well as the rate of failing two or more embedded PVTs, were calculated. Results Failure rates of two embedded PVTs (PSI, TMT-A), and the total number of PVTs failed, were higher in the AA sample. The PSI and TMT-A remained significantly impacted by race after accounting for age, education, sex, and presence of Mild Neurocognitive Disorder. There were PVT failure rates greater than 10% (and considered false positives) in both groups (AA: PSI, TMT-A, and BVMT-R, 12–24%; EA: BVMT-R, 17%). Failing 2 or more PVTs (AA = 9%, EA = 4%) was impacted by education and Mild Neurocognitive Disorder but not by race. Conclusions Individual (timed) PVTs showed higher false positive rates in the AA sample even after accounting for demographic factors and diagnosis of Mild Neurocognitive Disorder. Requiring failure on 2 or more embedded PVTs reduced false positive rates to acceptable levels across both groups (10% or less) and was not significantly influenced by race.

Cross-validation of the Trail Making Test as a non-memory-based embedded performance validity test among veterans with and without cognitive impairment

ABSTRACT Objective This study cross-validated multiple Trail Making Test (TMT) Parts A and B scores as non-memory-based embedded performance validity tests (PVTs) for detecting invalid neuropsychological performance among veterans with and without cognitive impairment. Method Data were collected from a demographically and diagnostically diverse mixed clinical sample of 100 veterans undergoing outpatient neuropsychological evaluation at a Southwestern VA Medical Center. As part of a larger battery of neuropsychological tests, all veterans completed TMT A and B and four independent criterion PVTs, which were used to classify veterans into valid (n = 75) and invalid (n = 25) groups. Among the valid group 47% (n = 35) were cognitively impaired. Results Among the overall sample, all embedded PVTs derived from TMT A and B raw and demographically corrected T-scores significantly differed between validity groups (ηp2 = .21–.31) with significant areas under the curve (AUCs) of .72–.78 and 32–48% sensitivity (≥91% specificity) at optimal cut-scores. When subdivided by cognitive impairment status (i.e., valid-unimpaired vs. invalid; valid-impaired vs. invalid), all TMT scores yielded significant AUCs of .80–.88 and 56%–72% sensitivity (≥90% specificity) at optimal cut-scores. Among veterans with cognitive impairment, neither TMT A or B raw scores were able to significantly differentiate the invalid from the valid-cognitively impaired group; however, demographically corrected T-scores were able to significantly differentiate groups but had poor classification accuracy (AUCs = .66–.68) and reduced sensitivity of 28%–44% (≥91% specificity). Conclusions Embedded PVTs derived from TMT Parts A and B raw and T-scores were able to accurately differentiate valid from invalid neuropsychological performance among veterans without cognitive impairment; however, the demographically corrected T-scores generally were more robust and consistent with prior studies compared to raw scores. By contrast, TMT embedded PVTs had poor accuracy and low sensitivity among veterans with cognitive impairment, suggesting limited utility as PVTs among populations with cognitive dysfunction.

21 A Comparison of the Memory and Non-Memory Based Performance Validity Measures for Detecting Invalid Neuropsychological Test Performance among Individuals with and without Memory Impairment

Objective:Few to no studies have directly compared the relative classification accuracies of the memory-based (Brief Visuospatial Memory Test-Revised Recognition Discrimination [BVMT-R RD] and Rey Auditory Verbal Learning Test Forced Choice [RAVLT FC]) and non-memory based (Reliable Digit Span [RDS] and Stroop Color and Word Test Word Reading trial [SCWT WR]) embedded performance validity tests (PVTs). This study’s main objective was to evaluate their relative classification accuracies head-to-head, as well as examine how their psychometric properties may vary among subgroups with and without genuine memory impairment.Participants and Methods:This cross-sectional study included 293 adult patients who were administered the BVMT-R, WAIS-IV Digit Span, RAVLT and SCWT during outpatient neuropsychological evaluation at a Midwestern academic medical center. The overall sample was 58.0% female, 36.2% non-Hispanic White, 41.3% non-Hispanic Black, 15.7% Hispanic, 4.8% Asian/Pacific Islander, and 2.0% other, with a mean age of 45.7 (SD=15.8) and a mean education of 13.9 years (SD=2.8). Three patients had missing data, resulting in a final sample size of 290. Two hundred thirty-three patients (80%) were classified as having valid neurocognitive performance and 57 (20%) as having invalid neurocognitive performance based on performance across four independent, criterion PVTs (i.e., Test of Malingering Memory Trial 1, Word Choice Test, Dot Counting Test, Medical Symptom Validity Test). Of those with valid neurocognitive performance, 76 (48%) patients were considered as having genuine memory impairment through a memory composite band score (T<37 for (RAVLT Delayed Recall T-score + BVMT-R Delay Recall T-score/2).Results:The average memory composite band score for valid neurocognitive scores was T = 49.63 as compared to T = 27.57 for genuine memory impairment individuals. Receiver operating characteristic [ROC] curve analyses yielded significant areas under the curve (AUCs=.79-.87) for all four validity indices (p’s < .001). When maintaining acceptable specificity (91%-95%), all validity indices demonstrated acceptable yet varied sensitivities (35%-65%). Among the subgroup with genuine memory impairment, ROC curve analyses yielded significantly lower AUCs (.64-.69) for three validity indices (p’s < .001), except RDS (AUC=.644). At acceptable specificity (88%-93%), they yielded significantly lower sensitivities across indices (19%-39%). In the current sample, RAVLT FC and BVMT-R RD had the largest changes in sensitivities, with 19% and 26% sensitivity/90%-92% specificity at optimal cut-scores of <10 and <2, respectively, for individuals with memory impairment, compared to 65% and 61% sensitivity/94% specificity at optimal cut-scores of <13 and <4, respectively, for those without memory impairment.Conclusions:Of the four validity scales, memory-based embedded PVTs yielded higher sensitivities while maintaining acceptable specificity compared to non-memory based embedded PVTs. However, they were also susceptible to the greatest declines in sensitivity among the subgroup with genuine memory impairment. As a result, careful consideration should be given to using memory-based embedded PVTs among individuals with clinically significant memory impairment based on other sources of information (e.g., clinical history, behavioral observation).

87 Examining the use of the Embedded Performance Validity Test in the Brief Visuospatial Memory Test-Revised Among Spanish-Speaking Patients

Objective:The Brief Visuospatial Memory Test-Revised (BVMT-R) Recognition Discrimination (RD) index has emerged as an embedded performance validity test (PVT). However, there do not appear to be any studies that have examined its utility in Spanish-speaking samples. This pilot study examined the classification accuracy of the BVMT-R RD for detecting performance invalidity in a Spanish-speaking forensic sample.Participants and Methods:This cross-sectional study utilized a sample of 89 Spanish speakers that were administered the BVMT-R during an outpatient neuropsychological evaluation. Out of the 89 Spanish speakers, 43 were subjects in litigation, 32 were neurological patients evaluated for clinical purposes, and 14 were healthy controls. The sample was 67% male/33% female, 53% South American, 33% Caribbean (Dominican, Puerto Rican, Cuban), 10% Central American, 3% North American (Mexican), and 1% Spanish, with a mean age of 44.2 years (SD = 14.2; range = 20-78) and mean education of 11 years (SD = 3.7; range = 0-20). Test administration for each patient was completed in Spanish by a fluent, Spanish-speaking examiner. In total, 64/89 (72%) were classified as valid and 25/89 (28%) as invalid based on performance across the Test of Memory Malingering (TOMM), at least one additional PVT (Rey-15 item memory test; Rey Dot Counting Test; Reliable Digit Span; WHO-AVLT recognition trial) and objective diagnostic criteria identifying invalid performance. Analyses included three univariate analyses of variance (ANOVA), with the groups (healthy vs neurological vs litigation) as independent variables and performance on BVMT-RD as the dependent variable.Results:Statistically significant differences among the groups were found F(2,86)=8.32, p < .001). Post-hoc analysis (Scheffe test) showed the mean of the litigation group to be significantly lower than the means of the other two groups (healthy and neurological), which showed no difference between them. An ANOVA with validity groups as the fixed factor and BVMT-R RD index as the dependent variable was significant F(1,85)= 21.02, p <.001). Results of a ROC curve analysis yielded statistically significant AUC (.794). The optimal cut-score was BVMT-R RD < 5 (48% sensitivity/88% specificity).Conclusions:Results of the BVMT-R RD index in this Spanish-speaking population differed by subgroup, with worse performance seen in individuals involved in litigation, compared to those who were not (healthy and neurological). Notably, the BVMT-R RD index significantly differentiated validity groups, maintaining adequate sensitivity and good specificity. Overall, results demonstrate promise for BVMT-RD as a PVT for Spanish-speaking populations.

80 Demographic Factors Affecting Sensitivity Rates of Performance Validity Tests in Spanish-Speaking Forensic Litigants.

Objective:This retrospective study compared base rates of failure on a series of standalone and embedded performance validity tests (PVTs) in a sample of Spanish-speaking forensic litigants and explored the impact of demographic factors on PVT performance.Participants and Methods:62 Spanish-speaking participants involved in litigation (primarily for work-related mTBI) underwent outpatient neuropsychological evaluation. Country of origin spanned South American (56.5%), Caribbean (22.5%), Central American (16.1%), North American (3.2%), and Spanish (1.6%) regions. Of this sample, 56 completed the Test of Memory Malingering (TOMM), 45 completed the Rey Fifteen Item Test (RFIT), and 49 completed the Dot Counting Test (DCT). Embedded validity measures, Reliable Digit Span (RDS) and the WHO-Auditory Verbal Learning Test (WHO-AVLT) were completed by 32 and 48 participants, respectively.Results:Effects of age (M=42.4, SD=11.72) and region of origin did not significantly impact overall performance on any measure. Mean scores across all standalone PVTs were below cutoffs, which have been previously suggested for use with Spanish-speaking populations (TOMM <40, RFIT total <21, DCT e-score >18). Overall base rates of failure were as follows: 52.5% TOMM (T1 M= 37.5, SD=10.7; T2 M=35.1, SD=10.6), 64.9% RFIT (M=17.8, SD=7.8) 57.6% DCT (M=18.3, SD=8.8), 51.1% RDS (M=6.1, SD=1.6), 29.4% WHO-AVLT (M=10.7, SD=3.9. Years of education (M=9.98, 3.96) was significantly correlated with RFIT total score (r(43) = .48, p<.01) and DCT e-score (r(47) = -.34, p<.05. When stratified by level of education (0-6, 7-11, and 12+), a large discrepancy in base rate of failure was observed on the RFIT, with failures in 92% of participants with less than six years of education, as compared to 52% and 59% failure in those with 7-11 and 12+ years, respectively. Variability in base rates of DCT failure across levels of education, although less extreme than on the RFIT, again demonstrated higher rates of failure in participants with less than six years of education (0-6: 71%, 7-11: 54%, 12+: 52%).Conclusions:These findings add to the existing literature surrounding measurement of suboptimal effort in Spanish-speaking populations. Base rates of PVT failure on both standalone and embedded measures were generally much higher than those reported in prior studies of forensic or compensation-seeking groups, including some with Spanish speaking participants. These high rates of failure are likely attributable, at least in part, to sample characteristics, due to the high proportion of individuals engaged in litigation associated with workplace injuries on construction sites at the study location. Such findings illustrate the importance of a thorough effort assessment for this population. Finally, results demonstrating reduced specificity of the RFIT in Spanish-speaking participants with less than six years of education, suggesting caution is warranted for its use in neuropsychological evaluations with such individuals.

A - 183 Novel Performance Validity Indicators for Frequently Used Memory Tests Using Infrequent Patterns of Memory Performance.

Determining performance validity is an imperative part of neuropsychological assessment in concussion management. Several existing embedded performance validity tests (PVTs) have been developed for popular measures of memory, Hopkins Verbal Learning Test Revised (HVLT-R) and Brief Visuospatial Memory Test Revised (BVMT-R). However, many PVTs utilize cognitive performance measures to classify performance invalidity, which increases the likelihood of false positives, especially in clinical populations. To reduce this possibility, a novel embedded PVT was developed utilizing infrequent patterns of performance in a primarily amnestic, dementia population. Neuropsychological data for 50 adult outpatients (Mean Age = 73, Range = 50-97) who met criteria for Major Neurocognitive Disorder (MND) were reviewed. Infrequent patterns of performance (IPOP) were identified on the HVLT-R and BVMT-R, including inconsistency between recall and recognition, unusually high intrusions, exceedingly poor learning, and lack of primary effect. Scaled scores were developed from the IPOP scores (0 = 0-74%, 1 = 75-94%, 2 = 95-99%). Total IPOP scaled scores were compared to existing embedded PVTs, which utilized recognition discrimination and percent retention scores, to examine classification rates. Failure rates on the embedded PVTs for the HVLT-R were 16-42% and BVMT-R 38-58% whereas elevated IPOP scaled scores were identified in less than 15% of the sample. High frequencies of embedded PVT failures were found in a clinical sample of patients with MND, which is concerning for false positive misclassification. Infrequent patterns of memory performances identified in a dementia population were less frequent and could potentially be utilized to examine performance validity in persons suspected of producing invalid memory performances.

H - 53 Performance Validity in a Mexican Population.

The aim of the present study was to cross-validate recent research on three embedded performance validity tests (PVTs) in a Spanish-speaking Mexican sample. The sample consisted of 52 male and 77 female participants residing in Mexico with a mean age of 25.26 (SD = 10.45). Participants were neurologically and psychologically healthy and completed a neuropsychological battery in Spanish. The Reliable Digit Span, Phonemic Fluency Test (i.e., FAS), and Semantic Fluency Tests were used to evaluate failure rates in our sample. Daugherty and colleagues' (2022) PVT cutoff scores were used. Chi-square analysis was used to evaluate failure rates between the groups. Chi-square analysis revealed no significant failure rate differences between gender groups on any PVTs. However, we found that 12% of the male group failed the Phonemic Fluency Test compared to 5% of the female group, X2 = 1.747, p = 0.186. Meanwhile, 2% of the male group failed the Semantic Fluency Test compared to 0% of the female group, X2 = 1.492, p = 0.222. Finally, 6% of the male group failed the Reliable Digit Span compared to 17% of the female group, X2 = 3.529, p = 0.060. Our data shows that male participants demonstrate higher failure rates on verbal fluency tasks than females. Moreover, males had lower failure rates compared to females on Reliable Digit Span. Further exploration with bigger sample sizes is needed to explore what may be influencing validity test performance between gender groups. More studies in this area will help researchers and clinicians determine if persons completing cognitive tasks in Spanish are proving adequate effort.

Logical memory, visual reproduction, and verbal paired associates are effective embedded validity indicators in patients with traumatic brain injury

Objective This study was design to evaluate the potential of the recognition trials for the Logical Memory (LM), Visual Reproduction (VR), and Verbal Paired Associates (VPA) subtests of the Wechsler Memory Scales–Fourth Edition (WMS-IV) to serve as embedded performance validity tests (PVTs). Method The classification accuracy of the three WMS-IV subtests was computed against three different criterion PVTs in a sample of 103 adults with traumatic brain injury (TBI). Results The optimal cutoffs (LM ≤ 20, VR ≤ 3, VPA ≤ 36) produced good combinations of sensitivity (.33–.87) and specificity (.92–.98). An age-corrected scaled score of ≤5 on either of the free recall trials on the VPA was specific (.91–.92) and relatively sensitive (.48–.57) to psychometrically defined invalid performance. A VR I ≤ 5 or VR II ≤ 4 had comparable specificity, but lower sensitivity (.25–.42). There was no difference in failure rate as a function of TBI severity. Conclusions In addition to LM, VR, and VPA can also function as embedded PVTs. Failing validity cutoffs on these subtests signals an increased risk of non-credible presentation and is robust to genuine neurocognitive impairment. However, they should not be used in isolation to determine the validity of an overall neurocognitive profile.

Base rates of non-credible performance in a post-secondary student sample seeking accessibility accommodations

Objective: Performance Validity Tests (PVTs) have been used to identify non-credible performance in clinical, medicolegal, forensic, and, more recently, academic settings. The inclusion of PVTs when administering psychoeducational assessments is essential given that specific accommodation such as flexible deadlines and increased writing time can provide an external incentive for students without disabilities to feign symptoms. Method: The present study used archival data to establish base rates of non-credible performance in a sample of post-secondary students (n = 1045) who underwent a comprehensive psychoeducational evaluation for the purposes of obtaining academic accommodations. In accordance with current guidelines, non-credible performance was determined by failure on two or more freestanding or embedded PVTs. Results: 9.4% of participants failed at least two of the PVTs they were administered, of which 8.5% failed two PVTs, and approximately 1% failed three PVTs. Base rates of failure for specific PVTs ranged from 25% (b Test) to 11.2% (TOVA). Conclusions: The present study found a lower base rate of non-credible performance than previously observed in comparable populations. This likely reflects the utilization of conservative criteria in detecting non-credible performance to avoid false positives. By contrast, inconsistent base rates previously found in the literature may reflect inconsistent methodologies. These results further emphasize the importance of administering multiple PVTs during psychoeducational assessments. The implications of these findings can further inform clinicians administering assessments in academic settings and aid in the appropriate utilization of PVTs in psychoeducational evaluation to determine accessibility accommodations.

New Child and Adolescent Memory Profile Embedded Performance Validity Test.

It is essential to interpret performance validity tests (PVTs) that are well-established and have strong psychometrics. This study evaluated the Child and Adolescent Memory Profile (ChAMP) Validity Indicator (VI) using a pediatric sample with traumatic brain injury (TBI). A cross-sectional sample of N = 110 youth (mean age = 15.1years, standard deviation [SD] = 2.4 range = 8-18) on average 32.7weeks (SD = 40.9) post TBI (71.8% mild/concussion; 3.6% complicated mild; 24.6% moderate-to-severe) were administered the ChAMP and two stand-alone PVTs. Criterion for valid performance was scores above cutoffs on both PVTs; criterion for invalid performance was scores below cutoffs on both PVTs. Classification statistics were used to evaluate the existing ChAMP VI and establish a new VI cutoff score if needed. There were no significant differences in demographics or time since injury between those deemed valid (n = 96) or invalid (n = 14), but all ChAMP scores were significantly lower in those deemed invalid. The original ChAMP VI cutoff score was highly specific (no false positives) but also highly insensitive (sensitivity [SN] = .07, specificity [SP] = 1.0). Based on area under the curve (AUC) analysis (0.94), a new cutoff score was established using the sum of scaled scores (VI-SS). A ChAMP VI-SS score of 32 or lower achieved strong SN (86%) and SP (92%). Using a 15% base rate, positive predictive value was 64% and negative predictive value was 97%. The originally proposed ChAMP VI has insufficient SN in pediatric TBI. However, this study yields a promising new ChAMP VI-SS, with classification metrics that exceed any other current embedded PVT in pediatrics.

Specificity of Embedded Performance Validity Tests in Elderly Veterans with Mild and Major Neurocognitive Disorder.

This study explored the specificity of four embedded performance validity tests (PVTs) derived from common neuropsychological tasks in a sample of older veterans with verified cognitive decline and whose performance was deemed valid by licensed psychologists. Participants were 180 veterans who underwent comprehensive neuropsychological evaluation, were determined to have valid performance following profile analysis/conceptualization, and were diagnosed with mild neurocognitive disorder (i.e., MCI; n = 64) or major neurocognitive disorder (i.e., Dementia; n =116). All participants completed at least one of four embedded PVTs: Reliable Digit Span (RDS), California Verbal Learning Test-2nd ed. Short Form (CVLT-II SF) Forced choice, Trails B:A, and Delis-Kaplan Executive Function System (DKEFS) Letter and Category Fluency. Adequate specificity (i.e., ≥90%) was achieved at modified cut-scores for all embedded PVTs across MCI and Dementia groups. Trails B:A demonstrated near perfect specificity at its traditional cut-score (Trails B:A < 1.5). RDS ≤ 5 and CVLT-II SF Forced Choice ≤7 led to <10% false positive classification errors across MCI and dementia groups. DKEFS Letter and Category Fluency achieved 90% specificity at extremely low normative cut-scores. RDS, Trails B:A, and CVLT-II SF Forced Choice reflect promising embedded PVTs in the context of dementia evaluations. DKEFS Letter and Category Fluency appear too sensitive to genuine neurocognitive decline and, therefore, are inappropriate PVTs in adults with MCI or dementia. Additional research into embedded PVT sensitivity (via known-groups or analogue designs) in MCI and dementia is needed.

Racial differences in positive findings on embedded performance validity tests

Introduction Embedded performance validity tests (PVTs) may show increased positive findings in racially diverse examinees. This study examined positive findings in an older adult sample of African American (AA) and European American (EA) individuals recruited as part of a study on aging and cognition. Method The project involved secondary analysis of deidentified National Alzheimer’s Coordinating Center data (N = 22,688). Exclusion criteria included diagnosis of dementia (n = 5,550), mild cognitive impairment (MCI; n = 5,160), impaired but not MCI (n = 1,126), other race (n = 864), and abnormal Mini Mental State Examination (MMSE < 25; n = 135). The initial sample included 9,853 participants (16.4% AA). Propensity score matching matched AA and EA participants on age, education, sex, and MMSE score. The final sample included 3,024 individuals with 50% of participants identifying as AA. Premorbid ability estimates were calculated based on demographics. Failure rates on five raw score and six age-adjusted scaled score PVTs were examined by race. Results Age, education, sex, MMSE, and premorbid ability estimate were not significantly different by race. Thirteen percent of AA and 3.8% of EA participants failed two or more raw score PVTs (p < .0001). On age-adjusted PVTs, 20.6% of AA and 5.9% of EA participants failed two or more (p < .0001). Conclusions PVT failure rates were significantly higher among AA participants. Findings indicate a need for cautious interpretation of embedded PVTs with underrepresented groups. Adjustments to embedded PVT cutoffs may need to be considered to improve diagnostic accuracy.

A Direct Comparison of 10 WAIS-IV Digit Span Embedded Validity Indicators among a Mixed Neuropsychiatric Sample with Varying Degrees of Cognitive Impairment.

Reliable Digit Span (RDS), RDS-Revised (RDS-R), and age-corrected scaled score (ACSS) have been previously validated as embedded performance validity tests (PVTs) from the Wechsler Adult Intelligence Scale-IV Digit Span subtest (WAIS-IV DS). However, few studies have directly compared the relative utility of these and other proposed WAIS-IV DS validity indicators within a single sample. This study compared classification accuracies of 10 WAIS-IV DS indices in a mixed neuropsychiatric sample of 227 outpatients who completed a standardized neuropsychological battery. Participants with ≤1 PVT failures of the four, freestanding criterion PVTs constituted the valid group (n = 181), whereas those with ≥2 PVT failures formed the invalid group (n = 46). Among the valid group, 113 met criteria for mild cognitive impairment (MCI). Classification accuracies for all DS indicators were statistically significant across the overall sample and subsamples with and without MCI, apart from indices derived from the Forward trial in the MCI sample. DS Sequencing ACSS, working memory RDS (wmRDS), and DS ACSS emerged as the most effective predictors of validity status, with acceptable to excellent classification accuracy for the overall sample (AUCs = 0.792-0.816; 35%-50% sensitivity/88%-96% specificity). Although most DS indices demonstrated clinical utility as embedded PVTs, DS Sequencing ACSS, wmRDS, and DS ACSS may be particularly robust to cognitive impairment, minimizing risk of false positive errors while identifying noncredible performance. Moreover, DS indices incorporating data from multiple trials (i.e., wmRDS, DS ACSS) also generally yielded greater classification accuracy than those derived from a single trial.

Embedded cognitive and emotional/affective self-reported symptom validity indices on the patient competency rating scale

ABSTRACT Objective Although there is an abundance of research on stand-alone and embedded performance validity tests and stand-alone symptom validity tests (SVTs), less emphasis has been placed on embedded SVTs. The goal of the current study was to examine the ability of embedded indicators within the Patient Competency Rating Scale (PCRS) to separately detect invalid cognitive and/or emotional/affective symptom responding. Method Participants included 299 veterans assessed in a VA medical center epilepsy monitoring unit from 2013–2017 (mean age = 48.8 years, SD = 13.5 years). Two SVT composites were created; self-reported cognitive symptom validity (SVT-C) and self-reported emotional/affective symptom validity (SVT-E). Groups were compared on PCRS total and index scores (i.e., cognitive, activities of daily living, emotional, and interpersonal competencies) using ANOVAs. Receiver operating characteristic (ROC) curve analyses assessed the classification accuracy of the PCRS total and index scores for SVT-C and SVT-E. Results In ANOVAs, SVT-C was significantly associated with all PCRS indices, while SVT-E was only significantly associated with the PCRS total, emotional, and interpersonal competency indices. Although the PCRS-T ≤ 90 had the strongest classification of SVT-C and SVT-E (specificities: .90, sensitivities: .44 to .50), PCRS index scores showed suggestive evidence of domain specificity, with PCRS-ADL ≤22, PCRS-C ≤ 20, and PCRS-CADL ≤45 best classifying SVT-C (specificities: .92, sensitivities: .33) and the PCRS-E ≤ 18 best classifying the SVT-E group (specificity: .93, sensitivity: .40). Conclusion Results suggest the PCRS may be used to obtain clinically useful information while including embedded indicators that can assess cognitive and/or emotional/affective symptom invalidity.