Word Choice Test Research Articles

B - 137 Analyzing Completion Time on a Created Forced Choice Trial of the Hopkins Verbal Learning Test – Revised (HVLT-R) to Measure Engagement in a Mixed Clinical Sample

Abstract Objective We created a forced choice recognition trial (FCR) for the HVLT-R, including recording completion time (FCompT), to evaluate the utility of combining completion time with accuracy scores when using performance validity tests (PVTs). Method Participants included 29 clinically referred patients at a large medical center who completed the HVLT-R in addition to multiple free-standing and embedded PVTs including the TOMM, Word Choice Test (WCT), Reliable Digit Span (RDS), and the Dot Counting Test (DCT) as part of a larger battery of neuropsychological tests in an outpatient setting. Data from the sample (M age = 64.79, SD = 13.86; M education = 15.65, SD = 2.47; 62.1% females) were analyzed using correlation and linear regression. Results The HVLT-R FCR trial correlated with the TOMM-10, r(19) = 0.75, p = <0.001, TOMM trial 1, r(17) = 0.75, p = <0.01, the DCT r(12) = −0.71, p = <0.01, and the FCompT, r(29) = −0.72, p = <0.001. The FCompT was correlated with RDS, r(28) = −0.46, p = 0.03, TOMM-10, r(19) = −0.64, p = <0.01, and TOMM trial 1, r(17) = −0.61, p = <0.01. FCompT significantly predicted FCR accuracy b = −0.72, t(29) = −5.41, p < 0.001. Conclusions The HVLT-R FC and FCompT strongly correlated with free-standing and embedded PVTs. Data demonstrates that a longer completion time predicts poorer performance on FCR. Results demonstrate the value of measuring completion time for specific PVTs, in addition to providing support for the addition of a FCR trial to the HVLT-R.

B - 131 Validation of the California Verbal Learning Test-II-Short Form Forced Choice Paradigm as an Indicator of Performance Validity

Abstract Objective Although the Forced Choice (FC) paradigm of the standard California Verbal Learning Test-Second Edition (CVLT-II) has been extensively validated as an embedded performance validity test (PVT), no studies have validated the CVLT-II Short Form (SF) against a standard battery of external PVTs. Method Participants included 98 veterans (Mage = 69.7 ± 9.0; 86.7% cisgender-male) who completed the CVLT-II-SF FC and the following independent criterion freestanding/embedded PVTS as part of comprehensive outpatient neuropsychological evaluations: the Advanced Clinical Solutions Word Choice Test, Test of Memory Malingering-Trial 1, and Reliable Digit Span from the Wechsler Adult Intelligence Test-Fourth Edition Digit Span subtest. Invalid performance was identified using two different criteria: (1) failing ≥2 PVTs in addition to other clinical indicators of poor engagement, or (2) failing one PVT. Receiver Operator Characteristic (ROC) curves tested the classification accuracy of identifying participants in the invalid group versus those with valid presentations and with or without cognitive impairment. Results Using both criteria, ROC analyses accurately identified invalidity (AUCs = 0.708–0.807) in the total sample, with cut-scores of ≤7 or ≤ 8 on the CVLT-II-SF FC exhibiting adequate specificities (0.900–0.986) and sensitivities of 0.407–0.500. Classification accuracy was acceptable for differentiating the invalid and cognitively intact groups (AUCs = 0.734–0.829) at a cut score of ≤8 (specificities = 0.923–0.958, sensitivities = 0.500–0.704), but was variably acceptable for differentiating the invalid and cognitively impaired groups (AUCs = 0.684–0.794; cut-score of ≤7 with specificities = 0.962–0.978 and sensitivities = 0.229–0.407). Conclusions Our findings support the CVLT-II-SF FC trial as an embedded PVT, particularly when differentiating cognitively intact examinees from those with noncredible presentations.

21 A Comparison of the Memory and Non-Memory Based Performance Validity Measures for Detecting Invalid Neuropsychological Test Performance among Individuals with and without Memory Impairment

Objective:Few to no studies have directly compared the relative classification accuracies of the memory-based (Brief Visuospatial Memory Test-Revised Recognition Discrimination [BVMT-R RD] and Rey Auditory Verbal Learning Test Forced Choice [RAVLT FC]) and non-memory based (Reliable Digit Span [RDS] and Stroop Color and Word Test Word Reading trial [SCWT WR]) embedded performance validity tests (PVTs). This study’s main objective was to evaluate their relative classification accuracies head-to-head, as well as examine how their psychometric properties may vary among subgroups with and without genuine memory impairment.Participants and Methods:This cross-sectional study included 293 adult patients who were administered the BVMT-R, WAIS-IV Digit Span, RAVLT and SCWT during outpatient neuropsychological evaluation at a Midwestern academic medical center. The overall sample was 58.0% female, 36.2% non-Hispanic White, 41.3% non-Hispanic Black, 15.7% Hispanic, 4.8% Asian/Pacific Islander, and 2.0% other, with a mean age of 45.7 (SD=15.8) and a mean education of 13.9 years (SD=2.8). Three patients had missing data, resulting in a final sample size of 290. Two hundred thirty-three patients (80%) were classified as having valid neurocognitive performance and 57 (20%) as having invalid neurocognitive performance based on performance across four independent, criterion PVTs (i.e., Test of Malingering Memory Trial 1, Word Choice Test, Dot Counting Test, Medical Symptom Validity Test). Of those with valid neurocognitive performance, 76 (48%) patients were considered as having genuine memory impairment through a memory composite band score (T<37 for (RAVLT Delayed Recall T-score + BVMT-R Delay Recall T-score/2).Results:The average memory composite band score for valid neurocognitive scores was T = 49.63 as compared to T = 27.57 for genuine memory impairment individuals. Receiver operating characteristic [ROC] curve analyses yielded significant areas under the curve (AUCs=.79-.87) for all four validity indices (p’s < .001). When maintaining acceptable specificity (91%-95%), all validity indices demonstrated acceptable yet varied sensitivities (35%-65%). Among the subgroup with genuine memory impairment, ROC curve analyses yielded significantly lower AUCs (.64-.69) for three validity indices (p’s < .001), except RDS (AUC=.644). At acceptable specificity (88%-93%), they yielded significantly lower sensitivities across indices (19%-39%). In the current sample, RAVLT FC and BVMT-R RD had the largest changes in sensitivities, with 19% and 26% sensitivity/90%-92% specificity at optimal cut-scores of <10 and <2, respectively, for individuals with memory impairment, compared to 65% and 61% sensitivity/94% specificity at optimal cut-scores of <13 and <4, respectively, for those without memory impairment.Conclusions:Of the four validity scales, memory-based embedded PVTs yielded higher sensitivities while maintaining acceptable specificity compared to non-memory based embedded PVTs. However, they were also susceptible to the greatest declines in sensitivity among the subgroup with genuine memory impairment. As a result, careful consideration should be given to using memory-based embedded PVTs among individuals with clinically significant memory impairment based on other sources of information (e.g., clinical history, behavioral observation).

A - 185 Validating 10 Critical Items of the Word Choice Test (WCT-CI-10).

The purpose of the study was to identify ten critical items of the Word Choice Test (WCT) and validate the items as an abbreviated PVT (WCT-CI-10). The study included a diverse sample of veterans (M age = 55.87, SD = 14.10; M education = 13.89, SD = 2.55; 85.4% Male; 49.4% White, 33.7% Hispanic, 12.4% African American, 2.2% Multiracial, 1.1% American Indian, 1.1% Other). Chi-square analyses were used to determine the 10 critical items (WCT-CI-10) from the full WCT. A receiver operating characteristic curve (ROC) analysis was then conducted to determine the sensitivity and specificity of the WCT-CI-10 as well as identify an optimal cut score. The relationship between the WCT-CI-10 and other established PVTs was conducted by using bivariate correlations. WCT-CI-10 total scores were found to be significantly correlated with widely used and validated PVTs at the p < 0.001 level ranging from.48 to 0.70. ROC curve analysis for the WCT-CI-10 indicated optimal cutoff of ≥2 errors, with 68% sensitivity and 97% specificity (AUC = 0.907, p < 0.001) which is comparable to standard 50-item WCT which indicated an optimal cutoff of ≥8 errors with 70% sensitivity and 91% specificity (AUC = 0.890, p < 0.001). In this study we have identified 10 critical items of the WCT with adequate sensitivity and specificity for the detection of invalid performance. These results provide preliminary evidence for future research including these ten critical items as a theoretical basis for validating them as an abbreviated PVT measure.

Combining accuracy scores with time cutoffs improves the specificity of the Word Choice Test.

Combining accuracy scores with time cutoffs improves the specificity of the Word Choice Test.

Standalone Performance Validity Tests May Be Differentially Related to Measures of Working Memory, Processing Speed, and Verbal Memory in Patients With Multiple Sclerosis.

Cognitive functioning may account for minimal levels (i.e., 5%-14%) of variance of performance validity test (PVT) scores in clinical examinees. The present study extended this research twofold: (a) by determining the variance cognitive functioning explains within three distinct PVTs (b) in a sample of patients with multiple sclerosis (pwMS). Seventy-five pwMS (Mage = 48.50, 70.6% female, 80.9% White) completed the Victoria Symptom Validity Test (VSVT), Word Choice Test (WCT), Dot Counting Test (DCT), and three objective measures of working memory, processing speed, and verbal memory as part of clinical neuropsychological assessment. Regression analyses in credible groups (ns ranged from 54 to 63) indicated that cognitive functioning explained 24% to 38% of the variance in logarithmically transformed PVT variables. Variance from cognitive testing differed across PVTs: verbal memory significantly influenced both VSVT and WCT scores; working memory influenced VSVT and DCT scores; and processing speed influenced DCT scores. The WCT appeared least related to cognitive functioning of the included PVTs. Alternative plausible explanations, including the apparent domain/modality specificity hypothesis of PVTs versus the potential sensitivity of these PVTs to neurocognitive dysfunction in pwMS were discussed. Continued psychometric investigations into factors affecting performance validity, especially in multiple sclerosis, are warranted.

Validation and psychometric properties of the Word Choice Test-10 as an abbreviated performance validity test

Objective: The objective of the current investigation was to validate and establish the psychometric properties of an abbreviated, 10-item version of the Word Choice Test (WCT). Method: Data from one hundred ten clinically-referred participants (M age = 55.92, SD = 14.07; M education = 13.74, SD = 2.43; 84.5% Male) in a Veterans Affairs neuropsychology outpatient clinic was analyzed. All participants completed the WCT, the TOMM T1, the WMT, and the Digit Span subtest of the WAIS-IV as part of a larger battery of neuropsychological tests. Results: Correlation analyses revealed significant relationships between the 10-item WCT-10, the TOMM T1, the RDS forward/backward, as well as the IR, DR, and CNS subtests of the WMT. ROC analysis for the WCT-10 indicated optimal cutoff of 2 or more errors, with 52% sensitivity and 97% specificity (AUC=.786, p<.001), compared with the standard administration of the WCT with a cutoff of 8 or more errors, which had 67% sensitivity and 91% specificity. Specificity/sensitivity values remained adequate at a cutoff of two or more errors when participants with cognitive impairment (Sensitivity=.52, Specificity=.92) and without cognitive impairment (Sensitivity=.52, Specificity = 1.0) were examined separately. Conclusions: The present investigation revealed that the WCT-10, an abbreviated free-standing PVT comprised of the initial 10 items of the WCT, demonstrated clinical utility in a mixed clinical sample of Veterans and was robust to cognitive impairment. This abbreviated PVT may benefit researchers and clinicians through adequate identification of invalid performance while minimizing completion time.

Replicating a Meta-Analysis: The Search for the Optimal Word Choice Test Cutoff Continues.

This study was designed to expand on a recent meta-analysis that identified ≤42 as the optimal cutoff on the Word Choice Test (WCT). We examined the base rate of failure and the classification accuracy of various WCT cutoffs in four independent clinical samples (N = 252) against various psychometrically defined criterion groups. WCT ≤ 47 achieved acceptable combinations of specificity (.86-.89) at .49 to .54 sensitivity. Lowering the cutoff to ≤45 improved specificity (.91-.98) at a reasonable cost to sensitivity (.39-.50). Making the cutoff even more conservative (≤42) disproportionately sacrificed sensitivity (.30-.38) for specificity (.98-1.00), while still classifying 26.7% of patients with genuine and severe deficits as non-credible. Critical item (.23-.45 sensitivity at .89-1.00 specificity) and time-to-completion cutoffs (.48-.71 sensitivity at .87-.96 specificity) were effective alternative/complementary detection methods. Although WCT ≤ 45 produced the best overall classification accuracy, scores in the 43 to 47 range provide comparable objective psychometric evidence of non-credible responding. Results question the need for designating a single cutoff as "optimal," given the heterogeneity of signal detection environments in which individual assessors operate. As meta-analyses often fail to replicate, ongoing research is needed on the classification accuracy of various WCT cutoffs.

Comprehensive Analysis of MMPI-2-RF Symptom Validity Scales and Performance Validity Test Relationships in a Diverse Mixed Neuropsychiatric Setting.

The utility of symptom (SVT) and performance (PVT) validity tests has been independently established in neuropsychological evaluations, yet research on the relationship between these two types of validity indices is limited to circumscribed populations and measures. This study examined the relationship between SVTs on the Minnesota Multiphasic Personality Inventory-2-Restructured Form (MMPI-2-RF) and PVTs in a mixed neuropsychiatric setting. This cross-sectional study included data from 181 diagnostically and demographically diverse patients with neuropsychiatric conditions referred for outpatient clinical neuropsychological evaluation at an academic medical center. All patients were administered a uniform neuropsychological battery, including the MMPI-2-RF and five PVTs (i.e., Dot Counting Test; Medical Symptom Validity Test; Reliable Digit Span; Test of Memory Malingering-Trial 1; Word Choice Test). Nonsignificant associations emerged between SVT and PVT performance. Although the Response Bias Scale was most predictive of PVT performance, MMPI-2-RF SVTs generally had low classification accuracy for predicting PVT performance. Neuropsychological test performance was related to MMPI-2-RF SVT status only when overreporting elevations were at extreme scores. The current study further supports that SVTs and PVTs measure unique and dissociable constructs among diverse patients with neuropsychiatric conditions, consistent with literature from other clinical contexts. Therefore, objective evidence of symptom overreporting on MMPI-2-RF SVTs cannot be interpreted as definitively indicating invalid performance on tests of neurocognitive abilities. As such, clinicians should include both SVTs and PVTs as part of a comprehensive neuropsychological evaluation as they provide unique information regarding performance and symptom validity.

Critical item (CR) analysis expands the classification accuracy of performance validity tests based on the forced choice paradigm-Replicating previously introduced CR cutoffs within the Word Choice Test.

This study was designed to replicate previous research on critical item analysis within the Word Choice Test (WCT). Archival data were collected from a mixed clinical sample of 119 consecutively referred adults (Mage = 51.7, Meducation = 14.7). The classification accuracy of the WCT was calculated against psychometrically defined criterion groups. Critical item analysis identified an additional 2%-5% of the sample that passed traditional cutoffs as noncredible. Passing critical items after failing traditional cutoffs was associated with weaker independent evidence of invalid performance, alerting the assessor to the elevated risk for false positives. Failing critical items in addition to failing select traditional cutoffs increased overall specificity. Non-White patients were 2.5 to 3.5 times more likely to Fail traditional WCT cutoffs, but select critical item cutoffs limited the risk to 1.5-2. Results confirmed the clinical utility of critical item analysis. Although the improvement in sensitivity was modest, critical items were effective at containing false positive errors in general, and especially in racially diverse patients. Critical item analysis appears to be a cost-effective and equitable method to improve an instrument's classification accuracy. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

A-230 A Story Told in Four Parts: Embedded Validity Indicators’ Effectiveness in Predicting Word Choice Test Validity Classification

Abstract Objective: Many individual embedded performance validity tests (PVTs) have been independently cross-validated as validity indices among various clinical populations; however, research examining the additional predictive value from combining multiple embedded PVTs is more limited. The study's purpose was to examine the accuracy of eight embedded PVTs for predicting pass/failure performance on a well-validated freestanding PVT, the Advanced Clinical Solutions: Word Choice Test (WCT). Method: The study consisted of 225 adults referred for outpatient neuropsychological evaluation that included the following embedded PVTs: Rey Auditory Verbal Learning Test (RAVLT) Forced Choice (FC), RAVLT Effort Score, Brief Visual Memory Test-Revised Recognition Discrimination (BVMT-RD), Digit Span Age Corrected Scaled Score, Reliable Digit Span, Stroop Color and Word Test Word Reading T-Score, Letter Fluency T-Score, and Trail Making Test Part A T-Score. The sample was 56% female and racially diverse (40% Black, 35% White, 17% Hispanic, 5% Asian, and 2% other ethnicities) with a mean age of 45.96 (SD=16.12) and mean education of 13.96 (SD=2.73) years. Results: A forward stepwise logistic regression was performed to predict WCT pass/fail using the embedded PVTs as predictors. The model was achieved in three steps (p95% accuracy in correctly classifying WCT pass group. Conclusion: RAVLT-FC, BVMT-RD, and TMT-A combined reliably predict WCT pass/fail group. Verbal and attention based embedded PVTs were found to have the most robust predictive capacity of WCT validity status.

A-237 Validation and Psychometric Properties of the Word Choice Test-10, An Abbreviated Performance Validity Test

Abstract Objective: The purpose of the study was to validate an abbreviated 10-item version of the Word Choice Test (WCT). Method: One hundred and ten clinically-referred participants (M age=55.92, SD=14.07; M education=13.74, SD=2.43; 84.5% Male; 47.3% White, 32.7% Hispanic, 15.5% African American, 2.7% Multiracial, 0.9% American Indian, 0.9% Other) completed the WCT as part of a larger battery of neuropsychological tests in a Veterans Affairs (VA) outpatient clinic. The sample included individuals with no cognitive diagnosis (n=37), individuals with a cognitive diagnosis (n=43), and individuals that were determined to have invalid performance across multiple criterion PVT measures (n=30). Results: Total scores on the 10-item abbreviated Word Choice Test (WCT-10) were significantly correlated with total scores on the 50-item WCT standard form (r(108) =.88, p &amp;lt; .01), the Test of Memory Malingering (TOMM) Trial 1 (r(108) =.60, p &amp;lt; .01), and the Reliable Digit Span (RDS) forward/backward (r(108) =.34, p &amp;lt; .01). Receiver operating characteristic curve (ROC) analysis for the WCT-10 indicated optimal cutoff of ≥ 2 errors, with 53% sensitivity and 96% specificity (AUC=.825, p &amp;lt; .001), while ROC analysis for the standard 50-item WCT indicated an optimal cutoff of ≥ 8 errors with 70% sensitivity and 91% specificity (AUC=.890, p &amp;lt; .001). Conclusions: Results provide evidence for the clinical utility of the WCT-10 as a brief, freestanding performance validity test in a mixed clinical sample of Veterans with and without cognitive impairment. Utilizing the WCT-10 may reduce overall neuropsychological evaluation time, while providing evidence for validity of cognitive test performance.

Preliminary evidence for digit span performance validity indicators within the neuropsychological assessment battery

The purpose of this study was to evaluate multiple embedded performance validity indicators within the Digits Forward and Digits Backward subtests of the Neuropsychological Assessment Battery (NAB), including Reliable Digit Span (RDS), as no published papers have examined embedded digit span validity indicators within these subtests of the NAB. Retrospective archival chart review was conducted at an outpatient neuropsychology clinic. Participants were 92 adults (ages 19−68) who completed NAB Digits Forward and Digits Backward, and the Word Choice Test (WCT). Receiver operating characteristic (ROC) curves, t-tests, and sensitivity and specificity analyses were conducted. Analyses showed that RDS demonstrated acceptable classification accuracy between those who passed the WCT and those who did not. The area under the curve (AUC) value for RDS was 0.702; however, AUC values for all other digit span indices were unacceptably low. The optimal cutoff for RDS was identified (<8). RDS for the NAB appears to be an adequate indicator of performance validity; however, considering the very small number of participants who were invalid on the WCT (n = 15), as well as the utilization of only one stand-alone PVT to classify validity status, these findings are preliminary and in need of replication.

Supplemental Material for Critical Item (CR) Analysis Expands the Classification Accuracy of Performance Validity Tests Based on the Forced Choice Paradigm—Replicating Previously Introduced CR Cutoffs Within the Word Choice Test

Supplemental Material for Critical Item (CR) Analysis Expands the Classification Accuracy of Performance Validity Tests Based on the Forced Choice Paradigm—Replicating Previously Introduced CR Cutoffs Within the Word Choice Test

Psychometric implications of failure on one performance validity test: a cross-validation study to inform criterion group definition

ABSTRACT Introduction: Research to date has supported the use of multiple performance validity tests (PVTs) for determining validity status in clinical settings. However, the implications of including versus excluding patients failing one PVT remains a source of debate, and methodological guidelines for PVT research are lacking. This study evaluated three validity classification approaches (i.e. 0 vs. ≥2, 0–1 vs. ≥2, and 0 vs. ≥1 PVT failures) using three reference standards (i.e. criterion PVT groupings) to recommend approaches best suited to establishing validity groups in PVT research methodology. Method: A mixed clinical sample of 157 patients was administered freestanding (Medical Symptom Validity Test, Dot Counting Test, Test of Memory Malingering, Word Choice Test), and embedded PVTs (Reliable Digit Span, RAVLT Effort Score, Stroop Word Reading, BVMT-R Recognition Discrimination) during outpatient neuropsychological evaluation. Three reference standards (i.e. two freestanding and three embedded PVTs from the above list) were created. Rey 15-Item Test and RAVLT Forced Choice were used solely as outcome measures in addition to two freestanding PVTs not employed in the reference standard. Receiver operating characteristic curve analyses evaluated classification accuracy using the three validity classification approaches for each reference standard. Results: When patients failing only one PVT were excluded or classified as valid, classification accuracy ranged from acceptable to excellent. However, classification accuracy was poor to acceptable when patients failing one PVT were classified as invalid. Sensitivity/specificity across two of the validity classification approaches (0 vs. ≥2; 0–1 vs. ≥2) remained reasonably stable. Conclusions: These results reflect that both inclusion and exclusion of patients failing one PVT are acceptable approaches to PVT research methodology and the choice of method likely depends on the study rationale. However, including such patients in the invalid group yields unacceptably poor classification accuracy across a number of psychometrically robust outcome measures and therefore is not recommended.

The Development and Validation of the Embedded Validity Indicator for the Neuropsychological Assessment Battery.

The purpose of this study was to develop and validate an embedded measure of performance validity within the Neuropsychological Assessment Battery (NAB). This study involved a retrospective chart review at an outpatient neuropsychology clinic. Participants were 183 adults (ages 18-70) who completed the attention and memory modules of the NAB, as well as the Word Choice Test, Green's Medical Symptom Validity Test (MSVT), and Green's Non-Verbal MSVT, as part of a clinical neuropsychological assessment (n=147) or as part of a forensic neuropsychological evaluation (n=36). Replicating methodology utilized by Silverberg etal. (2007) for the development of the Effort Index within the Repeatable Battery for the Assessment of Neuropsychological Status, an Embedded Validity Indictor (EVI) for the NAB was developed in the present study based on Digits Forward and List Learning Long Delayed Forced-Choice Recognition (list recognition) subtests. Receiver operating characteristic curve analyses indicated the newly developed NAB EVI was able to significantly differentiate between valid and invalid status on stand-alone performance-validity tests, with area under the curve values ranging from 0.797 to 0.977. Optimal cutoffs for medical, forensic, and mixed samples were identified. The newly developed NAB EVI shows promise as an embedded performance validity measure; however, due to moderate sensitivity, it should be used in combination with stand-alone performance validity tests to detect invalid performance.

A Systematic Review and Meta-Analysis of the Diagnostic Accuracy of the Advanced Clinical Solutions Word Choice Test as a Performance Validity Test.

Thorough assessment of performance validity has become an established standard of practice in neuropsychological assessment. While there has been a large focus on the development and cross-validation of embedded performance validity tests (PVTs) in recent years, new freestanding PVTs have also been developed, including the Word Choice Test (WCT) as part of the Advanced Clinical Solutions Effort System. And, while the WCT's general utility for identifying invalid performance has been demonstrated in the ensuing decade since its initial publication, optimal cut-scores and associated psychometric properties have varied widely across studies. This study sought to synthesize the existing diagnostic accuracy literature regarding the WCT via a systematic review and to conduct a meta-analysis to determine the performance validity cut-score that best maximizes sensitivity while maintaining acceptable specificity. A systematic search of the literature resulted in 14 studies for synthesis, with eight of those available for meta-analysis. Meta-analytic results revealed an optimal cut-score of ≤ 42 with 54% sensitivity and 93% specificity for identifying invalid neuropsychological test performance. Collectively, the WCT demonstrated adequate diagnostic accuracy as a PVT across a variety of populations. Recommendations for future studies are also provided.

A-211 Examination of the Personality Assessment Inventory and Performance Validity Tests: Correlation between Response Styles, Performance Effort, and Somatic Complaint

Abstract Objective Correlation between the performance validity measures from the Advanced Clinical Solution (ACS) package and select scales from the Personality Assessment Inventory (PAI) were examined to investigate relationships between performance effort and response styles involving negative impression (NIM), inconsistency (ICN), infrequency (INF), and somatic compliant (SOM) scales. Theoretical differences between symptom validity based on self-report questionnaires and credibility of performance on a performance validity tests (PVTs) were considered. Method Archival data from clinical neuropsychological evaluations between the years 2015–2018 were reviewed, and 120 consecutive adult cases involving 68 males and 52 females between the ages of 19–69 were collected. Examined measures included: ACS—Word Choice Test (WCT), Reliable Digit Span, Visual Reproduction II Recognition, Logical Memory II Recognition, and the PAI. Spearman’s rank-order correlation and point-biserial correlation analyses were conducted to examine the relationship between variables. Results WCT had significant negative correlations with the NIM scale (rs(118) = −.203, p = .013). Modest correlations were found between SOM and WCT when examining PVT raw score (rs (118) = −.192, p = .018) and base rate performance (rs(118) = −.222, p = .007). Point-biserial serial analysis found a weak negative correlation between performance credibility and the SOM scale, which was statistically significant (rpb = −.221, n = 120, p = .008). Conclusions NIM T-scores appear to be correlated with lower performance on the WCT, suggesting that an exaggerated or distorted impression of the self is associated with higher risk of poorer performance the stand-alone PVT. Correlation between embedded PVTs and PAI scales were inconsistent.

Feigning or forgetfulness: The effect of memory impairment severity on word choice test performance

Objective This study cross-validated the word choice test (WCT) in a diverse neuropsychiatric sample and examined the effect of increasing verbal memory impairment severity on WCT performance. Method Data from 147 clinically referred patients (113 valid/34 invalid) who completed the WCT, Rey Auditory Verbal Learning Test (RAVLT), and four independent criterion PVTs were analyzed. RAVLT memory impairment bands used were: ≥37T (normal memory); 30T–36T (below average scores/mild impairment); and ≤29T (extremely low scores/severe impairment). Results WCT and RAVLT were moderately correlated. The invalid group had significantly worse performance on the WCT and RAVLT. For the overall sample, the WCT yielded an area under the curve (AUC) = .79, with 62% sensitivity/93% specificity at a cut-score of ≤41. When the sample was subdivided by memory impairment severity, the severe impairment group had significantly lower WCT scores than the normal group. Moreover, the WCT retained moderate classification accuracy among the normal memory (AUC = .85) and mild memory impairment (AUC = .76) groups, with sensitivities of 65% and 62% (≥91% specificity) at their respective optimal cut-scores of ≤44 and ≤42. In contrast, the WCT had low classification accuracy among those with severe memory impairment (AUC = .66), with only 15% sensitivity/95% specificity at the optimal cut-score of ≤30. Conclusion The WCT is generally useful for detecting invalid neuropsychological test performance, although, its classification accuracy was diminished among patients with severe memory impairment. Therefore, while the WCT remains a viable option for performance validity assessment, neuropsychologists should carefully consider its use when this level of severe memory impairment is known or suspected.

The BNT-15 provides an accurate measure of English proficiency in cognitively intact bilinguals – a study in cross-cultural assessment

This study was designed to replicate earlier reports of the utility of the Boston Naming Test – Short Form (BNT-15) as an index of limited English proficiency (LEP). Twenty-eight English-Arabic bilingual student volunteers were administered the BNT-15 as part of a brief battery of cognitive tests. The majority (23) were women, and half had LEP. Mean age was 21.1 years. The BNT-15 was an excellent psychometric marker of LEP status (area under the curve: .990–.995). Participants with LEP underperformed on several cognitive measures (verbal comprehension, visuomotor processing speed, single word reading, and performance validity tests). Although no participant with LEP failed the accuracy cutoff on the Word Choice Test, 35.7% of them failed the time cutoff. Overall, LEP was associated with an increased risk of failing performance validity tests. Previously published BNT-15 validity cutoffs had unacceptably low specificity (.33–.52) among participants with LEP. The BNT-15 has the potential to serve as a quick and effective objective measure of LEP. Students with LEP may need academic accommodations to compensate for slower test completion time. Likewise, LEP status should be considered for exemption from failing performance validity tests to protect against false positive errors.