Word Choice Test Research Articles

Evaluation of the classification accuracy of multiple performance validity tests in a mixed clinical sample

The Test of Memory Malingering (TOMM) and Word Memory Test (WMT) are among the most well-known performance validity tests (PVTs) and regarded as gold standard measures. Due to the many factors that impact PVT selection, it is imperative that clinicians make informed clinical decisions with respect to additional or alternative PVTs that demonstrate similar classification accuracy as these well-validated measures. The present archival study evaluated the agreement/classification accuracy of a large battery consisting of multiple other freestanding/embedded PVTs in a mixed clinical sample of 126 veterans. We examined failure rates for all standalone/embedded PVTs using established cut-scores and calculated pass/fail agreement rates and diagnostic odds ratios for various combinations of PVTs using the TOMM and WMT as criterion measures. TOMM and WMT demonstrated the best agreement, followed by Word Choice Test (WCT). The Rey Fifteen Item Test had an excessive number of false-negative errors and reduced classification accuracy. The Digit Span age-corrected scaled score (DS-ACSS) had highest agreement. Findings lend further support to the use of a combination of embedded and standalone PVTs in identifying suboptimal performance. Results provide data to enhance clinical decision making for neuropsychologists who implement combinations of PVTs in a larger clinical battery.

Impact of criterion measures on the classification accuracy of TOMM-1

This study was designed to examine the effect of various criterion measures on the classification accuracy of Trial 1 of the Test of Memory Malingering (TOMM-1), a free-standing performance validity test (PVT). Archival data were collected from a case sequence of 91 (M Age = 42.2 years; M Education = 12.7) patients clinically referred for neuropsychological assessment. Trials 2 and Retention of the TOMM, the Word Choice Test, and three validity composites were used as criterion PVTs. Classification accuracy varied systematically as a function of criterion PVT. TOMM-1 ≤ 43 emerged as the optimal cutoff, resulting in a wide range of sensitivity (.47–1.00), with perfect overall specificity. Failing the TOMM-1 was unrelated to age, education or gender, but was associated with elevated self-reported depression. Results support the utility of TOMM-1 as an independent, free-standing, single-trial PVT. Consistent with previous reports, the choice of criterion measure influences parameter estimates of the PVT being calibrated. The methodological implications of modality specificity to PVT research and clinical/forensic practice should be considered when evaluating cutoffs or interpreting scores in the failing range.

Geographic Variation and Instrumentation Artifacts: in Search of Confounds in Performance Validity Assessment in Adults with Mild TBI

Regional fluctuations in cognitive ability have been reported worldwide. Given perennial concerns that the outcome of performance validity tests (PVTs) may be contaminated by genuine neuropsychological deficits, geographic differences may represent a confounding factor in determining the credibility of a given neurocognitive profile. This pilot study was designed to investigate whether geographic location affects base rates of failure (BRFail) on PVTs. BRFail were compared across a number of free-standing and embedded PVTs in patients with mild traumatic brain injury (mTBI) from two regions of the US (Midwest and New England). Retrospective archival data were collected from clinically referred patients with mTBI at two different academic medical centers (nMidwest = 76 and nNew England = 84). One free-standing PVT (Word Choice Test) and seven embedded PVTs were administered to both samples. The embedded validity indicators were combined into a single composite score using two different previously established aggregation methods. The New England sample obtained a higher score on the Verbal Comprehension Index of the WAIS-IV (d = .34, small-medium). The difference between the two regions in Full Scale IQ (FSIQ) was small (d = .28). When compared with the omnibus population mean (100), the effect of mTBI on FSIQ was small (d = .22) in the New England sample and medium (d = .53) in the Midwestern one. However, contrasts using estimates of regional FSIQ produced equivalent effect sizes (d: .47–.53). BRFail was similar on free-standing PVTs, but varied at random for embedded PVTs. Aggregating individual indices into a validity composite effectively neutralized regional variability in BRFail. Classification accuracy varied as a function of both geographic region and instruments. Despite small overall effect sizes, regional differences in cognitive ability may potentially influence clinical decision making, both in terms of diagnosis and performance validity assessment. There was an interaction between geographic region and instruments in terms of the internal consistency of PVT profiles. If replicated, the findings of this preliminary study have potentially important clinical, forensic, methodological, and epidemiological implications.

Cross-validation of three Advanced Clinical Solutions performance validity tests: Examining combinations of measures to maximize classification of invalid performance

Use of multiple performance validity tests (PVTs) may best identify invalid performance, though few studies have examined the utility and accuracy of combining PVTs. This study examined the following PVTs in the Advanced Clinical Solutions (ACS) package to determine their utility alone and in concert: Word Choice Test (WCT), Reliable Digit Span (RDS), and Logical Memory Recognition (LMR). Ninety-three veterans participated in clinical neuropsychological evaluations to determine presence of cognitive impairment; 25% of the performances were deemed invalid via criterion PVTs. Classification accuracy of the ACS measures was assessed via receiver operating characteristic curves, while logistic regressions determined utility of combining these PVTs. The WCT demonstrated superior classification accuracy compared to the two embedded measures of the ACS, even in veterans with cognitive impairment. The two embedded measures (even when used in concert) exhibited inadequate classification accuracy. A combined model with all three ACS PVTs similarly demonstrated little benefit of the embedded indicators over the WCT alone. Results suggest the ACS WCT has utility for detecting invalid performance in a clinical sample with likely cognitive impairment, though the embedded ACS measures (RDS and LMR) may have limited incremental utility, particularly in individuals with cognitive impairment.

Early bird fails the PVT? The effects of timing artifacts on performance validity tests.

This study was designed to investigate the effects of timing on the likelihood of failing the Recognition Memory Test-Words (RMT) and Word Choice Test (WCT). The RMT and WCT were administered in counterbalanced order either at the beginning (Time 1) or at the end (Time 2) of a test battery to a mixed clinical sample of 196 patients (Mage = 44.5 years, 55.1% female) medically referred for neuropsychological evaluation. The risk of failing the accuracy score was higher at Time 1 on both the RMT (relative risk [RR]: 1.44-1.64) and the WCT (RR: 1.21-1.50) across a range of cutoffs. Likewise, the risk of failing the time-to-completion score was higher at Time 1 on both the RMT (RR: 1.30-1.94) and the WCT (RR: 1.58-3.75). Established cutoffs failed to reach specificity standards at Time 1; more liberal cutoffs cleared specificity thresholds at Time 2. According to our findings, the RMT and WCT may be prone to false-positive errors at Time 1. Conversely, when administered at Time 2, existing cutoffs may have lower sensitivity, but they are highly specific to invalid performance. Timing should be considered during both test selection and the interpretation of RMT and WCT scores. Using conservative cutoffs for morning administrations and liberal cutoffs for afternoon administrations may be necessary to neutralize timing artifacts. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Convergent, Discriminant, and Concurrent Validity of Nonmemory-Based Performance Validity Tests.

To supplement memory-based Performance Validity Tests (PVTs) in identifying noncredible performance, we examined the validity of the two most commonly used nonmemory-based PVTs-Dot Counting Test (DCT) and Wechsler Adult Intelligence Scale-Fourth edition (WAIS-IV) Reliable Digit Span (RDS)-as well as two alternative WAIS-IV Digit Span (DS) subtest PVTs. Examinees completed DCT, WAIS-IV DS, and the following criterion PVTs: Test of Memory Malingering, Word Memory Test, and Word Choice Test. Validity groups were determined by passing 3 (valid; n = 69) or failing ⩾2 (noncredible; n = 30) criterion PVTs. DCT, RDS, RDS-Revised (RDS-R), and WAIS-IV DS Age-Corrected Scaled Score (ACSS) were significantly correlated (but uncorrelated with memory-based PVTs). Combining RDS, RDS-R, and ACSS with DCT improved classification accuracy (particularly for DCT/ACSS) for detecting noncredible performance among valid-unimpaired, but largely not valid-impaired examinees. Combining DCT with ACSS may uniquely assess and best supplement memory-based PVTs to identify noncredible neuropsychological test performance in cognitively unimpaired examinees.

Research methods in performance validity testing studies: Criterion grouping approach impacts study outcomes

Objective: Performance validity test (PVT) research studies commonly utilize a known-groups design, but the criterion grouping approaches within the design vary greatly from one study to another. At the present time, it is unclear as to what degree different criterion grouping approaches might impact PVT classification accuracy statistics. Method: To analyze this, the authors used three different criterion grouping approaches to examine how classification accuracy statistics of a PVT (Word Choice Test; WCT) would differ. The three criterion grouping approaches included: (1) failure of 2+ PVTs versus failure of 0 PVTs, (2) failure of 2+ PVTs versus failure of 0–1 PVT, and (3) failure of a stand-alone PVT versus passing of a stand-alone PVT (Test of Memory Malingering). Results: When setting specificity at ≥.90, WCT cutoff scores ranged from 41 to 44 and associated sensitivity values ranged from .64 to .88, depending on the criterion grouping approach that was utilized. Conclusions: When using a stand-alone PVT to define criterion group status, classification accuracy rates of the WCT were higher than expected, likely due to strong correlations between the reference PVT and the WCT. This held true even when considering evidence that this grouping approach results in higher rates of criterion group misclassification. Conversely, when using criterion grouping approaches that utilized failure of 2+ PVTs, accuracy rates were more consistent with expectations. These findings demonstrate that criterion grouping approaches can impact PVT classification accuracy rates and resultant cutoff scores. Strengths, weaknesses, and practical implications of each of the criterion grouping approaches are discussed.

Utility of various WAIS-IV Digit Span indices for identifying noncredible performance validity among cognitively impaired and unimpaired examinees

Objective: Wechsler Adult Intelligence Scale-Fourth Edition (WAIS-IV) Reliable Digit Span (RDS), RDS-revised (RDS-R), and age-corrected scaled score (ACSS) are validated and commonly used embedded performance validity tests (PVTs), though existing validation studies have largely examined younger (approximate ages 19–35) patients with mild traumatic brain injury or those without cognitive impairment. This study compared the classification accuracy of RDS, RDS-R, and ACSS in a mixed clinical sample of relatively older (M age = 54.61) veterans with and without neurocognitive impairment. Method: During a comprehensive neuropsychological evaluation, 113 clinically-referred veterans completed the WAIS-IV Digit Span subtest and the following criterion PVTs: Dot Counting Test, Word Choice Test, and Test of Memory Malingering. Those with ≤1 criterion PVT failure were classified as valid (n = 87), whereas those with ≥2 failures were classified as noncredible (n = 26). Among valid participants, 49% were cognitively impaired. Results: RDS, RDS-R, and ACSS all significantly predicted validity group membership with respective areas under the curve (AUCs) of .79, .81, and .85, and optimal cut scores of RDS ≤ 5, RDS-R ≤ 9, and ACSS ≤ 5. Lower accuracy and AUCs were observed for the valid-cognitively impaired subsample across indices, but to a greater degree for traditional RDS. ACSS evidenced maximal sensitivity/specificity for the total sample (≤5; .62/.87), cognitively unimpaired subsample (≤5; .62/.95), and cognitively impaired subsample (≤4; .39/.86). Conclusions: ACSS yielded better classification accuracy and sensitivity/specificity than RDS and RDS-R. While all three indices have utility as embedded PVTs, ACSS ≤ 5 may be most robust to cognitive impairment while identifying noncredible performance.

Validation of the Advanced Clinical Solutions Word Choice Test (WCT) in a Mixed Clinical Sample: Establishing Classification Accuracy, Sensitivity/Specificity, and Cutoff Scores.

This study examined the Word Choice Test's (WCT) utility as a performance validity test in a mixed clinical sample of veterans referred for neuropsychological evaluation. Participants completed Green's Word Memory Test (WMT), WCT, and Test of Memory Malingering (TOMM) Trial 1. Using the WMT as the criterion for valid performance, logistic regressions examined the WCT and TOMM's classification accuracy for those with and without cognitive impairment (CI). Receiver operating characteristic curves were used to establish cut scores which maximized the sensitivity/specificity of each measure. In those without CI, both tests showed good classification accuracy (86.7% and 85.0%, respectively). Among those with CI, the TOMM retained good classification accuracy (82.3%), while the WCT's decreased considerably (69.4%). Optimal WCT cut scores differed based on impairment status, with a higher sensitivity/specificity trade-off among those with CI. Successful performance on the WCT appeared to rely more heavily on cognitive processes unrelated to performance validity.

Utility of critical items within the Recognition Memory Test and Word Choice Test

ABSTRACTThis study was designed to examine the clinical utility of critical items within the Recognition Memory Test (RMT) and the Word Choice Test (WCT). Archival data were collected from a mixed clinical sample of 202 patients clinically referred for neuropsychological testing (54.5% male; mean age = 45.3 years; mean level of education = 13.9 years). The credibility of a given response set was psychometrically defined using three separate composite measures, each of which was based on multiple independent performance validity indicators. Critical items improved the classification accuracy of both tests. They increased sensitivity by correctly identifying an additional 2–17% of the invalid response sets that passed the traditional cutoffs based on total score. They also increased specificity by providing additional evidence of noncredible performance in response sets that failed the total score cutoff. The combination of failing the traditional cutoff, but passing critical items was associated with increased risk of misclassifying the response set as invalid. Critical item analysis enhances the diagnostic power of both the RMT and WCT. Given that critical items require no additional test material or administration time, but help reduce both false positive and false negative errors, they represent a versatile, valuable, and time- and cost-effective supplement to performance validity assessment.

The power of timing: Adding a time-to-completion cutoff to the Word Choice Test and Recognition Memory Test improves classification accuracy

ABSTRACTIntroduction: The Recognition Memory Test (RMT) and Word Choice Test (WCT) are structurally similar, but psychometrically different. Previous research demonstrated that adding a time-to-completion cutoff improved the classification accuracy of the RMT. However, the contribution of WCT time-cutoffs to improve the detection of invalid responding has not been investigated. The present study was designed to evaluate the classification accuracy of time-to-completion on the WCT compared to the accuracy score and the RMT. Method: Both tests were administered to 202 adults (Mage = 45.3 years, SD = 16.8; 54.5% female) clinically referred for neuropsychological assessment in counterbalanced order as part of a larger battery of cognitive tests. Results: Participants obtained lower and more variable scores on the RMT (M = 44.1, SD = 7.6) than on the WCT (M = 46.9, SD = 5.7). Similarly, they took longer to complete the recognition trial on the RMT (M = 157.2 s,SD = 71.8) than the WCT (M = 137.2 s, SD = 75.7). The optimal cutoff on the RMT (≤43) produced .60 sensitivity at .87 specificity. The optimal cutoff on the WCT (≤47) produced .57 sensitivity at .87 specificity. Time-cutoffs produced comparable classification accuracies for both RMT (≥192 s; .48 sensitivity at .88 specificity) and WCT (≥171 s; .49 sensitivity at .91 specificity). They also identified an additional 6–10% of the invalid profiles missed by accuracy score cutoffs, while maintaining good specificity (.93–.95). Functional equivalence was reached at accuracy scores ≤43 (RMT) and ≤47 (WCT) or time-to-completion ≥192 s (RMT) and ≥171 s (WCT). Conclusions: Time-to-completion cutoffs are valuable additions to both tests. They can function as independent validity indicators or enhance the sensitivity of accuracy scores without requiring additional measures or extending standard administration time.

A Comparison of the Degree of Effort Involved in the TOMM and the ACS Word Choice Test Using a Dual-Task Paradigm

The aims of the current study were to: (a) examine the predictive validity and efficacy of the Advanced Clinical Solutions Word Choice Test (WCT) as a measure of effort relative to the Test of Memory Malingering (TOMM); (b) investigate whether performing a dual (distraction) task would undermine performance on either test; (c) assess the effect of coaching on the diagnostic accuracy of both the WCT and the TOMM; and (d) establish an optimal cut score for the WCT. The current study used a simulation design based on an analogue design in which normal participants were instructed to either apply full effort or simulate a brain injury on the tasks without being detected. Participants included 93 undergraduate university students who were randomly assigned to 1 of 4 conditions: (a) distraction, (b) uncoached traumatic brain injury (TBI) simulators, (c) coached TBI simulators, or (d) full effort. The results demonstrated that the WCT and the TOMM were effective in detecting simulated cognitive impairment. Both tests were resistant to the effects of distraction and were equally effective in detecting coached and uncoached simulators. A cut score of 42 on the WCT was found to provide optimal specificity and sensitivity on the test.

Comparisons of Five Performance Validity Indices in Bona Fide and Simulated Traumatic Brain Injury

A number of performance validity tests (PVTs) are used to assess memory complaints associated with traumatic brain injury (TBI); however, few studies examine the concordance and predictive accuracy of multiple PVTs, specifically in the context of combined models in known-group designs. The present study compared five widely used PVTs: the Test of Memory Malingering (TOMM), Medical Symptom Validity Test (MSVT), Reliable Digit Span (RDS), Word Choice Test (WCT), and California Verbal Learning Test – Forced Choice (CVLT-FC). Participants were 51 adults with bona fide moderate to severe TBI and 58 demographically comparable healthy adults coached to simulate memory impairment. Classification accuracy of individual PVTs was evaluated using logistic regression and receiver operating characteristic (ROC) curves, examining both the dichotomous cutting scores as recommended by the test publishers and continuous scores for the measures. Results demonstrated nearly equivalent discrimination ability of the TOMM, MSVT, and CVLT-FC as individual predictors, all of which markedly outperformed the WCT and RDS. Models of combined PVTs were examined using Bayesian information criterion statistics, with results demonstrating that diagnostic accuracy showed only small to modest growth when the number of tests was increased beyond two. Considering the clinical and pragmatic issues in deriving a parsimonious assessment battery, these findings suggest that using the TOMM and CVLT in conjunction or the MSVT and CVLT in conjunction maximized predictive accuracy as compared to a single index or an assortment of these widely used measures.

Comparing the Recognition Memory Test and the Word Choice Test in a Mixed Clinical Sample: Are They Equivalent?

The Recognition Memory Test (RMT) was compared to the Word Choice Test (WCT) within the same fixed neuropsychological battery administered to a mixed clinical sample of 237 adults to empirically evaluate the psychometric equivalence of these two instruments. On average, there was a 3-point difference in raw scores between the two instruments (M RMT = 44.3, SD RMT = 6.8; M WCT = 47.1, SD WCT = 4.6; p < .001, d = .48). The probability density functions differ substantially at the two ends of the scale but are similarly ≤42. Cross-validation analyses suggest that the RMT cutoff of ≤42 is functionally equivalent to a WCT score of ≤45.

Detection of Insufficient Effort Using the Advanced Clinical Solutions for the Wechsler Memory Scale, Fourth Edition

This study investigated the ability of the Wechsler Memory Scale-4th Edition (WMS-IV) and the Advanced Clinical Solutions (ACS) package including the new Word Choice test (WCT) to distinguish poor performance due to intentional response bias among simulators of traumatic brain injury (TBI) from poor performance due to actual TBI. Participants were 45 survivors of moderate to severe TBI and 39 healthy adult coached to simulate TBI. Logistic regression indicated that a five-variable model containing all the ACS variables and a single-variable model using only the WCT were statistically reliable. Comparing predictive accuracy of each model found that adding the WCT to the ACS increased predictive accuracy. Diagnostic efficiency for the full ACS model was considered “excellent” according to interpretive guidelines.

Relation between deaf children's phonological skills in kindergarten and word recognition performance in first grade

The aim of the present study was twofold: 1) to determine whether phonological skills measured in deaf prereaders predict their later phonological and reading skills after one year of reading instruction as is the case for hearing children; 2) to examine whether the age of exposure to a fully specified phonological input such as Cued Speech may explain the inter-individual differences observed in deaf children's phonological and word recognition levels. Twenty-one 6-year-old deaf prereaders and 21 hearing children of the same chronological age performed two phonological tasks (rhyme decision and generation tasks); they were re-assessed 12 months later and presented with other phonological tasks (rhyme decision and common unit identification tasks) and a written word choice test. Phonological skills measured before learning to read predicted the written word recognition score the following year, both for hearing and for deaf participants. Age of onset of exposure to Cued Speech was also a strong predictor of phonological and written word recognition scores in beginning deaf readers. The evidence broadly supports the idea of a capacity for acquiring phonological skills in deaf children. Deaf children who are able to develop an implicitly structured phonological knowledge before learning to read will be better readers when this knowledge becomes explicit under the pressure of reading instruction.

A 77-Kilobase Region of Chromosome 6p22.2 Is Associated with Dyslexia in Families From the United Kingdom and From the United States

Several quantitative trait loci (QTLs) that influence developmental dyslexia (reading disability [RD]) have been mapped to chromosome regions by linkage analysis. The most consistently replicated area of linkage is on chromosome 6p23-21.3. We used association analysis in 223 siblings from the United Kingdom to identify an underlying QTL on 6p22.2. Our association study implicates a 77-kb region spanning the gene TTRAP and the first four exons of the neighboring uncharacterized gene KIAA0319. The region of association is also directly upstream of a third gene, THEM2. We found evidence of these associations in a second sample of siblings from the United Kingdom, as well as in an independent sample of twin-based sibships from Colorado. One main RD risk haplotype that has a frequency of approximately 12% was found in both the U.K. and U.S. samples. The haplotype is not distinguished by any protein-coding polymorphisms, and, therefore, the functional variation may relate to gene expression. The QTL influences a broad range of reading-related cognitive abilities but has no significant impact on general cognitive performance in these samples. In addition, the QTL effect may be largely limited to the severe range of reading disability.