Performance Validity Indicators Research Articles

Use of perceptual memory as a performance validity indicator: initial validation with simulated mild traumatic brain injury

ABSTRACT Introduction Many commonly employed performance validity tests (PVTs) are several decades old and vulnerable to compromise, leading to a need for novel instruments. Because implicit/non-declarative memory may be robust to brain damage, tasks that rely upon such memory may serve as an effective PVT. Using a simulation design, this experiment evaluated whether novel tasks that rely upon perceptual memory hold promise as PVTs. Method Sixty healthy participants were provided instructions to simulate symptoms of mild traumatic brain injury (TBI), and they were compared to a group of 20 honest responding individuals. Simulator groups received varying levels of information concerning TBI symptoms, resulting in naïve, sophisticated, and test-coached groups. The Word Memory Test, Test of Memory Malingering, and California Verbal Learning Test-II Forced Choice Recognition Test were administered. To assess perceptual memory, selected images from the Gollin Incomplete Figures and Mooney Closure Test were presented as visual perception tasks. After brief delays, memory for the images was assessed. Results No group differences emerged on the perception trials of the Gollin and Mooney figures, but simulators remembered fewer images than the honest responders. Simulator groups differed on the standard PVTs, but they performed equivalently on the Gollin and Mooney figures, implying robustness to coaching. Relying upon a criterion of 90% specificity, the Gollin and Mooney figures achieved at least 90% sensitivity, comparing favorably to the standard PVTs. Conclusions The Gollin and Mooney figures hold promise as novel PVTs. As perceptual memory tests, they may be relatively robust to brain damage, but future research involving clinical samples is necessary to substantiate this assertion.

Comparing embedded performance validity indicators within the WAIS-IV Letter-Number sequencing subtest to Reliable Digit Span among adults referred for evaluation of attention-deficit/hyperactivity disorder

Objectives: This study investigated the Wechsler Adult Intelligence Scale-Fourth Edition Letter-Number Sequencing (LNS) subtest as an embedded performance validity indicator among adults undergoing an attention-deficit/hyperactivity disorder (ADHD) evaluation, and its potential incremental value over Reliable Digit Span (RDS). Method: This cross-sectional study comprised 543 adults who underwent neuropsychological evaluation for ADHD. Patients were divided into valid (n = 480) and invalid (n = 63) groups based on multiple criterion performance validity tests. Results: LNS total raw scores, age-corrected scaled scores, and age- and education-corrected T-scores demonstrated excellent classification accuracy (area under the curve of .84, .83, and .82, respectively). The optimal cutoff for LNS raw score (≤16), age-corrected scaled score (≤7), and age- and education-corrected T-score (≤36) yielded .51 sensitivity and .94 specificity. Slightly lower sensitivity (.40) and higher specificity (.98) was associated with a more conservative T-score cutoff of ≤33. Multivariate models incorporating both LNS and RDS improved classification accuracy (area under the curve of .86), and LNS scores explained a significant but modest proportion of variance in validity status above and beyond RDS. Chaining LNS T-score of ≤33 with RDS cutoff of ≤7 increased sensitivity to .69 while maintaining ≥.90 specificity. Conclusions: Findings provide preliminary evidence for the criterion and construct validity of LNS as an embedded validity indicator in ADHD evaluations. Practitioners are encouraged to use LNS T-score cutoff of ≤33 or ≤36 to assess the validity of obtained test data. Employing either of these LNS cutoffs with RDS may enhance the detection of invalid performance.

Performance and symptom validity indicators among children undergoing cognitive surveillance following treatment for craniopharyngioma.

Performance validity tests (PVTs) and symptom validity tests (SVTs) are essential to neuropsychological evaluations, helping ensure findings reflect true abilities or concerns. It is unclear how PVTs and SVTs perform in children who received radiotherapy for brain tumors. Accordingly, we investigated the rate of noncredible performance on validity indicators as well as associations with fatigue and lower intellectual functioning. Embedded PVTs and SVTs were investigated in 98 patients with pediatric craniopharyngioma undergoing proton radiotherapy (PRT). The contribution of fatigue, sleepiness, and lower intellectual functioning to embedded PVT performance was examined. Further, we investigated PVTs and SVTs in relation to cognitive performance at pre-PRT baseline and change over time. SVTs on parent measures were not an area of concern. PVTs identified 0-31% of the cohort as demonstrating possible noncredible performance at baseline, with stable findings 1 year following PRT. Reliable digit span (RDS) noted the highest PVT failure rate; RDS has been criticized for false positives in pediatric populations, especially children with neurological impairment. Objective sleepiness was strongly associated with PVT failure, stressing need to consider arousal level when interpreting cognitive performance in children with craniopharyngioma. Lower intellectual functioning also needs to be considered when interpreting task engagement indices as it was strongly associated with PVT failure. Embedded PVTs should be used with caution in pediatric craniopharyngioma patients who have received PRT. Future research should investigate different cut-off scores and validity indicator combinations to best differentiate noncredible performance due to task engagement versus variable arousal and/or lower intellectual functioning.

Cognitive Underperformance in a Mixed Neuropsychiatric Sample at Diagnostic Evaluation of Adult ADHD.

(1) Background: The clinical assessment of attention-deficit/hyperactivity disorder (ADHD) in adulthood is known to show non-trivial base rates of noncredible performance and requires thorough validity assessment. (2) Objectives: The present study estimated base rates of noncredible performance in clinical evaluations of adult ADHD on one or more of 17 embedded validity indicators (EVIs). This study further examines the effect of the order of test administration on EVI failure rates, the association between cognitive underperformance and symptom overreporting, and the prediction of cognitive underperformance by clinical information. (3) Methods: A mixed neuropsychiatric sample (N = 464, ADHD = 227) completed a comprehensive neuropsychological assessment battery on the Vienna Test System (VTS; CFADHD). Test performance allows the computation of 17 embedded performance validity indicators (PVTs) derived from eight different neuropsychological tests. Further, all participants completed several self- and other-report symptom rating scales assessing depressive symptoms and cognitive functioning. The Conners' Adult ADHD Rating Scale and the Beck Depression Inventory-II were administered to derive embedded symptom validity measures (SVTs). (4) Results and conclusion: Noncredible performance occurs in a sizeable proportion of about 10% up to 30% of individuals throughout the entire battery. Tests for attention and concentration appear to be the most adequate and sensitive for detecting underperformance. Cognitive underperformance represents a coherent construct and seems dissociable from symptom overreporting. These results emphasize the importance of performing multiple PVTs, at different time points, and promote more accurate calculation of the positive and negative predictive values of a given validity measure for noncredible performance during clinical assessments. Future studies should further examine whether and how the present results stand in other clinical populations, by implementing rigorous reference standards of noncredible performance, characterizing those failing PVT assessments, and differentiating between underlying motivations.

76 Cross-Cultural Utility of Performance Validity Indicators in a Community Sample from Kampala, Uganda

Objective:Empirical support for inclusion of performance validity testing (PVTs) in neuropsychological assessment continues to grow (Sweet et al., 2021). However, considerable validation is still needed to understand the impact of culturally mediated factors on the reliability of current, commonly used PVTs to accurately classify effort among various cultural groups. This study sought to contribute to the literature by examining the utility of several PVTs in a non-clinical, community-dwelling sample in Kampala, Uganda.Participants and Methods:Participants included 52 residents (25 Female, 27 Male) who were born between 1953-2003 from the Wabigalo community of central Kampala. Individuals were recruited by community leaders and volunteered to participate. All 52 participants were administered the Dot Counting Test (DCT; Boone et al., 2002), Test of Memory Malingering (TOMM; Tombaugh, 1997), and Rey 15-Item Memorization Test (Rey 15; Lezak, 1995). Twenty-five participants also completed Green’s Non-Verbal Medical Symptom Validity Test (NV-MSVT; Green, 2006). Data from three participants was excluded due to suspected memory concerns. Instructions for all tests were translated into Luganda by a professional translator with experience in Luganda and were administered by Luganda-speaking individuals.Results:Using test manual-derived cut scores, 71.4% (n = 35) participants scored in the invalid range on the DCT, 10.2% (n = 5) produced total combined scores in the invalid range on Rey 15, 6.1% (n = 3) failed TOMM Trial 2, and one participant (4.3%) exceeded cut-offs on Green’s NV-MSVT.Conclusions:In this non-clinical sample, manual cutoffs for DCT contributed to a high type-1 error rate. These findings suggest that culturally mediated factors may contribute to differences in engagement or performance on DCT. Future studies should explore these factors and continue to examine the utility of widely used tests in diverse samples.

78 Utility of the D-KEFS Color Word Interference Test as a Measure of Performance Validity in Adults Referred for a Psychoeducational Evaluation

Objective:Previous investigations have demonstrated the clinical utility of the Delis-Kaplan Executive Function System (D-KEFS) Color Word Interference Test (CWIT) as an embedded validity indicator in mixed clinical samples and traumatic brain injury. The present study sought to cross-validate previously identified indicators and cutoffs in a sample of adults referred for psychoeducational testing.Participants and Methods:Archival data from 267 students and community members self-referred for a psychoeducational evaluation at a university clinic in the South were analyzed. Referrals included assessment for attention-deficit hyperactivity disorder, specific learning disorder, autism spectrum disorder, or other disorders (e.g., anxiety, depression). Individuals were administered subtests of the D-KEFS including the CWIT and several standalone and embedded performance validity indicators as part of the evaluation. Criterion measures included The b Test, Victoria Symptom Validity Test, Medical Symptom Validity Test, Dot Counting Test, and Reliable Digit Span. Individuals who failed 0 criterion measures were included in the credible group (n = 164) and individuals failing 2 or more criterion measures were included in the non-credible group (n = 31). Because a subset of the sample were seeking external incentives (e.g., accommodations), individuals who failed only 1 of the criterion measures were excluded (n = 72). Indicators of interest included all test conditions examined separately, the inverted Stroop index (i.e., better performance on the interference trial than the word reading or color naming trials), inhibition and inhibition/switching composite, and sum of all conditions.Results:Receiver Operating Characteristics (ROC) curves were significant for all four conditions (p < .001) and the inverted stroop index (p = .032). However, only conditions 2, 3 and 4 met minimal acceptable classification accuracy (AUC = .72 - 81). ROC curves with composite indicators were also significant (p < .001), with all three composite indicators meeting minimal acceptable classification accuracy (AUC = .71- .80). At the previously identified cutoff of age corrected scale score of 6 for all four conditions, specificity was high (.88 -.91), with varying sensitivity (.23 - .45). At the previously identified cutoff of .75 for the inverted stroop index, specificity was high (.87) while sensitivity was low (.19). Composite indicators yielded high specificity (.88 - .99) at previously established cutoffs with sensitivity varying from low to moderate (.19 - .48). Increasing the cutoffs (i.e., requiring higher age corrected scale score to pass) for composite indicators increased sensitivity while still maintaining high specificity. For example, increasing the total score cutoff from 18 to 28 resulted in moderate sensitivity (.26 vs .52) with specificity of .91.Conclusions:While a cutoff of 6 resulted in high specificity for most conditions, the sum of all four conditions exhibited the strongest classification accuracy and appears to be the most robust indicator which is consistent with previous research (Eglit et al., 2019). However, a cutoff of 28 as opposed to 18 may be most appropriate for psychoeducational samples. Overall, the results suggest that the D-KEFS CWIT can function as a measure of performance validity in addition to a measure of processing speed/executive functioning.

81 An Examination of a Multiple Cutoff Performance Validity Index

Objective:Recent consensus guidelines have advocated for the use of multivariate performance validity assessment on ability-based measures such those used in neuropsychological assessment. Further, previous research has demonstrated that aggregating performance validity indicators may produce superior classification accuracy. The present study builds upon this research by aggregating data from three of the most commonly used performance validity measures (Test of Memory Malingering [TOMM], Rey Fifteen Item Test with recognition trial [FIT plus recognition], and Reliable Digit Span [RDS]) to create a performance validity composite measure in a veteran mild traumatic brain injury (mTBI) population.Participants and Methods:Data of patients evaluated at a VA hospital who had completed the RDS, FIT plus recognition, and TOMM as part of their clinical neuropsychological evaluation were analyzed (n = 20). Two composite performance validity indexes were created: a Single Cutoff Performance Validity Index (SC-PVI), which measures the quantity of failures across performance validity measures (PVMs) by summing the total number of PVM failures, and a Multiple Cutoff Performance Validity Index (MC-PVI) which measures the number of failures as well as degree of failure(s) across measures of performance validity (e.g., a participant would attain a score of 3 if their PVM performance failed to reach a conservative cut point; they would obtain a score of 1 if they met conservative cut point, yet failed to reach a liberal cut point).Results:Only one participant (5%) attained a score of 0 on the SC-PVI (i.e., passing all PVTs using standard cutoffs) and MC-PVI (i.e., passing the most liberal cut points on all three PVMs). Conversely, eight participants (40%) attained a score of 3 on the SC-PVI (i.e., failed all three PVMs) and four participants (20%) attained a score of 9 (i.e., failed the most conservative cut points on all three PVMs). Results showed a significant (p < .001) ordinal association between the two indices (G = .984); however, there was no significant agreement between SC-PVI and MC-PVI models (k = -.087; p = .127).Conclusions:Data revealed discordant findings between the three PVMs utilized. The majority of participants (75%) scored between 2-8 on the MC-PVI, meaning that they did not exceed all liberal cut points or fail all conservative cut points. These “grey area” scores suggest an indeterminate range of performance validity, which cannot be captured by a solitary cut point or neatly classified as pass or fail. The utility of multiple cutoff performance validity models (i.e., aggregating PVMs to consider the severity of failure and number of failures) is that they capture the nuance of these data when determining and discussing the credibility of a profile. Multiple cut point data also highlight how the choice of cutoff influences the outcome of performance validity research and clinical decision making. As such, future research on the classification accuracy of this MC-PVI is needed.

A - 184 Validity of the Infrequent Patterns of Performance Embedded Performance Validity Indicators in a Mixed Clinical Sample.

Novel performance validity tests (PVTs) for the Hopkins Verbal Learning Test-Revised (HVLT-R) and Brief Visuospatial Memory Test-Revised (BVMT-R), derived from infrequent patterns of performance (IPOP), were developed to examine invalid memory performance while reducing false positive misclassification. To examine their classification accuracy, these indicators were compared to several existing embedded PVTs in a mixed clinical sample. A chart review was completed for 75 adult outpatients (Mean Age = 52, Range = 19-85) referred to an academic medical center outpatient clinic for neuropsychological evaluation. Twenty-eight percent of the sample failed the Medical Symptom Validity Test (MSVT), which was used as the criterion to indicate invalid performance. Several established embedded validity indicators were calculated from HVLT-R and BVMT-R scores, including the recognition discrimination index (RDI) and percent retention (PR), as well as the Reliable Digit Span (RDS) validity indicator. Classification statistics, including sensitivity, specific, likelihood ratios, predictive values, and overall accuracy, were calculated and compared. The HVLT-R IPOP had a higher overall classification accuracy (72%) compared to the RDI indicator (68%) while maintaining excellent specificity (97%). The BVMT-R IPOP had identical classification accuracy (75%) compared to the RDI and PR indicators with very high specificity (96%). Both indicators were more accurate than RDS (72%). The HVLT-R and BVMT-R IPOP performance validity indicators produced adequate classification statistics when compared to existing embedded PVTs. Both indicators maintained excellent specificity, which is crucial in reducing false positive classification. Future research will focus on further examining the validity of these indicators in other clinical groups and known groups, including medico-legal populations.

A - 183 Novel Performance Validity Indicators for Frequently Used Memory Tests Using Infrequent Patterns of Memory Performance.

Determining performance validity is an imperative part of neuropsychological assessment in concussion management. Several existing embedded performance validity tests (PVTs) have been developed for popular measures of memory, Hopkins Verbal Learning Test Revised (HVLT-R) and Brief Visuospatial Memory Test Revised (BVMT-R). However, many PVTs utilize cognitive performance measures to classify performance invalidity, which increases the likelihood of false positives, especially in clinical populations. To reduce this possibility, a novel embedded PVT was developed utilizing infrequent patterns of performance in a primarily amnestic, dementia population. Neuropsychological data for 50 adult outpatients (Mean Age = 73, Range = 50-97) who met criteria for Major Neurocognitive Disorder (MND) were reviewed. Infrequent patterns of performance (IPOP) were identified on the HVLT-R and BVMT-R, including inconsistency between recall and recognition, unusually high intrusions, exceedingly poor learning, and lack of primary effect. Scaled scores were developed from the IPOP scores (0 = 0-74%, 1 = 75-94%, 2 = 95-99%). Total IPOP scaled scores were compared to existing embedded PVTs, which utilized recognition discrimination and percent retention scores, to examine classification rates. Failure rates on the embedded PVTs for the HVLT-R were 16-42% and BVMT-R 38-58% whereas elevated IPOP scaled scores were identified in less than 15% of the sample. High frequencies of embedded PVT failures were found in a clinical sample of patients with MND, which is concerning for false positive misclassification. Infrequent patterns of memory performances identified in a dementia population were less frequent and could potentially be utilized to examine performance validity in persons suspected of producing invalid memory performances.

Multivariate examination of embedded indicators of performance validity for ADHD evaluations: A targeted approach

This study investigated the individual and combined utility of 10 embedded validity indicators (EVIs) within executive functioning, attention/working memory, and processing speed measures in 585 adults referred for an attention-deficit/hyperactivity disorder (ADHD) evaluation. Participants were categorized into invalid and valid performance groups as determined by scores from empirical performance validity indicators. Analyses revealed that all of the EVIs could meaningfully discriminate invalid from valid performers (AUCs = .69-.78), with high specificity (≥90%) but low sensitivity (19%-51%). However, none of them explained more than 20% of the variance in validity status. Combining any of these 10 EVIs into a multivariate model significantly improved classification accuracy, explaining up to 36% of the variance in validity status. Integrating six EVIs from the Stroop Color and Word Test, Trail Making Test, Verbal Fluency Test, and Wechsler Adult Intelligence Scale-Fourth Edition was as efficacious (AUC = .86) as using all 10 EVIs together. Failing any two of these six EVIs or any three of the 10 EVIs yielded clinically acceptable specificity (≥90%) with moderate sensitivity (60%). Findings support the use of multivariate models to improve the identification of performance invalidity in ADHD evaluations, but chaining multiple EVIs may only be helpful to an extent.

Cross-validating the embedded performance validity indicators in the Rey Auditory Verbal Learning Test in mixed neuropsychiatric and attention-deficit/hyperactivity disorder clinical samples.

Cross-validating the embedded performance validity indicators in the Rey Auditory Verbal Learning Test in mixed neuropsychiatric and attention-deficit/hyperactivity disorder clinical samples.

A-298 Comparing the Use of the Automatized Sequences Task to the Rey-15 Item Test with an Inpatient Adolescent Population

Abstract Objective: In adult populations, performance validity indicators are well established, as they are efficient and can be a standalone to measure effort. There less known about the validity of effort testing with adolescents, especially when conducting neuropsychological evaluations on an inpatient unit. Method: This study examined the performance of adolescents admitted to an inpatient psychiatric unit on the Automatized Sequences Task (AST) (aged 13-19; n = 74) and Rey-15 Item Test (aged 13-19; n = 74). Results: 86.5% of the adolescent passed the AST (M = .86; SD = .34) and 97.5% of adolescents passed the Rey-15 (M = 13.6 items correct, SD = 2.64). Conclusion(s): Performance on both PVTs demonstrated the helpfulness in measuring effort with adolescents on an inpatient unit. Further research is needed to determine whether the adolescents who performed below recommended cutoffs on AST reflects genuine suboptimal effort, cognitive difficulties with these adolescents, and/or other factors.

A-232 Head-To-Head Comparison of the Rey Auditory Verbal Learning Test Effort Score and Forced Choice Embedded Performance Validity Indicators Among Patients with and without Verbal Memory Impairment

Abstract Objective: This study directly compared the classification accuracies of the Rey Auditory Verbal Learning Test (RAVLT) Effort Score (ES) and Forced Choice (FC) embedded performance validity indicators among patients with and without verbal memory impairment. Methods: This study included data from 268 ethnoracially and diagnostically diverse (41% Black; 36% White; 16 % Hispanic; 5% Asian; 2% other), with mean age of 46.13 (SD=16.08) and mean education of 13.91 years (SD=2.80). Receiver operating characteristic (ROC) curve analyses assessed classification accuracy via areas under the curve (AUCs) and facilitated selection of optimal cut-scores and their associated sensitivities/specificities for the RAVLT ES and FC among the unimpaired verbal memory (≥37T) (n=135) and impaired memory (≤36T) (n=76) subsamples. Results: Among the overall sample, ES has acceptable classification accuracy (AUC=.724), with an optimal cutoff of ≤2 (26% sensitivity/90% specificity). FC also had acceptable accuracy (AUC = .764), with an optimal cut score ≤12 (47% sensitivity/90% specificity). For the unimpaired group, ES (AUC = .863) had an optimal cut score of ≤10 (67% sensitivity/90% specificity) and Forced Choice (AUC = .836) had an optimal cutoff of ≤14 (75% sensitivity/87% specificity). In the memory impaired group, ES no longer had significant classification accuracy (AUC = .477), whereas FC remained significant (AUC = .639), with an optimal cut score of c11 (30% sensitivity/90% specificity). Conclusion: When compared head-to-head, RAVLT FC was a superior performance validity measure for both memory impaired and unimpaired patients compared to the ES, which only worked well as a validity index among those without significant memory impairment.

Preliminary evidence for digit span performance validity indicators within the neuropsychological assessment battery

The purpose of this study was to evaluate multiple embedded performance validity indicators within the Digits Forward and Digits Backward subtests of the Neuropsychological Assessment Battery (NAB), including Reliable Digit Span (RDS), as no published papers have examined embedded digit span validity indicators within these subtests of the NAB. Retrospective archival chart review was conducted at an outpatient neuropsychology clinic. Participants were 92 adults (ages 19−68) who completed NAB Digits Forward and Digits Backward, and the Word Choice Test (WCT). Receiver operating characteristic (ROC) curves, t-tests, and sensitivity and specificity analyses were conducted. Analyses showed that RDS demonstrated acceptable classification accuracy between those who passed the WCT and those who did not. The area under the curve (AUC) value for RDS was 0.702; however, AUC values for all other digit span indices were unacceptably low. The optimal cutoff for RDS was identified (<8). RDS for the NAB appears to be an adequate indicator of performance validity; however, considering the very small number of participants who were invalid on the WCT (n = 15), as well as the utilization of only one stand-alone PVT to classify validity status, these findings are preliminary and in need of replication.

They are not destined to fail: a systematic examination of scores on embedded performance validity indicators in patients with intellectual disability

ABSTRACT This study was designed to determine the clinical utility of embedded performance validity indicators (EVIs) in adults with intellectual disability (ID) during neuropsychological assessment. Based on previous research, unacceptably high (>16%) base rates of failure (BR Fail ) were predicted on EVIs using on the method of threshold, but not on EVIs based on alternative detection methods. A comprehensive battery of neuropsychological tests was administered to 23 adults with ID (M Age = 37.7 years, M FSIQ = 64.9). BR Fail were computed at two levels of cut-offs for 32 EVIs. Patients produced very high BR Fail on 22 EVIs (18.2%-100%), indicating unacceptable levels of false positive errors. However, on the remaining ten EVIs BR Fail was <16%. Moreover, six of the EVIs had a zero BR Fail , indicating perfect specificity. Consistent with previous research, individuals with ID failed the majority of EVIs at high BR Fail . However, they produced BR Fail similar to cognitively higher functioning patients on select EVIs based on recognition memory and unusual patterns of performance, suggesting that the high BR Fail reported in the literature may reflect instrumentation artefacts. The implications of these findings for clinical and forensic assessment are discussed.

The emotion word fluency test as an embedded performance validity indicator – Alone and in a multivariate validity composite

Objective This project was designed to cross-validate existing performance validity cutoffs embedded within measures of verbal fluency (FAS and animals) and develop new ones for the Emotion Word Fluency Test (EWFT), a novel measure of category fluency. Method The classification accuracy of the verbal fluency tests was examined in two samples (70 cognitively healthy university students and 52 clinical patients) against psychometrically defined criterion measures. Results A demographically adjusted T-score of ≤31 on the FAS was specific (.88–.97) to noncredible responding in both samples. Animals T ≤ 29 achieved high specificity (.90–.93) among students at .27–.38 sensitivity. A more conservative cutoff (T ≤ 27) was needed in the patient sample for a similar combination of sensitivity (.24–.45) and specificity (.87–.93). An EWFT raw score ≤5 was highly specific (.94–.97) but insensitive (.10–.18) to invalid performance. Failing multiple cutoffs improved specificity (.90–1.00) at variable sensitivity (.19–.45). Conclusions Results help resolve the inconsistency in previous reports, and confirm the overall utility of existing verbal fluency tests as embedded validity indicators. Multivariate models of performance validity assessment are superior to single indicators. The clinical utility and limitations of the EWFT as a novel measure are discussed.

Critical Item Analysis Enhances the Classification Accuracy of the Logical Memory Recognition Trial as a Performance Validity Indicator

ABSTRACT Objective : Replicate previous research on Logical Memory Recognition (LM Recog ) and perform a critical item analysis. Method : Performance validity was psychometrically operationalized in a mixed clinical sample of 213 adults. Classification of the LM Recog and nine critical items (CR-9) was computed. Results : LM Recog ≤20 produced a good combination of sensitivity (.30-.35) and specificity (.89-.90). CR-9 ≥5 and ≥6 had comparable classification accuracy. CR-9 ≥5 increased sensitivity by 4% over LM Recog ≤20; CR-9 ≥6 increased specificity by 6–8% over LM Recog ≤20; CR-9 ≥7 increased specificity by 8–15%. Conclusions : Critical item analysis enhances the classification accuracy of the optimal LM Recog cutoff (≤20).

Examining Conners Continuous Performance Test-3 (CPT-3) Embedded Performance Validity Indicators in an Adult Clinical Sample Referred for ADHD Evaluation

ABSTRACT This study evaluated multiple previously-identified Continuous Performance Test-Third Edition (CPT-3) scores as embedded validity indicators (EVIs) among 201 adults undergoing neuropsychological evaluation for Attention-Deficit/Hyperactivity Disorder (ADHD) divided into valid (n = 169) and invalid (n = 32) groups based on seven criterion measures. Although 6/10 CPT-3 scores accurately detected invalidity, only two reached minimally acceptable classification accuracy of ≥0.70. The remaining four had unacceptably low accuracy (AUCs = 0.62–0.69) with 0.19–0.41 sensitivity at ≥0.90 specificity. Composite scores did not provide better classification accuracy than individual CPT-3 scores. In sum, CPT-3 individual and composite scores generally are not accurate PVTs among adults undergoing clinical evaluation for ADHD.

Cross-validation of RAVLT performance validity indicators and the RAVLT/RO discriminant function in a large known groups sample

Objective: To cross-validate RAVLT performance validity cut-offs and the RAVLT/RO discriminant function in a large neuropsychological sample. Method: RAVLT scores and the RAVLT/RO discriminant function were compared in credible (n = 100) and noncredible (n = 353) neuropsychology referrals. Results: Noncredible patients scored lower than credible patients on RAVLT scores and the RAVLT/RO discriminant function. With cut-offs set to ≥90% specificity, highest sensitivities were observed for the discriminant function (cut-off ≤.064; 55.8%), recognition total (cut-off ≤9; 53.1%), the recognition combination score (≤10; 47.7%), and total learning across trials (cut-off ≤31; 45.3%). Individuals with histories of learning difficulties were over-represented in the 10% of credible patients exceeding cut-offs. When these individuals were removed, cut-offs could be tightened while still maintaining at least 90% specificity, and thereby increasing sensitivity (e.g., recognition total cut-off ≤10, 65% sensitivity; RAVLT/RO discriminant function cut-off ≤.176, 58% sensitivity). When three of the most sensitive, non-overlapping scores were considered in combination, 17% of credible patients failed ≥1 of the three cut-offs, while 3% failed two, and only 1% failed all three. In contrast, in the noncredible sample, more than two-thirds failed one or more of the three cut-offs, nearly half failed ≥2, and nearly a quarter failed all three. Conclusions: RAVLT PVT cut-offs and the RAVLT/RO discriminant function achieve approximately 50% sensitivity, and approach 65% sensitivity when cut-offs specific to samples without histories of learning problems are employed, confirming that RAVLT cut-offs and the RAVLT/RO discriminant function continue to be valuable techniques in the identification of performance invalidity.

Detecting Coached Feigning of Schizophrenia with the Inventory of Problems – 29 (IOP-29) and Its Memory Module (IOP-M): A Simulation Study on a French Community Sample

Evaluating the credibility of the respondent’s cognitive and psychological complaints is a core component of forensic mental health assessment. The Inventory of Problems-29 (IOP-29) and a new IOP-Memory module (IOP-M) are designed specifically for that purpose. Initial research suggests that they might provide practitioners with a quick and thus potentially efficient check of an examinee’s presentation credibility, based on the integration of both symptom and performance validity indicators. To contribute to emerging research on this topic, this study used a simulation design to examine the validity of the IOP-29 and IOP-M and their robustness in detecting coached feigning of schizophrenia. It is the first to extend this duo of measures to a French sample. Results from 115 volunteers supported the effectiveness of both IOP instruments. Because all participants were male, however, our findings may not generalize to females.