Performance Validity Assessment Research Articles

Seeing Clearly in the Twilight: The Clinical and Forensic Relevance of the Indeterminate/Borderline Range in Multivariate Models of Performance Validity Assessment

Seeing Clearly in the Twilight: The Clinical and Forensic Relevance of the Indeterminate/Borderline Range in Multivariate Models of Performance Validity Assessment

Detecting feigned cognitive impairment using pupillometry on the Warrington Recognition Memory Test for Words

ABSTRACT Objective Pupillometry provides information about physiological and psychological processes related to cognitive load, familiarity, and deception, and it is outside of conscious control. This study examined pupillary dilation patterns during a performance validity test (PVT) among adults with true and feigned impairment of traumatic brain injury (TBI). Participants and Methods Participants were 214 adults in three groups: adults with bona fide moderate to severe TBI (TBI; n = 51), healthy comparisons instructed to perform their best (HC; n = 72), and healthy adults instructed and incentivized to simulate cognitive impairment due to TBI (SIM; n = 91). The Recognition Memory Test (RMT) was administered in the context of a comprehensive neuropsychological battery. Three pupillary indices were evaluated. Two pure pupil dilation (PD) indices assessed a simple measure of baseline arousal (PD-Baseline) and a nuanced measure of dynamic engagement (PD-Range). A pupillary-behavioral index was also evaluated. Dilation-response inconsistency (DRI) captured the frequency with which examinees displayed a pupillary familiarity response to the correct answer but selected the unfamiliar stimulus (incorrect answer). Results All three indices differed significantly among the groups, with medium-to-large effect sizes. PD-Baseline appeared sensitive to oculomotor dysfunction due to TBI; adults with TBI displayed significantly lower chronic arousal as compared to the two groups of healthy adults (SIM, HC). Dynamic engagement (PD-Range) yielded a hierarchical structure such that SIM were more dynamically engaged than TBI followed by HC. As predicted, simulators engaged in DRI significantly more frequently than other groups. Moreover, subgroup analyses indicated that DRI differed significantly for simulators who scored in the invalid range on the RMT (n = 45) versus adults with genuine TBI who scored invalidly (n = 15). Conclusions The findings support continued research on the application of pupillometry to performance validity assessment: Overall, the findings highlight the promise of biometric indices in multimethod assessments of performance validity.

Cutoff Elasticity in Multivariate Models of Performance Validity Assessment as a Function of the Number of Components and Aggregation Method

Cutoff Elasticity in Multivariate Models of Performance Validity Assessment as a Function of the Number of Components and Aggregation Method

27 Green’s Word Memory Test (WMT) Immediate Recall as a Screening Tool for Performance Invalidity

Objective:Assessment of performance validity during neuropsychology evaluation is essential to reliably interpret cognitive test scores. Studies (Webber et al., 2018; Wisdom, et al., 2012) have validated the use of abbreviated measures, such as Trial 1 (T1) of the Test of Memory Malingering (TOMM), to detect invalid performance. Only one study (Bauer et al., 2007) known to these authors has examined the utility of Green’s Word Memory Test (WMT) immediate recall (IR) as a screening tool for invalid performance. This study explores WMT IR as an independent indicator of performance validity in a mild TBI (mTBI) veteran population.Participants and Methods:Participants included 211 (Mage = 32.1, SD = 7.4; Medu = 13.1, SD = 1.64; 94.8% male; 67.8% White) OEF/OIF/OND veterans with a history of mTBI who participated in a comprehensive neuropsychological evaluation at one of five participating VA Medical Centers. Performance validity was assessed using validated cut scores from the following measures: WMT IR and delayed recall (DR); TOMM T1; WAIS-IV reliable digit span; CVLT-II forced choice raw score; Wisconsin Card Sorting Test failure to maintain set; and the Rey Memory for Fifteen Items test, combo score. Sensitivity and specificity were calculated for each IR score compared with failure on DR. In addition, sensitivity and specificity were calculated for each WMT IR score compared to failure of at least one additional performance validity measure (excluding DR), two or more measures, and three or more measures, respectively.Results:Results indicated that 46.8% participants failed to meet cut offs for adequate performance validity based on the standard WMT IR cut score (i.e., 82.5%; M = 81.8%, SD = 17.7%); however, 50.2% participants failed to meet criteria based on the standard WMT DR cut score (M = 79.8% SD = 18.6%). A cut score of 82.5% or below on WMT IR correctly identified 82.4% (i.e., sensitivity) of subjects who performed below cut score on DR, with a specificity of 94.2%. Examination of IR cutoffs compared to failure of one or more other PVTs revealed that the standardized cut score of 82.5% or below had a sensitivity of 78.2% and a specificity of 72.4%; whereas, a cut score of 65% or below had a sensitivity of 41% and a specificity of 91.3%. Similarly, examination of IR cutoffs compared to failure of two or more additional PVTs revealed that the cut score of 60% or below had a sensitivity of 45.7% and specificity of 93.1% ; whereas, a cut score of 57.5% or below had a sensitivity of 57.9% and specificity of 90.9% when using failure of three or more PVTs as the criterion.Conclusions:Results indicated that a cut score of 82.5% or below on WMT IR may be sufficient to detect invalid performance when considering WMT DR as criterion. Furthermore, WMT IR alone, with adjustments to cut scores, appears to be a reasonable way to assess symptom validity compared to other PVTs. Sensitivity and specificity of WMT IR scores may have been adversely impacted by lower sensitivity of other PVTs to independently identify invalid performance.

81 An Examination of a Multiple Cutoff Performance Validity Index

Objective:Recent consensus guidelines have advocated for the use of multivariate performance validity assessment on ability-based measures such those used in neuropsychological assessment. Further, previous research has demonstrated that aggregating performance validity indicators may produce superior classification accuracy. The present study builds upon this research by aggregating data from three of the most commonly used performance validity measures (Test of Memory Malingering [TOMM], Rey Fifteen Item Test with recognition trial [FIT plus recognition], and Reliable Digit Span [RDS]) to create a performance validity composite measure in a veteran mild traumatic brain injury (mTBI) population.Participants and Methods:Data of patients evaluated at a VA hospital who had completed the RDS, FIT plus recognition, and TOMM as part of their clinical neuropsychological evaluation were analyzed (n = 20). Two composite performance validity indexes were created: a Single Cutoff Performance Validity Index (SC-PVI), which measures the quantity of failures across performance validity measures (PVMs) by summing the total number of PVM failures, and a Multiple Cutoff Performance Validity Index (MC-PVI) which measures the number of failures as well as degree of failure(s) across measures of performance validity (e.g., a participant would attain a score of 3 if their PVM performance failed to reach a conservative cut point; they would obtain a score of 1 if they met conservative cut point, yet failed to reach a liberal cut point).Results:Only one participant (5%) attained a score of 0 on the SC-PVI (i.e., passing all PVTs using standard cutoffs) and MC-PVI (i.e., passing the most liberal cut points on all three PVMs). Conversely, eight participants (40%) attained a score of 3 on the SC-PVI (i.e., failed all three PVMs) and four participants (20%) attained a score of 9 (i.e., failed the most conservative cut points on all three PVMs). Results showed a significant (p < .001) ordinal association between the two indices (G = .984); however, there was no significant agreement between SC-PVI and MC-PVI models (k = -.087; p = .127).Conclusions:Data revealed discordant findings between the three PVMs utilized. The majority of participants (75%) scored between 2-8 on the MC-PVI, meaning that they did not exceed all liberal cut points or fail all conservative cut points. These “grey area” scores suggest an indeterminate range of performance validity, which cannot be captured by a solitary cut point or neatly classified as pass or fail. The utility of multiple cutoff performance validity models (i.e., aggregating PVMs to consider the severity of failure and number of failures) is that they capture the nuance of these data when determining and discussing the credibility of a profile. Multiple cut point data also highlight how the choice of cutoff influences the outcome of performance validity research and clinical decision making. As such, future research on the classification accuracy of this MC-PVI is needed.

89 Detecting Feigned Cognitive Impairment Using Pupillometry on the Warrington Recognition Memory Test for Words

Objective:Traditional methods of assessing performance validity have numerous weaknesses, among them, results can be consciously manipulated by examinees who wish to feign cognitive impairment. This study tested the ability of pupillary dilation patterns during a performance validity test (PVT) to enhance diagnostic accuracy in discriminating true from feigned impairment of traumatic brain injury (TBI). Pupillometry provides information about physiological and psychological processes related to cognitive load, familiarity, and deception and is outside of conscious control. Patrick, Rapport, Kanser, Hanks, and Bashem (2021) established proof of concept for the utility of pupillometry with PVTs applied to the Test of Memory Malingering (TOMM). This study replicated and extended this work by evaluating the incremental utility of pupillary-derived indices on the Warrington Recognition Memory Test for Words (RMT).Participants and Methods:Participants included 214 adults in three groups: adults with bona fide TBI (TBI; n = 51) healthy comparisons instructed to perform their best (HC; n = 72), and healthy adults instructed and incentivized to simulate cognitive impairment due to TBI (SIM; n = 91). Moreover, this study examined pupillary pattern differences among successful (i.e., failed < 1 PVT and performed impaired on cognitive tests) and unsuccessful (i.e., failed > 2 PVTs or did not score impaired on a cognitive test) SIM, including SIM who did and did not fail the RMT. The RMT was administered in the context of a comprehensive neuropsychological battery. Indices included two pure pupil dilation (PD) indices: a simple measure of baseline arousal (PD-Baseline) and a nuanced measure of dynamic engagement (PD-Range). A pupillo-behavioral index was also evaluated: Dilation-response inconsistency (DRI) captured the frequency with which examinees displayed a pupillary familiarity response to the correct answer but selected the unfamiliar stimulus (incorrect answer).Results:The results generally replicated Patrick et al. (2021), as all three indices were useful in discriminating between groups and provided incremental utility to traditional accuracy scores. PD-Baseline appeared sensitive to oculomotor dysfunction due to TBI (i.e., increasing accurate identification of that group); adults with TBI displayed significantly lower chronic arousal as compared to the two groups of healthy adults (SIM, HC). In fact, the TBI group showed significantly lower PD-Baseline than both unsuccessful simulators who were detected as feigners and successful simulators who passed PVTs but effectively feigned TBI on other tests. Dynamic engagement (PD-Range) yielded a hierarchical structure such that SIM were more dynamically engaged than TBI followed by HC. As predicted, simulators engaged in DRI significantly more frequently than other groups. Moreover, DRI added unique information to RMT accuracy in classifying unsuccessful simulators from all other groups. Each of these three pupillary indices showed large effect sizes, and logistic regressions indicated that each contributed unique variance in predicting group membership on one or more of the paired contrasts (i.e., SIM-TBI, SIM-HC, HC-TBI).Conclusions:Taken together, the findings support continued research on the application of pupillometry to performance validity assessment: Pupillometry provided unique information in enhancing classification accuracy beyond traditional PVT accuracy scores. Overall, the findings highlight the promise of biometric indices in multimethod assessments of performance validity.

Questioning What We Thought We Knew: Commentary on Leonhard's Performance Validity Assessment Articles.

Neuropsychologists have long understood that valid examinee performance is needed in order to understand the constructs of interest that are at the heart of clinical and forensic evaluations. The assessment of performance validity has evolved over time, from very rudimentary and subjective clinical impressions of examinee task engagement to psychometrically based, multi-method, algorithm-driven, and consensus-informed approaches. Christoph Leonhard has further advanced that evolution in a meaningful way, forcing us to reconsider much of what we thought we knew about the psychometric assessment of performance validity. Although a structured, systematic, and objective approach to validity assessment is necessary, Leonhard has brought to our attention some significant concerns that need to be addressed. This commentary describes professional, ethical, and legal implications of Leonhard's articles. Through an ongoing process of examining, revising, and improving our methods and procedures, we will be better positioned to provide services of value to those we serve. Leonhard has provided an opportunity for us to do just that.

Performance and symptom validity assessment in attention deficit/hyperactivity disorder: Base rates of invalidity, concordance, and relative impact on cognitive performance

Objective: Differential diagnosis of attention deficit/hyperactivity disorder (ADHD) is one of the most common referral questions for neuropsychological evaluation but is complicated by the presence of external incentives. Validity assessment is therefore critical in such evaluations, employing symptom validity tests (SVTs) and performance validity tests (PVTs) to assess the validity of reported symptoms and cognitive test performance, respectively. This study aimed to establish the base rate of symptom and performance invalidity in adults referred for ADHD, compare concordance between performance and symptom validity, and assess the impact of each type of validity on cognitive test performance. Method: This consecutive case series included data from 392 demographically-diverse adults who underwent outpatient neuropsychological evaluation for ADHD. All patients were administered the Clinical Assessment of Attention Deficit-Adult (CAT-A) and a uniform cognitive test battery, including seven PVTs. Results: Invalid symptom reporting and PVT performance were found in 22% and 16% of the sample, respectively. Sixty-eight percent had concordantly valid SVTs/PVTs and 6% had invalid SVTs/PVTs, whereas the remaining 26% had either invalid SVTs or PVTs (but not both). Invalid PVT performance resulted in a significant decrease across all cognitive test scores, with generally large effects (ηp 2=.01-.18). Invalid symptom reporting had minimal effects on cognitive test performance (ηp 2= ≤.04). Conclusions: PVTs and SVTs are dissociable and therefore should not be used interchangeably in the context of adult ADHD evaluations. Rather, symptom and performance validity should continue to be assessed independently as they provide largely non-redundant information.

Assessment of Cultural Bias on the PdPVTS Across Gender and Racial/Ethnic Groups

Performance validity assessment has become a standard component of psychoeducational and neuropsychological evaluations (Sweet et al., 2021) but there is evidence to demonstrate that cultural variables can influence performance validity test outcomes. Given the critical role that performance validity assessment plays in determining the accuracy of neuropsychological and psychoeducational test data, it is imperative that these tests are unbiased across culturally diverse samples. The purpose of the current investigation was to examine the impact of race/ethnicity and gender on the Pediatric Performance Validity Test Suite (PdPVTS; McCaffrey, Lynch, Leark, & Reynolds, 2020). A general population sample of n = 838 examinees was collected and demographically matched subsamples were established to compare gender and racial/ethnic groups while controlling for other confounding demographic variables. Classification/failure rates revealed little evidence of adverse impact across gender and racial/ethnic groups. Mean score equivalency between demographic groups was achieved in most cases. Instances where evidence of equivalence could not be established, namely for the Story Questions, were associated with smaller sample sizes and lower statistical power. Taken together, the current study demonstrates that examiners may use the PdPVTS with confidence that pass/fail classifications will not be associated with gender or race/ethnicity for Black, White, and Hispanic examinees.

Attention Performance as an Embedded Validity Indicator in the Cognitive Assessment of Early Retirement Claimants

The assessment of performance validity is essential in any neuropsychological evaluation. However, relatively few measures exist that are based on attention performance embedded within routine cognitive tasks. The present study explores the potential value of a computerized attention test, the Cognitrone, as an embedded validity indicator in the neuropsychological assessment of early retirement claimants. Two hundred and sixty-five early retirement claimants were assessed with the Word Memory Test (WMT) and the Cognitrone. WMT scores were used as the independent criterion to determine performance validity. Speed and accuracy measures of the Cognitrone were analyzed in receiver operating characteristics (ROC) to classify group membership. The Cognitrone was sensitive in revealing attention deficits in early retirement claimants. Further, 54% (n = 143) of the individuals showed noncredible cognitive performance, whereas 46% (n = 122) showed credible cognitive performance. Individuals failing the performance validity assessment showed slower (AUC = 79.1%) and more inaccurate (AUC = 79.5%) attention performance than those passing the performance validity assessment. A compound score integrating speed and accuracy revealed incremental value as indicated by AUC = 87.9%. Various cut scores are suggested, resulting in equal rates of 80% sensitivity and specificity (cut score = 1.297) or 69% sensitivity with 90% specificity (cut score = 0.734). The present study supports the sensitivity of the Cognitrone for the assessment of attention deficits in early retirement claimants and its potential value as an embedded validity indicator. Further research on different samples and with multidimensional criteria for determining invalid performance are required before clinical application can be suggested.

Specificity of performance validity tests in patients with confirmed epilepsy

Objective: While assessment of performance validity is essential to neuropsychological evaluations, use of performance validity tests (PVTs) in an epilepsy population has raised concerns due to factors that may result in performance fluctuations. The current study assessed whether specificity was maintained at previously suggested cutoffs in a confirmed epilepsy population on the Warrington Recognition Memory Test (WRMT) – Words and Test of Memory Malingering (TOMM). Method: Eighty-two confirmed epilepsy patients were administered the WRMT-Words and TOMM as part of a standardized neuropsychological evaluation. Frequency tables were utilized to investigate specificity rates on these two PVTs. Results: The suggested WRMT-Words Accuracy Score cutoff of ≤42 was associated with a specificity rate of 90.2%. Five out of the 8 individuals falling below the Accuracy Score cutoff scored 42, suggesting specificity could be further improved by slightly lowering the cutoff. The WRMT-Words Total Time cutoff of ≥207 seconds was associated with 95.1% specificity. A TOMM Trial 1 cutoff of <40 was associated with 93.9% specificity, while the established cutoff of <45 on Trial 2 and the Retention Trial yielded specificity rates of 98.6% and 100.0%, respectively. Conclusions: Our findings demonstrate acceptable performance on two PVTs in a select confirmed epilepsy population without a history of brain surgery, active seizures during testing, and/or low IQ, irrespective of various factors such as seizure type, seizure lateralization/localization, and language lateralization. The possible presence of interictal discharges were not controlled for in the current study, which may have contributed to reduced PVT performances.

On the Use of Eye Movements in Symptom Validity Assessment of Feigned Schizophrenia

Assessing the credibility of reported mental health problems is critical in a variety of assessment situations, particularly in forensic contexts. Previous research has examined how the assessment of performance validity can be improved through the use of bio-behavioral measures (e.g., eye movements). To date, however, there is a paucity of literature on the use of eye tracking technology in assessing the validity of presented symptoms of schizophrenia, a disorder that is known to be associated with oculomotor abnormalities. Thus, we collected eye tracking data from 83 healthy individuals during the completion of the Inventory of Problems – 29 and investigated whether the oculomotor behavior of participants instructed to feign schizophrenia would differ from those of control participants asked to respond honestly. Results showed that feigners had a longer dwell time and a greater number of fixations in the feigning-keyed response options, regardless of whether they eventually endorsed those options (d > 0.80). Implications on how eye tracking technology can deepen comprehension on simulation strategies are discussed, as well as the potential of investigating eye movements to advance the field of symptom validity assessment.

Multivariate models provide an effective psychometric solution to the variability in classification accuracy of D-KEFS Stroop performance validity cutoffs

ObjectiveThe study was designed to expand on the results of previous investigations on the D-KEFS Stroop as a performance validity test (PVT), which produced diverging conclusions. Method The classification accuracy of previously proposed validity cutoffs on the D-KEFS Stroop was computed against four different criterion PVTs in two independent samples: patients with uncomplicated mild TBI (n = 68) and disability benefit applicants (n = 49). Results Age-corrected scaled scores (ACSSs) ≤6 on individual subtests often fell short of specificity standards. Making the cutoffs more conservative improved specificity, but at a significant cost to sensitivity. In contrast, multivariate models (≥3 failures at ACSS ≤6 or ≥2 failures at ACSS ≤5 on the four subtests) produced good combinations of sensitivity (.39-.79) and specificity (.85-1.00), correctly classifying 74.6-90.6% of the sample. A novel validity scale, the D-KEFS Stroop Index correctly classified between 78.7% and 93.3% of the sample. Conclusions A multivariate approach to performance validity assessment provides a methodological safeguard against sample- and instrument-specific fluctuations in classification accuracy, strikes a reasonable balance between sensitivity and specificity, and mitigates the invalid before impaired paradox.

Multivariate Models of Performance Validity: The Erdodi Index Captures the Dual Nature of Non-Credible Responding (Continuous and Categorical).

This study was designed to examine the classification accuracy of the Erdodi Index (EI-5), a novel method for aggregating validity indicators that takes into account both the number and extent of performance validity test (PVT) failures. Archival data were collected from a mixed clinical/forensic sample of 452 adults referred for neuropsychological assessment. The classification accuracy of the EI-5 was evaluated against established free-standing PVTs. The EI-5 achieved a good combination of sensitivity (.65) and specificity (.97), correctly classifying 92% of the sample. Its classification accuracy was comparable with that of another free-standing PVT. An indeterminate range between Pass and Fail emerged as a legitimate third outcome of performance validity assessment, indicating that the underlying construct is an inherently continuous variable. Results support the use of the EI model as a practical and psychometrically sound method of aggregating multiple embedded PVTs into a single-number summary of performance validity. Combining free-standing PVTs with the EI-5 resulted in a better separation between credible and non-credible profiles, demonstrating incremental validity. Findings are consistent with recent endorsements of a three-way outcome for PVTs (Pass, Borderline, and Fail).

CVLT-II short form forced choice recognition in a clinical dementia sample: Cautions for performance validity assessment

Performance validity tests are susceptible to false positives from genuine cognitive impairment (e.g., dementia); this has not been explored with the short form of the California Verbal Learning Test II (CVLT-II-SF). In a memory clinic sample, we examined whether CVLT-II-SF Forced Choice Recognition (FCR) scores differed across diagnostic groups, and how the severity of impairment [Clinical Dementia Rating Sum of Boxes (CDR-SOB) or Mini-Mental State Examination (MMSE)] modulated test performance. Three diagnostic groups were identified: subjective cognitive impairment (SCI; n = 85), amnestic mild cognitive impairment (a-MCI; n = 17), and dementia due to Alzheimer’s Disease (AD; n = 50). Significant group differences in FCR were observed using one-way ANOVA; post-hoc analysis indicated the AD group performed significantly worse than the other groups. Using multiple regression, FCR performance was modeled as a function of the diagnostic group, severity (MMSE or CDR-SOB), and their interaction. Results yielded significant main effects for MMSE and diagnostic group, with a significant interaction. CDR-SOB analyses were non-significant. Increases in impairment disproportionately impacted FCR performance for persons with AD, adding caution to research-based cutoffs for performance validity in dementia. Caution is warranted when assessing performance validity in dementia populations. Future research should examine whether CVLT-II-SF-FCR is appropriately specific for best-practice testing batteries for dementia.

Symptom and Performance Validity Assessment: European Trends in Research and Practice

This is a short editorial to the special issue of Psychological Injury and Law. The first of the articles of this issue (Merten et al., Symptom and performance validity assessment in European countries: an update, in this issue) was conceived as the proper introduction. The collection contains eight contributions from authors doing their research in eight different countries (Austria, Belgium, Germany, Great Britain, Italy, the Netherlands, Spain, and Switzerland), using a variety of approaches (review and empirical articles, social security and criminal background, etc.). They describe major research trends and the state-of-the-art of validity assessment across the continent. They also reflect the transition from malingering assessment to validity testing and the extension of this field from forensic applications to clinical and rehabilitation referral contexts.

Symptom and performance validity in samples of adults at clinical evaluation of ADHD: a replication study using machine learning algorithms

ABSTRACT Introduction Research has shown non-trivial base rates of noncredible symptom report and performance in the clinical evaluation of attention-deficit/hyperactivity disorder (ADHD) in adulthood. The goal of this study is to estimate and replicate base rates of symptom and performance validity test failure in the clinical evaluation of adult ADHD and derive prediction models based on routine clinical measures. Methods This study reuses data of a previous publication of 196 adults seeking ADHD assessment and replicates the findings on an independent sample of 700 adults recruited in the same referral context. Measures of symptom and performance validity (one SVT, two PVTs) were applied to estimate base rates. Prediction models were developed using machine learning. Results Both samples showed substantial rates of noncredible symptom report (one SVT failure: 35.7% – 36.6%), noncredible test performance (one PVT failure: 32.1% – 49.3%; two PVT failures: 18.9% – 27.3%), or both (each one SVT and PVT failure: 13.3% – 22.4%; one SVT and two PVT failures: 9.7% – 13.7%). Machine learning algorithms resulted in generally moderate to weak prediction models, with advantages of the reused sample compared to the independent replication sample. Associations between measures of symptom and performance validity were negligible to small. Conclusions This study highlights the necessity to include measures of symptom and performance validity in the clinical evaluation of adult ADHD. Further, this study demonstrates the difficulty to characterize the group failing symptom or performance validity assessment.

Symptom and Performance Validity Assessment in European Countries: an Update.

In 2013, a special issue of the Spanish journal Clínica y Salud published a review on symptom and performance validity assessment in European countries (Merten et al. in Clínica y Salud, 24(3), 129–138, 2013). At that time, developments were judged to be in their infancy in many countries, with major publication activities stemming from only four countries: Spain, The Netherlands, Great Britain, and Germany. As an introduction to a special issue of Psychological Injury and Law, this is an updated report of developments during the last 10 years. In that period of time, research activities have reached a level where it is difficult to follow all developments; some validity measures were newly developed, others were adapted for European languages, and validity assessment has found a much stronger place in real-world evaluation contexts. Next to an update from the four nations mentioned above, reports are now given from Austria, Italy, and Switzerland, too.

Five shades of gray: Conceptual and methodological issues around multivariate models of performance validity.

This study was designed to empirically investigate the signal detection profile of various multivariate models of performance validity tests (MV-PVTs) and explore several contested assumptions underlying validity assessment in general and MV-PVTs specifically. Archival data were collected from 167 patients (52.4%male; MAge = 39.7) clinicially evaluated subsequent to a TBI. Performance validity was psychometrically defined using two free-standing PVTs and five composite measures, each based on five embedded PVTs. MV-PVTs had superior classification accuracy compared to univariate cutoffs. The similarity between predictor and criterion PVTs influenced signal detection profiles. False positive rates (FPR) in MV-PVTs can be effectively controlled using more stringent multivariate cutoffs. In addition to Pass and Fail, Borderline is a legitimate third outcome of performance validity assessment. Failing memory-based PVTs was associated with elevated self-reported psychiatric symptoms. Concerns about elevated FPR in MV-PVTs are unsubstantiated. In fact, MV-PVTs are psychometrically superior to individual components. Instrumentation artifacts are endemic to PVTs, and represent both a threat and an opportunity during the interpretation of a given neurocognitive profile. There is no such thing as too much information in performance validity assessment. Psychometric issues should be evaluated based on empirical, not theoretical models. As the number/severity of embedded PVT failures accumulates, assessors must consider the possibility of non-credible presentation and its clinical implications to neurorehabilitation.

Future Directions in Performance Validity Assessment to Optimize Detection of Invalid Neuropsychological Test Performance: Special Issue Introduction.

Future Directions in Performance Validity Assessment to Optimize Detection of Invalid Neuropsychological Test Performance: Special Issue Introduction.