92 Validation of Coin-in-Hand Procedure in a Veteran Population

Abstract

Objective:Performance validity tests (PVTs) provide a methodological approach to detecting credible neurocognitive performances. This proves invaluable to the diagnostic process, as it allows neuropsychologists to objectively determine if an evaluation reflects a patient’s true neurocognitive abilities or if external factors are impacting the results. However, their addition to a testing battery can increase an already lengthy evaluation. As such, there is a need for sensitive but less time intensive PVTs. The purpose of this study is to validate the Coin-in-Hand (CIH) procedure as a quick and effective PVT within a veteran population.Participants and Methods:68 English-speaking patients were identified from an outpatient neuropsychological assessment dataset. Performances were correlated to the well- validated Reliable Digit Span (RDS), and several other soft indicators of task engagement including expanded COWAT, BVMT-False Alarms (FA), WCST Failure to Maintain Set (FTM), TOMM, and the RBANS Effort Index (EI). All participants attempted CIH and RDS, testing was discontinued if 2 or more PVTs were invalid. An AUC analysis was conducted to determine how well the CIH discriminated between valid and invalid performance and determine the tests optimal cut-off score (sensitivity > 0.90 while maintaining the highest possible specificity). Logistic Regression was conducted to determine how well the CIH predicted performance validity.Results:Subject mean(SD) age and education were 55.25 (16.06) and 13.41 (2.55) years, respectively. 17% female, 60% Caucasian, and 32% Black. Descriptive statistics for each of the other performance validity tests were gathered. The CIH demonstrated low diagnostic accuracy (AUC = .66; p >.05; CI = .51 -.81); a cut score of <8 resulted in a sensitivity of .96 and a specificty of .64. Logistic Regression showed that CIH performance significantly predicted performance validity (X2 = -0.93; df = 1; N = 68; p < .05), accounting for 18-28% of the variance in performance classification (Cox & Snell R2 = .18; Nagelkerke R2 = .28). It correctly classified 96% of valid performers, but only correctly classified 35% of invalid performers, with an overall correct prediction rate of 83%. A predicted chase in log odds (B= -.93) and odd ratio [Exp (B) =.40] indicated that every unit increase in CIH score was associated with a decrease probability of performance invalidity. Logistic regression was also used to calculate the probability of performance invalidity at each possible CIH score (Table 1).Conclusions:Results suggests that poor performance on CIH does not necessarily equate to invalid performances, but instead, should act as a screener to cue neuropsychologists working with Veterans that additional PVTs should be considered. Overall, it was determined that CIH was able to correctly predict 35% of invalid performers and 96% of valid performers, with an overall correct prediction rate of 83%, suggesting the procedure may be too simple to be an effective standalone PVT for clinical use. These results also highlight that every correct response on the CIH was associated with a decreased probability of performance invalidity. Additionally, an AUC analysis determined the tests optimal cut off score to be <8, suggesting that shortening the procedure may be as effective as giving the full 10 trials.