Using machine learning to detect noncredible cognitive test performance

Abstract

Objective: Advanced algorithmic methods may improve the assessment of performance validity during neuropsychological testing. This study investigated whether unsupervised machine learning (ML) could serve as one such method. Method: Participants were 359 adult outpatients who underwent a neuropsychological evaluation for various referral reasons. Data relating to participants’ performance validity test scores, medical and psychiatric history, referral reason, litigation status, and disability status were examined in an unsupervised ML model. The model was programmed to synthesize the data into an unspecified number of clusters, which were then compared to predetermined ratings of whether patients had valid or invalid test performance. Ratings were established according to multiple empirical performance validity test scores. To further understand the model, we examined which data were most helpful in its clustering decision-making process. Results: Similar to the clinical determination of patients’ performance on neuropsychological testing, the model identified a two-cluster profile consisting of valid and invalid data. The model demonstrated excellent predictive accuracy (area under the curve of .92 [95% CI .88, .97]) when referenced against participants’ predetermined validity status. Performance validity test scores were the most influential in the differentiation of clusters, but medical history, referral reason, and disability status were also contributory. Conclusions: These findings serve as a proof of concept that unsupervised ML can accurately assess performance validity using various data obtained during a neuropsychological evaluation. The manner in which unsupervised ML evaluates such data may circumvent some of the limitations with traditional validity assessment approaches. Importantly, unsupervised ML is adaptable to emerging digital technologies within neuropsychology that can be used to further improve the assessment of performance validity.