32 A Comparison of Cut-Off Points for Invalid Cognitive Test Performance Established on Nonclinical Versus Clinical Samples for South African Educationally Disadvantaged Individuals

Abstract

Objective:In South Africa, most of the cognitive tests employed for neuropsychological evaluation are those developed in educationally advantaged settings such as the US, but the normative data accompanying the tests are unsuitable for use with South African examinees who have a disadvantaged quality of education, and/or whose primary language is other than English. A recently completed collation of Africa-based normative data (Shuttleworth-Edwards & Truter, 2022) includes a chapter on Performance Validity Tests (PVTs) with proposed cut-off points to assist in the identification of noncredible performance. The aim of this study was to compare the cut-off points established using educationally disadvantaged South African nonclinical normative samples for which only specificity percentages are available, with those established using clinical samples with designated valid and invalid performers for which both specificity and sensitivity data are available. A further aim was to compare the Africa-based cut-off points with age-equivalent cut-off points where available for US-based data on the targeted tests.Participants and Methods:The collation of Africa-based studies delineates cut-off scores for invalid test performance based on both nonclinical as well as clinical populations for three stand-alone PVTs especially developed to identify invalid performance including the Dot Counting Test (DCT), the Rey Fifteen Item Test (FIT), and the Test of Memory Malingering (TOMM); and three commonly employed cognitive tests for which there are embedded validity indicators including the Digit Span Age-Corrected Scaled Score (ACSS) and Reliable Digit Span (RDS), the Rey Auditory Verbal Learning Test (RAVLT), and the Trail Making Test A and B (TMT A and B). For studies using nonclinical norming data alone, specificity percentages to derive the cut-off points were set at a minimum of 90%. For studies using clinical samples specificity was set at a minimum of 90%, and the associated sensitivity percentages were reported indicating each test’s ability to correctly identify those with an invalid performance. The studies included participants stratified for both child and adult age groups (age 8 to 79 years) from South African educationally disadvantaged backgrounds. The data were tabled together for descriptive comparison purposes, including a column for the US-base cut-off points for equivalent age stages where available.Results:There was a high level of compatibility between the proposed cut-off points established for the South African nonclinical normative samples compared with those using clinical samples of designated valid and invalid performers. There was a trend for more lenient cut-offs for younger children and older adults compared to older children and younger adults. Compared with US-based data where available, adjustments towards leniency were called-for on all indicators.Conclusions:Cut-off scores for invalid cognitive test performance can be verified by perusing data derived from nonclinical norming samples as well as those from clinical samples, although the latter have the advantage of providing the sensitivity data to demonstrate the efficacy of a proposed cut-off score for identifying noncredible test performance. Adjustments towards leniency need to be made for cut-off scores for young children and older adults within an educationally disadvantaged population, and for disadvantaged adult populations compared with US-based educationally advantaged populations.