Computerized Assessment Of Response Bias Research Articles

Estimating the accuracy of neurocognitive effort measures in the absence of a “gold standard”.

Psychologists frequently use symptom validity tests (SVTs) to help determine whether evaluees' test performance or reported symptoms accurately represent their true functioning and capability. Most studies evaluating the accuracy of SVTs have used either known-group comparisons or simulation designs, but these approaches have well-known limitations (potential misclassifications or lack of ecological validity). This study uses latent class modeling (LCM) implemented in a Bayesian framework to estimate SVT classification accuracy based on data obtained from real-life forensic evaluations. We obtained archival data from 1,301 outpatient evaluees who underwent testing with the Computerized Assessment of Response Bias (CARB), the Test of Memory Malingering (TOMM), and the Word Memory Test (WMT) in a forensic evaluation context. Under various data models, Markov chain Monte Carlo methods implemented via WinBUGS converged to target distributions that permitted Bayesian estimates of SVT accuracy. Under the most plausible model (conditional dependence in test results), classification accuracies (expressed as area under the "trapezoidal" receiver operating characteristic curve ± standard deviation) were as follows: CARB = 0.765 ± 0.030, WMT = 0.929 ± 0.020, and TOMM = 0.771 ± 0.034. At decision thresholds that hold false positive rates at 0.02, the SVTs would detect invalid responses (true positives) at rates of approximately 35%, 65%, and 49%, respectively, for the 3 tests. Though LCM methods have limitations, this study suggests that they offer an approach to SVT evaluation that avoids methodological pitfalls of known-group research designs while retaining ecological validity that is absent in simulation studies.

Development and Validation of a Response Bias Scale (RBS) for the MMPI-2

This study describes the development of a Minnesota Multiphasic Personality Inventory (MMPI-2) scale designed to detect negative response bias in forensic neuropsychological or disability assessment settings. The Response Bias Scale (RBS) consists of 28 MMPI-2 items that discriminated between persons who passed or failed the Word Memory Test (WMT), Computerized Assessment of Response Bias (CARB), and/or Test of Memory Malingering (TOMM) in a sample of 1,212 nonhead-injury disability claimants. Incremental validity of the RBS was evaluated by comparing its ability to detect poor performance on four separate symptom validity tests with that of the F and F(P) scales and the Fake Bad Scale (FBS). The RBS consistently outperformed F, F(P), and FBS. Study results suggest that the RBS may be a useful addition to existing MMPI-2 validity scales and indices in detecting symptom complaints predominantly associated with cognitive response bias and overreporting in forensic neuropsychological and disability assessment settings.

Age Related Effects in Children Taking the Computerized Assessment of Response Bias and Word Memory Test

The assessment of effort is a fundamental component of test performance analysis, since effort determines whether a psychological evaluation is valid. The assessment of effort in children has proven problematic. This may be related to the variable and inconsistent nature of children’s developing self-regulatory systems, and the fact that measures commonly used to assess effort were standardized on adults. If one uses effort measures designed for adults to assess children, then one must presume that the maintenance of effort in children is comparable to the same behavior in adults. However, because children’s executive functioning, including their abilities to self-regulate, attend, concentrate, and to engage in various cognitive activities improve with time (Barkley, 1997, pp. 209-234), our hypothesis is that young children’s effort regulation is dissimilar to that of adults, and the presumption of similarity is implausible. The purpose of this study was to determine whether age is a significant influence upon young children’s performances on the Computerized Assessment of Response Bias (CARB) and Word Memory Test (WMT). Statistical analysis suggests that younger children (those under 10 years of age) tended to produce poorer performance on these instruments. Younger children’s scores differed significantly from children ages 10 and older. Children 11 years and older produced CARB and WMT results similar to adult participants, suggesting a viability for adult normative comparisons with children in this age range. The current investigation concluded that children’s maintenance of effort appears to be significantly related to age and reading ability level. Consequently, the use of current adult-based norms with the CARB and WMT, without regard for a child’s developmental status and other contextual factors such as the child’s ability to read, appears ill-advised especially with children under 11 years of age.

Computerized Assessment of Response Bias in Forensic Neuropsychology

SUMMARY The Computerized Assessment of Response Bias (CARB) is a forced-choice digit recognition test designed to detect suboptimal performance during a psychological or neuropsychological evaluation. First developed in 1992, CARB has incorporated a number of refinements such as early termination for excellent early performance and analysis of response times and patterns. Patients with moderate or severe brain injury, or neurological disease, easily pass CARB. These groups provide primary validation for the test, along with more recent analogue malingering studies. CARB is sensitive to poor effort or suboptimal performance in patients with a wide variety of diagnoses, including outpatients with mild traumatic brain injuries, pain disorders, fibromyalgia, and depression. When used properly, CARB will likely hold up under Daubert challenges due to its specific focus and purpose, and extensive validation research.

Validation of the Computerized Assessment of Response Bias in Litigating Patients with Head Injuries

The detection of malingering in neuropsychological assessment requires valid measures of incomplete effort. The Computerized Assessment of Response Bias (CARB) is a digit-recognition procedure desiged to detect poor effort during a neuropsychological evaluation. We examined the CARB performance of a consecutive series of litigating patients (N = 119) with a full range of head-injury severity, from trivial to severe. Patients with trivial or mild head injuries performed more poorly than did patients with frank brain injuries. Specifically, patients with moderate to severe traumatic brain injuries obtained higher total scores and demonstrated briefer response latencies than did patients with trivial or mild head injuries.

Effects of injury severity and cognitive exaggeration on olfactory deficits in head injury compensation claims

The purpose of this study was to examine the relationship between exaggeration and scores on a test of olfactory discrimination in patients being assessed in connection with a claim for financial benefits. Participants were 448 patients referred to a private practice in Edmonton, Alberta, Canada for psychological or neuropsychological assessment, related to evaluation of impairment and disability resulting from a work-related or non-work related accident. All patients were involved in some form of compensation claim at the time of their evaluation. All patients completed two tests designed to detect exaggerated cognitive deficits, the Computerized Assessment of Response Bias (CARB) and the Word Memory Test (WMT) as part of their evaluation. The diagnostic groups included 322 head injury cases, varying from very minor to very severe. Normative data for the smell test were derived from 126 patients with orthopedic injuries who passed both the CARB and the WMT. Patients with more severe traumatic brain injuries were 10-12 times more likely to have olfactory deficits than persons with trivial to mild head injuries. In a subgroup of patients who failed either the CARB or the WMT, there was no relationship between injury severity and total scores on the smell test. Therefore, the dose-response relationship between brain injury severity and olfactory deficits is severely attenuated when patients who are probably exaggerating their cognitive deficits are included in the analyses. Those patients with trivial to mild head injuries who demonstrated adequate effort on both the CARB and the WMT were no more likely to show olfactory deficits than the non-head-injured orthopedic control subjects. Therefore, anosmia following mild traumatic brain injury should not be concluded from self-reports or from tests of smell unless tests of effort have been passed. Effort should also be controlled in group studies of olfaction.

Can Malingering Be Identified With the Judgment of Line Orientation Test?

The purpose of this study was to evaluate recently proposed (Meyers, Galinsky, & Volbrecht, 1999) cutoff scores for biased responding on the Judgment of Line Orientation Test (JLO). A large sample of individuals involved in head injury litigation (N = 294) took the JLO and 2 tests designed to detect biased responding, the Computerized Assessment of Response Bias (CARB) and the Word Memory Test (WMT), as part of a comprehensive neuropsychological evaluation. Patients were divided into groups on the basis of brain injury severity and whether or not they scored in the suspicious range on the CARB or WMT. The patients who were identified as providing biased responding on the CARB or WMT also scored significantly lower on the JLO. However, the Meyers et al. (1999) cutoff score correctly identified only 9.9% of this group, with a 1% possible false-positive rate. A different cutoff score was selected that had .22 sensitivity and .96 specificity. Overall, these results suggest that the JLO has limited utility as a screen for biased responding; however, clinicians are encouraged to evaluate these scores carefully if they do not seem to make biological or psychometric sense.

REVIEW

ABSTRACT ABSTRACT Test Reviews: COMPUTERIZED ASSESSMENT OF RESPONSE BIAS (CARB), WORD MEMORY TEST (WMT), AND MEMORY COM PLAINTS INVENTORY iMCI). Reviewed by Timothy F. Wynkoop and Robert L. Denney.

California Verbal Learning Test Indicators of Suboptimal Performance in a Sample of Head-Injury Litigants

Cutoff scores suggested by Millis, Putnam, Adams, and Ricker (1995) for detecting suboptimal performance on indices from the California Verbal Learning Test (CVLT) were evaluated using data from 193 compensation-seeking participants. All participants claimed to have suffered a blow to the head in an accident causing subsequent deterioration in cognitive function. The participants were divided into those with negligible or possible mild brain injuries and those with clear evidence of moderate to severe brain injuries. In addition to the CVLT, all participants were administered the Computerized Assessment of Response Bias (CARB), a two-alternative forced choice test of recognition memory that is used to detect feigned cognitive impairment. For all CVLT indices, the distributions of outcome (valid vs. suboptimal performance) was unrelated to age and brain injury severity, and only weakly associated with education. However, a significantly higher proportion of males than females obtained scores in the suboptimal performance range. The CVLT indices were not fully redundant with each other with respect to binary participant classifications; substantial disagreement between pairwise classifications was found among those participants who obtained at least one score in the suboptimal performance range. CVLT index classifications were also found to be non-redundant with classifications based on CARB scores. The CVLT may thus add useful data over and above that obtained from symptom validity testing. However, the data suggest that the use of the strategy where any one or more below-cutoff CVLT scores are considered a positive indicator of suboptimal performance may be associated with a higher than acceptable false-positive error rate.