Measures Of Response Bias Research Articles

Effects of Training with Knowledge of Results on Diagnosticity in Vigilance Performance

Making accurate diagnostic decisions about signal presence/absence is critical for success in many failure intolerant monitoring technologies requiring sustained attention or vigilance. This study examined the effects of training with knowledge of results (KR) on observer diagnosticity in a vigilance task. Diagnosticty was measured in terms of decision theory measures of positive predictive power (PPP) -precision in indicating when signals were actually present and negative predictive power (NPP) - precision when indicating signal absence. Initial training with KR enhanced observers' diagnosticity on a subsequent test task in terms of PPP but not NPP. The picture of performance efficiency reflected by both diagnostic measures differed from results indexed by signal detection theory (SDT) measures of perceptual sensitivity (d') and response bias (c). However as predicted from the computational mechanics of the decision theory and SDT measures, both diagnostic measures correlated positively with d' while NPP correlated negatively with c. These findings indicate that combinations of perceptual ability and level of responding can influence the behavioral metrics signifying diagnosticity in vigilance performance

Individual differences in perceptual sensitivity and response bias in anxiety: Evidence from emotional faces

We investigated the perception of emotional stimuli in anxious individuals and non-anxious cohorts. Signal detection theory analysis was applied to the discrimination of emotionally charged faces at several points along a continuum of emotional intensity. This design permitted the derivation of multiple measures of sensitivity and response bias for fearful and for happy faces. Anxious individuals lacked a conservative bias in judging fearful stimuli and a liberal bias in judging positive stimuli compared with non-anxious individuals. In addition, anxious participants had lower perceptual sensitivity (d′) than non-anxious participants for mildly threatening stimuli, as well as a trend towards lower perceptual sensitivity for moderately positive stimuli. These results suggest that the processing of threat information in anxiety is affected by sensitivity and bias differently at different levels of affective intensity.

The Need For Continuous and Comprehensive Sampling of Effort/Response Bias During Neuropsychological Examinations

While most neuropsychologists are now administering measures of response bias in neuropsychological evaluations, it is still likely that detection of non-credible test performance is inadequate due to faulty assumptions regarding poor effort, namely that it remains constant across a battery of tests. Four cases are described that illustrate the variability in negative response bias that occurs during neuropsychological evaluations; if effort had not been periodically sampled with heterogeneous types of effort indicators during these examinations, the suspect performance would not have been detected. These examples argue for both continuous and comprehensive sampling of effort, specifically that negative response bias be routinely monitored throughout neuropsychological evaluations, and that effort indicators involving differing cognitive abilities be employed to assess for feigning of selective deficits.

Predicting Mathematical Achievement and Mathematical Learning Disability with a Simple Screening Tool

The Number Sets Test was developed to assess the speed and accuracy with which children can identify and process quantities represented by Arabic numerals and object sets. The utility of this test for predicting mathematics achievement and risk for mathematical learning disability (MLD) was assessed for a sample of 223 children. A signal detection analysis of first grade Number Sets Test scores provided measures of children's sensitivity to number and their response bias. The sensitivity measure, d', but not the response bias measure was predictive of third grade mathematics achievement scores, above and beyond the influence of intelligence, working memory, and first grade achievement scores. Further analyses assessed the sensitivity and specificity of the test and revealed that first grade d' scores identified 2 out of 3 children diagnosed as MLD in third grade and correctly identified about 9 out of 10 children who were not at risk for MLD.

Interpreting the Meaning of Multiple Symptom Validity Test Failure

While it is recommended that judgments regarding the credibility of test performance be based on the results of more than one effort indicator, and recent efforts have been made to improve interpretation of multiple effort test failure, the field currently lacks adequate guidelines for using multiple measures of effort in concert with one another. A total of 103 patients were referred for outpatient neuropsychological evaluation, which included multiple measures of negative response bias embedded in standard test batteries. Using any pairwise failure combination to predict diagnostic classification was superior (sensitivity = 83.8%, specificity = 93.9%, overall hit rate = 90.3%) to using any one test by itself and to using any three-test failure combination. Further, the results were comparable to the results of logistical regression analyses using the embedded indicators as continuous predictors. Given its parsimony and clinical utility, the pairwise failure model is therefore a recommended criterion for identifying non-credible performance; however, there are of course other important contextual factors and influences to consider, which are also discussed.

Cross-racial lineup identification: assessing the potential benefits of context reinstatement

The current research examined the potential benefit of context reinstatement on the cross-race effect in lineup identification. Participants viewed a series of own- and other-race faces and subsequently attempted identification of these faces from target-present and target-absent lineups. The traditional cross-race effect was found on measures of discrimination accuracy and response bias; however, discrimination accuracy across own- and other-race faces was shown to interact with context reinstatement such that only own-race faces benefited from the provision of contextual information. This finding is discussed in light of encoding-based theories of the cross-race effect, and with regard to the theoretical and practical limitations of mitigating the phenomenon at the time of identification.

Interpretation of Emotionally Ambiguous Faces in Older Adults

Research suggests that there is an age-related decline in the processing of negative emotional information, which may contribute to the reported decline in emotional problems in older people. We used a signal detection approach to investigate the effect of normal aging on the interpretation of ambiguous emotional facial expressions. High-functioning older and younger adults indicated which emotion they perceived when presented with morphed faces containing a 60% to 40% blend of two emotions (mixtures of happy, sad, or angry faces). They also completed measures of mood, perceptual ability, and cognitive functioning. Older and younger adults did not differ significantly in their ability to discriminate between positive and negative emotions. Response-bias measures indicated that older adults were significantly less likely than younger adults to report the presence of anger in angry-happy face blends. Results are discussed in relation to other research into age-related effects on emotion processing.

Relationship between Tomm performance and PAI validity scales in a mixed clinical sample

This study addressed the relationship between Personality Assessment Inventory (PAI) validity indicators and cognitive effort measures on the Test of Memory Malingering (TOMM). Significant correlations were found between TOMM and some PAI validity scales. Factor analysis results found separate cognitive and personality components, but the Negative Impression Management (NIM) scale, a measure of response bias, had factor loadings on both the cognitive and the personality components. Follow-up hierarchical multiple regression and t-test analysis generally confirmed this result, and found that NIM and the Infrequency (INF) scale have significant relationships with the TOMM. The results indicate that individuals with elevations on the PAI's INF and NIM scales often display decreased cognitive effort on the TOMM. The current results support the hypothesis that personality assessment validity indicators have a modest but significant relationship with poor cognitive effort.

Differential sensitivity of the Response Bias Scale (RBS) and MMPI-2 validity scales to memory complaints

The MMPI-2 Response Bias Scale (RBS) is designed to detect response bias in forensic neuropsychological and disability assessment settings. Validation studies have demonstrated that the scale is sensitive to cognitive response bias as determined by failure on the Word Memory Test (WMT) and other symptom validity tests. Exaggerated memory complaints are a common feature of cognitive response bias. The present study was undertaken to determine the extent to which the RBS is sensitive to memory complaints and how it compares in this regard to other MMPI-2 validity scales and indices. This archival study used MMPI-2 and Memory Complaints Inventory (MCI) data from 1550 consecutive non-head-injury disability-related referrals to the first author's private practice. ANOVA results indicated significant increases in memory complaints across increasing RBS score ranges with large effect sizes. Regression analyses indicated that the RBS was a better predictor of the mean memory complaints score than the F, FB, and FP validity scales and the FBS. There was no correlation between the RBS and the CVLT, an objective measure of verbal memory. These findings suggest that elevated scores on the RBS are associated with over-reporting of memory problems, which provides further external validation of the RBS as a sensitive measure of cognitive response bias. Interpretive guidelines for the RBS are provided.

Hearing a voice in the noise: auditory hallucinations and speech perception

It has recently been suggested that auditory hallucinations are the result of a criterion shift when deciding whether or not a meaningful signal has emerged. The approach proposes that a liberal criterion may result in increased false-positive identifications, without additional perceptual deficit. To test this hypothesis, we devised a speech discrimination task and used signal detection theory (SDT) to investigate the underlying cognitive mechanisms. Schizophrenia patients with and without auditory hallucinations and a healthy control group completed a speech discrimination task. They had to decide whether a particular spoken word was identical to a previously presented speech stimulus, embedded in noise. SDT was used on the accuracy data to calculate a measure of perceptual sensitivity (Az) and a measure of response bias (beta). Thresholds for the perception of simple tones were determined. Compared to healthy controls, perceptual thresholds were higher and perceptual sensitivity in the speech task was lower in both patient groups. However, hallucinating patients showed increased sensitivity to speech stimuli compared to non-hallucinating patients. In addition, we found some evidence of a positive response bias in hallucinating patients, indicating a tendency to readily accept that a certain stimulus had been presented. Within the context of schizophrenia, patients with auditory hallucinations show enhanced sensitivity to speech stimuli, combined with a liberal criterion for deciding that a perceived event is an actual stimulus.

Symptom Exaggeration by College Adults in Attention-Deficit Hyperactivity Disorder and Learning Disorder Assessments

To test the hypothesis that sub-optimal effort detected by one popular symptom validity measure, the Word Memory Test (WMT), should be interpreted as symptom exaggeration, the authors examined attention-deficit hyperactivity disorder (ADHD) and learning disorder (LD) assessment data collected from healthy adult patients over the past four years at one mid-size Southeastern college. They conducted six tests of this hypothesis, drawing upon extant research. Rates of apparent symptom exaggeration comparable to those found in medicolegal settings (e.g., personal injury cases), particularly in the context of ADHD evaluations, were found. WMT scores were positively correlated with intellectual and neurocognitive test scores, and negatively correlated with self-report symptom inventory scores. Measures of negative response bias embedded in one common self-report measure of psychopathology (the Personality Assessment Inventory) were not correlated with WMT performance. Unattended WMT administrations led to somewhat higher failure rates than were found when the examiners were present in the room during all phases of the test's administration. In light of considerable secondary gain motives in this population, the authors conclude that poor effort as evidenced by low WMT scores implies symptom exaggeration and not other factors in these assessments. The routine inclusion of empirically supported symptom validity measures in these evaluations is recommended, and future research directions are suggested.

Über die Validität von Persönlichkeitsprofilen in der nervenärztlichen Begutachtung

Although symptom validity tests have become available to German test users during the last few years, there is very little research into measures of negative response bias in personality assessment. The present study investigates the effects of negative response bias as measured by the Word Memory Test (WMT) and the Structured Inventory of Malingered Symptomatology (SIMS) on self-report personality scales. A retrospective analysis was performed on data from 93 patients who had undergone neuropsychiatric assessment in the context of independent medical examination. Complete data sets were available for the WMT, the SIMS, and the Freiburg Personality Inventory-Revised (FPI-R). Significant differences were found for a number of personality scales, depending on WMT and SIMS classification. The FPI-R validity scale (Openness) was linked to neither WMT nor SIMS, whereas the results in the latter two instruments showed a significant overlap of classification results (phi=0.44). A principal axis analysis yielded corresponding results. It is concluded that self-report personality measures may be considerably distorted by negative response bias. FPI-R Openness scale scores do not allow any interpretation in terms of negative response bias. More effort should be directed in German-speaking countries towards the development and validation of appropriate validity scales.

A survey of neuropsychologists’ beliefs and practices with respect to the assessment of effort

The current study investigated neuropsychologists’ beliefs and practices with respect to assessing effort and malingering by surveying a sample of NAN professional members and fellows ( n = 712). The results from 188 (26.4%) returned surveys indicated that 57% of respondents frequently included measures of effort when conducting a neuropsychological evaluation. While a majority of respondents (52%) rarely or never provide a warning that effort indicators will be administered, 27% of respondents often or always provide such a warning. The five most frequently used measures of effort or response bias were the Test of Memory Malingering (TOMM), MMPI-2 F–K ratio, MMPI-2 FBS, Rey 15-item test, and the California Verbal Learning Test. However, the TOMM, Validity Indicator Profile, Word Memory Test, Victoria Symptom Validity Test, and the Computerized Assessment of Response Bias were rated as most accurate for detecting suboptimal effort. These results and other findings are presented and discussed.

Activation of Brain Attention Systems in Individuals with Symptoms of ADHD

Previous research investigating attention and impulse control in individuals with Attention-Deficit/Hyperactivity Disorder (ADHD) has largely ignored the symptomatic differences among the three subtypes of ADHD: ADHD-Inattentive Type, ADHD-Hyperactive/Impulsive Type, and ADHD-Combined Type. The present study examined attention and impulse control by focusing on these subtypes. Based on their self-reported symptoms of ADHD, participants belonged to one of four groups: ADHD-Inattentive, ADHD-Hyperactive/Impulsive, ADHD-Combined, and control. Cortical activity was recorded from participants during performance of a Go/NoGo task. The event-related potentials (ERP) measured at frontal and posterior sites discriminated between the control group and participants with symptoms of ADHD. The control group consistently exhibited a higher P3 amplitude than all the ADHD groups. The main difference occurred at the frontal site, indicating that individuals with ADHD symptoms have deficits in the anterior attentional system, which mediates signal detection. Behavioral measures of signal sensitivity revealed that the ADHD-Inattentive and the ADHD-Hyperactive/Impulsive groups had more difficulty with the attention-demanding Go/NoGo respond-to-target task, while behavioral measures of response bias indicated that the ADHD-Hyperactive/Impulsive and the ADHD-Combined groups responded more liberally in the inhibition-demanding Go/NoGo suppress-to-target task.

Crying Wolf: Rectifying Poverty Rates in Kosova

Kosova is one of the most impoverished areas in Europe. The approach, however, for calculating poverty rates adopted, in all good faith, by the international community, has resulted in unrealistically high overall poverty rates and questionable relative rates between ethnic groups, urban and rural areas, and among Kosova’s regions. Two reasons are proposed for this: questionnaire designs which encourage refusals or response bias and inappropriate measures of adult equivalencies. Employing data from a 1,000 observation household survey, the implications of alternative approaches for estimating poverty rates are explored. The results suggest that current poverty rate estimates are two to three times the actual rates. Moreover, these estimates overstate differences between segments of Kosova’s society. Of particular concern, current estimates exaggerate differences in poverty rates between ethnic Serbians and ethnic Albanians, thereby exacerbating tensions.

A More Parsimonious Approach to Estimating Signal Detection Theory Measures

Signal Detection Theory (SDT) has proven to be an effective tool for analyzing perceptual and decision-making performance in many different areas. The main contribution of SDT is its capability to separate the discrimination process from the decision process by distinguishing between measures of sensitivity and measures of response bias. However, the usefulness of the traditional SDT measures of sensitivity, d', and response bias, c, is questionable in applied settings where observers do not make decisions based on an underlying psychophysical continuum. The purpose of our research was to derive alternative measures of sensitivity and response bias, overcoming the limitations of the traditional SDT model. Results from a Monte Carlo simulation showed support for our more parsimonious model.

Evaluating Chronic Pain Impact among Patients in Primary Care: Further Validation of a Brief Assessment Instrument

To evaluate the psychometric properties of the Profile of Chronic Pain: Screen (PCP:S) in primary care. The PCP:S is a brief assessment device designed to gauge pain severity, interference, and emotional burden. Questionnaire survey with short-term reliability and validity follow-up. Two hundred and forty-four outpatients (52% female) with chronic pain recruited from five primary care settings completed the 15-item PCP:S at an initial testing session. Approximately 1 week later, two subgroups participated in a retest reliability and validity follow-up. The battery of follow-up questionnaires included a measure of social desirability response bias and several instruments designed to provide convergent validity for the PCP:S. Retest reliability, internal consistency, factor structure, and social desirability bias were all found to be acceptable. Likewise, preliminary evidence suggests that, by virtue of its correlation with established measures of pain adjustment, the PCP:S is a valid pain assessment device. Although the PCP:S was initially developed and validated using a national community sample of adults with chronic pain who provided data via telephone interviews, the present findings support its use as a brief and psychometrically sound paper-and-pencil measure for the assessment of three key components of chronic pain in primary care.

Associations between Hofstede’s Cultural Constructs and Social Desirability Response Bias

This paper examines the associations among social desirability response bias, cultural constructs and gender. The study includes the responses of 1537 students from 12 countries including Australia, Canada, China, Colombia, Ecuador, Hong Kong, Ireland, Japan, Nepal, South Africa, Spain, and the United States. The results of the analysis indicate that, on average, social desirability response bias decreases (increases) as a country’s Individualism (Uncertainty Avoidance) increases. The analysis also indicates that women scored significantly higher on Paulhus’ Image Management Subscale on an overall basis and for seven of the 12 country comparisons. This research serves as a caution when considering the research findings of prior international survey-based ethics research that do not include a direct measure of social desirability response bias. For example, the finding that women score higher on Paulhus’ measure of social desirability response bias calls into question prior research that does not control for social desirability response bias indicating women are more ethically sensitive than men.

Signal detection theory, the approach of choice: Model-based and distribution-free measures and evaluation

New and old methods of analyzing two-choice experiments with confidence ratings are evaluated. These include the theory of signal detectability (TSD), Luce's choice theory,nonparametric techniques based on areas under receiver-operating characteristic (ROC) functions, andmethods based on S' and omega, proposed by Balakrishnan and his colleagues. New methods for assessing the bias of a complete ROC function are proposed, together with an additional area-based measure of response bias. Area measures of both sensitivity and bias proved the most consistent. Response bias for a full ROC function was larger than bias at the cut point and also provided additional information. Participants showed voluntary control of bias for all measures except omega. Unequal variance versions of TSD and choice models gave similar fits to data, with the choice model closer to an equal variance version. Discrimination data from Balakrishnan (1999) formed the empirical test bed.

The MMPI-2 Fake Bad Scale: Concordance and Specificity of True and Estimated Scores

A number of recent studies have supported the use of the MMPI-2 Fake Bad Scale (FBS) as a measure of negative response bias, the scale at times demonstrating greater sensitivity to negative response bias than other MMPI-2 validity scales. However, clinicians may not always have access to True FBS (T-FBS) scores, such as when True-False answer sheets are unavailable or published research studies do not report FBS raw scores. Under these conditions, Larrabee (2003a) suggests a linear regression formula that provides estimated FBS (E-FBS) scores derived from weighted validity and clinical T-Scores. The present study intended to validate this regression formula of MMPI-2 E-FBS scores and demonstrate its specificity in a sample of non-litigating, clinically referred, medically intractable epilepsy patients. We predicted that the E-FBS scores would correlate highly (>.70) with the T-FBS scores, that the E-FBS would show comparable correlations with MMPI-2 validity and clinical scales relative to the T-FBS, and that the E-FBS would show an adequate ability to match T-FBS scores using a variety of previously suggested T-FBS raw score cutoffs. Overall, E-FBS scores correlated very highly with T-FBS scores (r = .78, p < .0001), though correlations were especially high for women (r = .85, p < .0001) compared to men (r = .62, p < .001). Thirty-one of 32 (96.9%) comparisons made between E-FBS/T-FBS correlates with other MMPI-2 scales were nonsignificant. When matching to T-FBS “high” and “low” scores, the E-FBS scores demonstrated the highest hit rate (92.5%) through use of Lees-Haley's (1992) revised cutoffs for men and women. These same cutoffs resulted in excellent overall specificity for both the T-FBS scores (92.5%) and E-FBS scores (90.6%). The authors conclude that the E-FBS represents an adequate estimate of T-FBS scores in the current epilepsy sample. Use of E-FBS scores may be especially useful when clinicians conduct the MMPI-2 short form, which does not include all of the 43 FBS items but does include enough items to compute each of the validity and clinical T-Scores. Future studies should examine E-FBS sensitivity in compensation-seekers with incomplete effort.