Distinct Methods, Biased Nonetheless: Insufficient Effort Responding as a Potential Confound

Abstract

Following research that demonstrates insufficient effort responding (IER) may confound scores on survey measures (Huang, Liu, & Bowling, 2015), a question emerges as to whether and when IER can act as a confounding variable between objective tests (e.g., general cognitive ability) and survey measures (e.g., core self-evaluations). The present paper reports two studies that address this overall question. Study 1 (N = 243) demonstrated that (a) IER was negatively related to performance on objective tests, and (b) IER was related to survey measures depending on the means of these measures. As a result, IER could inflate an objective test’s associations with other object tests and survey measures. Using simulations, Study 2 examined the impact of two IER parameters – within-person consistency of IER and proportion of IER in the sample. When predicting the confounding effects of IER, within-person consistency had positive linear and quadratic effects, proportion of IER cases had a positive linear effect, and consistency and proportion had a positive interactive effect. In contrast, when predicting the attenuating effects of IER, consistency and proportion each had linear effect while their interactive effect strengthened each other.