Insufficient Effort Responding as a Potential Confound between Survey Measures and Objective Tests

Abstract

Following research that demonstrates insufficient effort responding (IER) may confound survey measures and inflate observed correlations (Huang, Liu, & Bowling, 2015c), a question emerges as to whether and when IER can act as a confound between objective tests and surveys. Using data (N = 243) originally designed to examine training and transfer, study 1 demonstrates that (a) IER is negatively related to performance on tests, and (b) IER’s influence on surveys depends on the sample means of these measures. As a result, IER could inflate a test’s association with other tests and surveys. Study 2 investigates the impact of two parameters—within-person consistency of IER and percentage of IER cases in the sample—by randomly replacing bootstrapped attentive responses (10,000 bootstrapped samples of 200 cases identified from study 1). When predicting the confounding effects of IER, within-person consistency has positive linear and quadratic effects, percentage of IER cases has a positive linear effect, and consistency and percentage have a positive interactive effect. Research and practical implications for the design and evaluation of surveys and tests are discussed.