Using ad hoc measures for response styles: a cautionary note

Abstract

This research note addresses the challenge of how to optimally measure acquiescence response style (ARS) and extreme response style (ERS). This is of crucial importance in assessing results from studies that have tried to identify antecedents of response styles (such as age, education level, national culture). Using survey data from the Netherlands, a comparison is made between the traditional method and a more recently proposed method of measuring ARS and ERS (i.e., the convergent validity across both methods is assessed). The traditional method is based on an ad hoc set of related items. The alternative method uses a set of randomly sampled items to optimize heterogeneity and representativeness of the items. It is found that the traditional method may lead to response style measures that are suboptimal for estimating levels of ARS and ERS as well as relations of ARS and ERS with other variables (like hypothesized antecedents). Recommendations on how to measure response styles are provided.