Abstract
AbstractCollection and analysis of self-reported information on an ordered Likert scale is ubiquitous across the social sciences. Inference from such analyses is valid where the response scale employed means the same thing to all individuals. That is, if there is no differential item functioning (DIF) present in the data. A priori this is unlikely to hold across all individuals and cohorts in any sample of data. For this reason, anchoring vignettes have been proposed as a way to correct for DIF when individuals self-assess their health (or well-being, or satisfaction levels, or disability levels, etc.) on an ordered categorical scale. Using an example of self-assessed pain, we illustrate the use of vignettes to adjust for DIF using the compound hierarchical ordered probit model (CHOPIT). The validity of this approach relies on the two underlying assumptions of response consistency (RC) and vignette equivalence (VE). Using a minor amendment to the specification of the standard CHOPIT model, we develop easy-to-implement score tests of the null hypothesis of RC and VE both separately and jointly. Monte Carlo simulations show that the tests have good size and power properties in finite samples. We illustrate the use of the tests by applying them to our empirical example. The tests should aid more robust analyses of self-reported survey outcomes collected alongside anchoring vignettes.
Highlights
It is common in social surveys to use subjective categorical scales to elicit information in the form of self-reports; for example, levels of health, work2 W
We show that the proposed modification is innocuous in terms of parameter estimates of the mean function of the (CHOPIT ) model which links characteristics of respondents to outcome on the latent scale
We refer to the Hierarchical Ordered Probit Model (HOPIT) model with vignettes as the compound hierarchical ordered probit model (CHOPIT ) model (e.g. Vonkova & Hullegie (2011), Paccagnella (2013), Van Soest & Vonkova (2014))
Summary
It is common in social surveys to use subjective categorical scales to elicit information in the form of self-reports; for example, levels of health, work. Anchoring vignettes have been proposed as a method to overcome DIF (King et al 2004) and have received wide attention in the applied literature - for example, in self-reported data on health status (Soloman et al 2004, Bago d’Uva et al 2008, Peracchi & Rossetti 2012, Grol-Prokopczyk et al 2011, Vonkova & Hullegie 2011); healthy behaviours (Van Soest et al 2011); satisfaction with health systems performance (Sirven et al 2012, Rice et al 2012); work disability (Kapteyn et al 2007, 2011, Angelini et al 2011, Paccagnella 2011); political efficacy (King et al 2004); job satisfaction (Kristensen & Johansson 2008); life satisfaction (Angelini et al 2014); satisfaction with income (Kapteyn et al 2013) and consumer satisfaction with products and services (Rossi et al 2001) Together with their own situation, respondents are asked to evaluate one or more vignettes describing situations of hypothetical individuals. As is typical with specification tests, it relies on standard parametric assumptions underlying the (CHOPIT ) model, that the model is correctly specified, with no omitted variables, endogeneity and so on
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callMore From: Journal of the Royal Statistical Society Series A: Statistics in Society
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
