Comparing and validating measures of non-cognitive traits: Performance task measures and self-reports from a nationally representative internet panel

Abstract

Education researchers and labor economists are increasingly focusing on skills such as conscientiousness, grit and emotional stability. This is part of a larger research program that sees “non-cognitive skills” as drivers of educational attainment and labor market outcomes. However, data on these important non-cognitive traits is not always available and, when available, researchers have raised concerns on potential biases in self-reported measures. In this paper, we use data collected from the Understanding America Study, a nationally representative internet panel, to study the validity of measures of non-cognitive traits, including novel measures based on survey effort. We propose that surveys can be viewed as performance tasks and that respondents provide meaningful information about relevant non-cognitive traits based on the effort they put forward in completing them. In particular, we examine the extent to which respondents provide careless answers or intentionally skip questions. We compare self-reported measures of grit, conscientiousness, neuroticism, and other personality traits, with survey-effort measures. We study the relationship of survey effort measures to these self-reported measures, educational attainment, and labor-market outcomes. Our results show that measures of careless answering in surveys show promise for use as proxies of traits related to conscientiousness and neuroticism.