Retrospective questions: data quality, task difficulty, and the use of a checklist

Abstract

Data from a national panel study with waves in 1987 and 1991 (N=1257), present evidence of the low accuracy of responses to retrospective questions, concerning both attitudes and behaviour. Applying a split ballot design, it is investigated whether using a checklist improves the response accuracy for a retrospective question about one single event: how respondents (N=363) did obtain the job they had four years ago. Furthermore interaction effects of ‘task difficulty’ are examined. The response accuracy indeed increases by using a checklist; however, this increase is not statistically significant. The expected increase of the checklist effect with higher ‘task difficulty’ appears for longer recall intervals, but not for more frequent changes of jobs. It turned out that for male respondents all the predicted effects are indeed clearly present. But for female respondents the checklist appeared to have no effect, irrespective of the task difficulty. A tentative explanation for this unexpected gender effect is suggested by pointing at indications that the male and female respondents differed in the way they obtained their job.