Measuring Respondents’ Cognitive Load Related to Making Factual and Normative Judgments

Abstract

The use of various methods for measuring cognitive load and mental effort in recent years has become increasingly popular in various fields of social and affective neuroscience, in applied research on the comparative effectiveness of teaching methods and training platforms, in the study of the distribution of attention in solving various problems and using informational tips in decision making, etc. In this broader context, the specific request for a multimodal assessment of the cognitive load of interviewers and respondents and of its impact on the quality of the survey data, including the use of paradata and webcams for this purpose, has been also growing recently. We conducted a within-subject methodological experiment (N = 50) aiming at comparative measurement of task-evoked cognitive load of respondents related to two tasks of making factual and normative judgments. The first task implied making causal and blame judgments for two institutional domains (medical, work dress-code) using the similar factorial vignettes, while the second task presupposed making lay factual and normative-deontic judgments concerning migrant rights to free health care. We used in parallel two measures of task-evoked cognitive load — pupillometry (Pupil Lab glasses), and the Paas scale of mental effort. The results provide limited evidence in support of the difference that exists between ordinary judgments of cause, blame, and severity of harm in terms of their propensity to evoke psychosensory pupillary response and subjectively perceived mental effort, both reflecting the variability in the cognitive load imposed on survey respondent when performing a pertinent survey task. We also briefly discuss the evidence obtained in support of the task-specific difference in sensitivity and validity of neurophysiological and self-report-based measures of survey-related cognitive load. Acknowledgement. The research was supported by RSF (project number 22-28-00968, project title: “Eye-tracking data and pupillometry in multimodal measurement of the respondents' cognitive load”). Disclosure statement. No potential conflict of interest was reported by the authors.