Measuring Emotional Response: Comparing Alternative Approaches to Measurement

Abstract

Scholarly interest in the role of emotion in accounting for how people react to political figures, events, and messages has escalated over the past two plus decades in political science and psychology. However, research on the validity of the measurement of subjective self-report of emotional responses is rather limited. We introduce here a new measurement approach, a “slider” format and compare it with the long used “radio button” item format. We assess the reliability and validity of these two approaches to the measurement of affect. The study examines self-report measures of emotion to three generated news stories about terrorist threats. We report that both measurement formats are able to extract the expected threefold affect structure from a ten affect word battery. The slider format is, however, modestly more reliable, and more efficient in time to complete, has the ability to limit missing data, and generates continuous data that is less truncated than data derived from the radio button format. Finally, we report on three tests of construct validity. Both approaches exhibit equivalent results on two of those tests. However, the radio button format does poorly on one test of construct validity, that on the anticipated relationship between anxiety and interest in novel information. We present an assessment of two methods for measuring emotional reactions to stimuli such as political issues, political figures, or events. Both methods are suitable for use in online surveys or computer-driven experiments. The traditional method utilizes labeled “radio buttons” that enable a participant in a study to select by clicking on one of an array of typically five response options, ranging from lower to higher of some identified affect term (e.g., how angry one might feel). Second, the slider method offers a participant the ability to move an “arrow” up or down to indicate how much (up) or little (down) they feel. The goal of both measures is to ascertain the level of a targeted emotion, i.e., how little or how much, say anger. The slider method has been specifically developed to be used with participants using a computer. The slider approach falls within the category of visual analog scales. This method for measuring affective responses to stimuli of whatever sort has not hitherto been examined to determine its reliability and validity. The literature on the reliability and validity of these measurement strategies is thin and we found no studies including an explicit comparison.