Visual comparison of two data sets: Do people use the means and the variability?

Robin S S Kramer,Caitlin G R Telfer,Alice Towler

doi:10.5964/jnc.v3i1.100

Abstract

In our everyday lives, we are required to make decisions based upon our statistical intuitions. Often, these involve the comparison of two groups, such as luxury versus family cars and their suitability. Research has shown that the mean difference affects judgements where two sets of data are compared, but the variability of the data has only a minor influence, if any at all. However, prior research has tended to present raw data as simple lists of values. Here, we investigated whether displaying data visually, in the form of parallel dot plots, would lead viewers to incorporate variability information. In Experiment 1, we asked a large sample of people to compare two fictional groups (children who drank ‘Brain Juice’ versus water) in a one-shot design, where only a single comparison was made. Our results confirmed that only the mean difference between the groups predicted subsequent judgements of how much they differed, in line with previous work using lists of numbers. In Experiment 2, we asked each participant to make multiple comparisons, with both the mean difference and the pooled standard deviation varying across data sets they were shown. Here, we found that both sources of information were correctly incorporated when making responses. Taken together, we suggest that increasing the salience of variability information, through manipulating this factor across items seen, encourages viewers to consider this in their judgements. Such findings may have useful applications for best practices when teaching difficult concepts like sampling variation.

Highlights

In our everyday lives, we are required to make decisions based upon our statistical intuitions
With a greater number of mean and standard deviation conditions, and longer lists of numbers, Saito (2015) found that participants correctly judged an increase in effect size for larger mean differences but perceived incorrectly that effect size increased as standard deviations increased
Researchers have shown that between-group variability influences decisions when displaying raw data but participants did not respond to changes in within-group variability when comparing groups (Masnick & Morris, 2008)

Summary

Introduction

We are required to make decisions based upon our statistical intuitions. We suggest that increasing the salience of variability information, through manipulating this factor across items seen, encourages viewers to consider this in their judgements Such findings may have useful applications for best practices when teaching difficult concepts like sampling variation. Recent research provides some evidence that both the mean differences and the set variances correctly influence decisions about which of two groups is larger when the data are presented as lists of raw values (Morris & Masnick, 2015). Obrecht, Chapman, and Gelman (2007) showed that participants gave little consideration to either the sample size or the standard deviation when presented with two lists of raw data Instead their judgements were primarily driven by the mean differences. Work with box plots as a presentation method has shown that using these in combination with in-depth instruction may facilitate students’ understanding of sampling variability, along with how to compare two sets of data (Bakker, Biehler, & Konold, 2005; Pfannkuch, 2006; Pfannkuch, Arnold, & Wild, 2015). Reading and Reid (2005, 2006) model the learning progression in students as a shift from thinking about only the means to a strong consideration of variation

Objectives

Methods

Results

Conclusion