Chapter 3 - Experimental design

Abstract

When designing an experiment, researchers need to determine the dependent and independent variables and how to measure or control those variables. There are three commonly adopted designs in human-computer interaction (HCI) studies: the between-group design, the within-group design, and the split-plot design. The between-group design is cleaner, avoids the learning effect, and is less likely to be affected by fatigue and frustration. But this design is weaker due to the high noise level of individual differences and usually requires larger number of participants. The within-group design effectively isolates individual differences and, therefore, is a much stronger test than the between-group design. Another advantage is that fewer participants are required. However within-group designs are more vulnerable to learning effects and fatigue. The appropriate design method needs to be selected based on the nature of the application, the participant, and the tasks examined in the experiment. Researchers should control potential bias in HCI studies through accurate and appropriate measurement devices and scales; clearly defined and detailed experimental procedures; carefully recruited participants; well-trained and professional experimenters; and well-controlled environments.