Effects of Re-Using a Conceptual Examination Question in Physics

Abstract

We report on a study of what happened when we recycled a conceptual examination question in a first-year university physics course. The question, which was used for three consecutive years, asked about an astronaut's experience of weighing in an orbiting space-craft. Our original intention was to use a phenomenographic approach to look for differences in students' descriptive answers. Having done that, we decided to add a study of the marks that were awarded to those answers. The first time that the question was re-used, the distribution of answers amongst our phenomenographic categories showed a decrease in the common conception that gravity is zero in the satellite and an increase in explanations in terms of free fall. When the question was re-used a second time, that difference was maintained but it was not significantly increased. The distribution of marks for the question was different over the three years in a way that appears to be unrelated to differences in students' conceptual understandings. Differences in the distribution of marks are more likely to be related to differences in marking procedures. We conclude that studies like this one have the potential to contribute to improvements in university assessment procedures. In particular we propose that phenomenographic analysis could be used in the design of marking schemes.