Student Reaction to a Modified Force Concept Inventory: The Impact of Free-Response Questions and Feedback

Mark A J Parker,Sally Jordan,Nicholas Braithwaite,Holly Hedgeland

doi:10.30935/scimath/11882

Abstract

The study investigated student reaction to the alternative mechanics survey (AMS), a modified force concept inventory, which used automatically marked free-response questions and offered limited feedback to students after their answers had been submitted. Eight participants were observed in completing the AMS, and they were interviewed to gain insight into what had been observed; the resultant data set was analyzed by thematic analysis. This revealed six key themes: “use of free-response questions supported deep learning”, “interpretation of the AMS instructions affected answer length”, “the idea of being marked by a computer did not affect answer structure”, “participant reaction to the usability of the AMS was mostly positive”, “reactions to the AMS depended upon what participants thought it was for”, and “limited feedback was a useful addition to the AMS”. Participants gave answers of differing length, being guided by the question wording as well as by their own preferences. It was found that participants valued being given feedback on their performance. Participants reacted positively to the free-response questions and could see potential for the use of this question type, opening up possibilities for the use of automatically marked free-response questions in concept inventories in the future.

Highlights

Concept inventories have been used in science education for decades
Thematic analysis identified 17 codes, which grouped into 6 themes. These themes were “use of free-response questions supported deep learning”, “interpretation of the alternative mechanics survey (AMS) instructions affected answer length”, “the idea of being marked by a computer did not affect answer structure”, “participant reaction to the usability of the AMS was mostly positive”, “reactions to the AMS depended upon what participants thought it was for”, and “limited feedback was a useful addition to the AMS”
The “limited feedback was a useful addition to the AMS” theme was coded 183 times overall, making it the theme that was coded most frequently

Summary

Introduction

Concept inventories have been used in science education for decades The objective of this use varies (Smith & Tanner, 2010), but most are designed with the notion of testing different teaching approaches by measuring the learning gain (Porter et al, 2014). This is done by having students attempt the concept inventory as a pre-test before instruction on a topic, and have them attempt the same concept inventory again as a post-test after instruction on the topic. The first concept inventory was the force concept inventory (FCI) (Hestenes et al, 1992) This contains thirty multiple-choice questions and was designed to test understanding of the physics concept of Newtonian mechanics. The FCI has been widely used and the subject of much discussion within the research community (Eaton, 2021; Lasry et al, 2011; Yasuda et al, 2021)

Objectives

Methods

Results

Discussion

Conclusion