Abstract
When considering the variety of questions that can be used to measure students’ learning, instructors may choose to use multiple-choice questions, which are easier to score than responses to open-ended questions. However, by design, analyses of multiple-choice responses cannot describe all of students’ understanding. One method that can be used to learn more about students’ learning is the analysis of the open-ended responses students’ provide when explaining their multiple-choice response. In this study, we examined the extent to which introductory astronomy students’ performance on multiple-choice questions was comparable to their ability to provide evidence when asked to respond to an open-ended question. We quantified students’ open-ended responses by developing rubrics that allowed us to score the amount of relevant evidence students’ provided. A minimum rubric score was determined for each question based on two astronomy educators perception of the minimum amount of evidence needed to substantiate a scientifically accurate multiple-choice response. The percentage of students meeting both criteria of (1) attaining the minimum rubric score and (2) selecting the correct multiple-choice response was examined at three different phases of instruction: directly before lab instruction, directly after lab instruction, and at the end of the semester. Results suggested that a greater proportion of students were able to choose the correct multiple-choice response than were able to provide responses that attained the minimum rubric score at both the post-lab and post-instruction phases.8 MoreReceived 1 November 2013DOI:https://doi.org/10.1103/PhysRevSTPER.10.020103This article is available under the terms of the Creative Commons Attribution 3.0 License. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI.Published by the American Physical Society
Highlights
The percentage of students meeting both criteria of (1) attaining the minimum rubric score and (2) selecting the correct multiple-choice response was examined at three different phases of instruction: directly before lab instruction, directly after lab instruction, and at the end of the semester
This study investigated whether or not introductory astronomy students could provide a sufficient amount of evidence in writing to support their correct multiple-choice responses on assessments that asked them to explain their reasoning for their multiple-choice response
We developed a minimum rubric score, unique to each assessment, to allow us to compare students’ ability to provide minimal lines of reasoning to their ability to provide a correct multiple-choice response
Summary
When instructors seek to assess their students’ knowledge, a variety of methods may be employed including multiple-choice questions, open-ended questions, projects, written reports, and presentations. We investigated the extent to which students could provide a minimal amount of evidence to warrant their scientifically accurate multiple-choice responses. To do this we quantified students’ open-ended responses based on rubrics we developed. A minimum rubric score was determined for each question based on two astronomy educators perception of the minimal evidence needed to substantiate a scientifically accurate multiple-choice response. The assessments and rubrics used for analysis are provided for instructors and researchers interested in either using our rubrics for classroom or research analysis or as an aid to developing their own rubrics for open-ended response analysis
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
More From: Physical Review Special Topics - Physics Education Research
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.