Abstract
Historically, the implementation of research-based assessments (RBAs) has been a driver of educational change within physics and helped motivate adoption of interactive engagement pedagogies. Until recently, RBAs were given to students exclusively on paper and in-class; however, this approach has important drawbacks including decentralized data collection and the need to sacrifice class time. Recently, some RBAs have been moved to online platforms to address these limitations. Yet, online RBAs present new concerns such as student participation rates, test security, and students' use of outside resources. Here, we report on a study addressing these concerns in both upper-division and lower-division undergraduate physics courses. We gave RBAs to courses at five institutions; the RBAs were hosted online and featured embedded JavaScript code which collected information on students' behaviors (e.g., copying text, printing). With these data, we examine the prevalence of these behaviors, and their correlation with students' scores, to determine if online and paper-based RBAs are comparable. We find that browser loss of focus is the most common online behavior while copying and printing events were rarer. We found that correlations between these behaviors and student performance varied significantly between introductory and upper-division student populations, particularly with respect to the impact of students copying text in order to utilize internet resources. However, while the majority of students engaged in one or more of the targeted online behaviors, we found that, for our sample, none of these behaviors resulted in a significant change in the population's average performance that would threaten our ability to interpret this performance or compare it to paper-based implementations of the RBA.
Highlights
AND MOTIVATIONResearch-based assessments (RBAs) have become a cornerstone of physics education research (PER) due in large part to their ability to provide a standardized measure of students’ learning that can be compared across different learning environments or curricula [1]
We examine data on students’ behaviors on online RBAs to determine how prevalent specific online behaviors were for this population of students
We examine correlations between these behaviors and students’ performance on the assessments overall
Summary
AND MOTIVATIONResearch-based assessments (RBAs) have become a cornerstone of physics education research (PER) due in large part to their ability to provide a standardized measure of students’ learning that can be compared across different learning environments or curricula [1]. Significant prior work has been done to address some of the concerns around online conceptual assessment as part of the Learning Assistant Student Supported Outcomes (LASSO) study They have investigated concerns about changes in scores and participation rates between online and paper-based administrations of the Force Concept Inventory (FCI) [11] and Conceptual Survey of Electricity and Magnetism (CSEM) [12] in the context of introductory courses. The best practices identified in the LASSO study include multiple email and in-class reminders, and offering participation credit for completing both the pretests and post-tests They found that, when participation rates were similar, students’ overall performance was statistically comparable [15]
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
