Abstract
Measuring student learning is a complicated but necessary task for understanding the effectiveness of instruction and issues of equity in college STEM courses. Our investigation focused on the implications on claims about student learning that result from choosing between one of two commonly used methods for analyzing shifts in concept inventories. The methods are: Hake's gain (g), which is the most common method used in physics education research and other discipline based education research fields, and Cohen's d, which is broadly used in education research and many other fields. Data for the analyses came from the Learning Assistant Supported Student Outcomes (LASSO) database and included test scores from 4,551 students on physics, chemistry, biology, and math concept inventories from 89 courses at 17 institutions from across the United States. We compared the two methods across all of the concept inventories. The results showed that the two methods led to different inferences about student learning and equity due to g being biased in favor of high pretest populations. Recommendations for the analysis and reporting of findings on student learning data are included.
Highlights
The methods for measuring change or growth and interpretations of results have been hotly discussed in the research literature for over 50 years [1]
As researchers commonly measure change and report gains and effects, it is incumbent on researchers to do so in the most accurate and informative manner possible
We collected data at scale and compared it to existing normative data to examine several statistical issues related to characterizing change or gain in student understanding
Summary
The methods for measuring change or growth and interpretations of results have been hotly discussed in the research literature for over 50 years [1]. The idea of measuring a single state (let alone change) in an individual’s understanding of a concept, conceptualized as a latent construct, is wrought with issues both philosophical and statistical [2]. Despite these unresolved issues, education researchers use measurement of growth for quantifying the effectiveness of interventions, treatments, and innovations in teaching and learning. We collected data at scale and compared it to existing normative data to examine several statistical issues related to characterizing change or gain in student understanding. Rise in the use of CIs to measure student knowledge
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
