Motion Conceptual Evaluation Research Articles

Performance Assessment of the First Year Engineering Students using Force and Motion Conceptual Evaluation (FMCE)

The admission of non-STEM graduates to engineering programs creates an opportunity to determine if the Senior High School strand does really dictate the performance of students. This study administered the Force and Motion Conceptual Evaluation (FMCE) to the first-year engineering students to determine if STEM graduates perform better than non-STEM graduates. The FMCE was administered to six hundred-seven (n = 607) first year engineering students of a state-run university in the Philippines, school year 2019-2020 and 2020-2021. The results reveal that there is no significant difference between the mean scores obtained by the STEM and non-STEM graduates (p = 0.912) in the FMCE. It was noted that the students who graduated from private and public high schools have similar performance (p = 0.242). Nevertheless, there is an urban-rural gap in performance among the respondents, where students who graduated from schools in cities have better performance than those who graduated from rural schools (p = 0.019). Finally, in the field dominated by male, the results suggest that female students are at par with male student students (p = 0.123) and both have statistically the same level of confidence with their answers (p = 0.176).

Impact of Inquiry-Based Learning Using the 5E Model on Teachers’ Practices and Learners’ Achievement in Force and Motion in Secondary Schools of Jinja District, Uganda

This study aimed to assess the impact of Inquiry-Based Learning using the 5E model on teacher’s practices and students’ achievement in Force and Motion in Secondary Schools in Jinja District. The study employed a quantitative research method and a pretest-posttest non-equivalent quasi-experimental research design. The study sample was one hundred forty-one students and teachers who were sampled through simple random sampling procedures. Four teachers were trained in implementing the IBL in the classroom and they taught the experimental classes while two teachers were not trained and they taught the control classes with the traditional teaching method. Reformed Teaching Observation Protocol (RTOP) was used to collect observation data while Force and Motion Conceptual Evaluation was used to test students' performance and conceptual understanding of mechanics-related topics. Data from student achievement tests were analysed under SPSS using the independent samples t-test while data from classroom observation was analysed by the MS Excel 2016. The study concludes that IBL has the potential to improve teachers’ classroom practices as well as students’ achievement in physics, especially if the teachers are given training on IBL. The study recommends that additional support in form of training be given to teachers to help them adapt to IBL. Also, teacher education programs ought to provide IBL instruction and training to educators.

Exploring the pre-instruction gender gap in physics.

There is a substantial body of work in physics education looking at gender disparities in physics. Recent work has linked gender disparities in college physics course performance to disparities in high school physics preparation, but to our knowledge, the origin of the disparity in high school physics preparation is still underexplored. In a select sample, we found that women on average had lower force and motion conceptual evaluation (FMCE) pre-scores (the FMCE is a short conceptual assessment of Newton's laws), and FMCE pre-score entirely mediated the effects of high school preparation and social-psychological factors on exam performance. The gender gap in FMCE pre-scores could not be explained by differences in the number of physics courses taken in high school. Instead, we find that the gender gap in the FMCE is partially explained by female students' higher levels of general test anxiety. We hypothesize that the format of the FMCE, a timed assessment, triggers stereotype threat in female students despite being a low-stakes assessment. Therefore, instructors and researchers should take care in interpreting the results of such concept inventory scores and should re-think the way they assess understanding of physics concepts. Results of this work aligned with previous findings on gender disparity in timed exams call upon investigating gender equitable assessment formats for evaluating physics knowledge to replace timed assessments, either high or low stakes.

Replicating analyses of item response curves using data from the Force and Motion Conceptual Evaluation

Ishimoto, Davenport, and Wittmann have previously reported analyses of data from student responses to the Force and Motion Conceptual Evaluation (FMCE), in which they used item response curves (IRCs) to make claims about American and Japanese students' relative likelihood to choose certain incorrect responses to some questions. We have used an independent data set of over 6,500 American students' responses to the FMCE to generate IRCs to test their claims. Converting the IRCs to vectors, we used dot product analysis to compare each response item quantitatively. For most questions, our analyses are consistent with Ishimoto, Davenport, and Wittmann, with some results suggesting more minor differences between American and Japanese students than previously reported. We also highlight the pedagogical advantages of using IRCs to determine the differences in response patterns for different populations to better understand student thinking prior to instruction.

Detecting the influence of item chaining on student responses to the Force Concept Inventory and the Force and Motion Conceptual Evaluation

The assumption of local item independence is not valid for many items on the Force Concept Inventory and the Force and Motion Conceptual Evaluation.

Retention of conceptual learning after an interactive introductory mechanics course

The cyclic format of the undergraduate physics curriculum depends on students' ability to recall and utilize material covered in prior courses in order to reliably build on that knowledge in later courses. However, there is evidence to suggest that people often do not retain all, or even most, of what they learned previously. How much information is retained appears to be dependent both on the individuals' approach to learning as well as the style of instruction. In particular, there is evidence to suggest that active engagement techniques in the classroom can improve students' retention of the material over time. Here, we report the findings of a longitudinal investigation of students' retention of conceptual understanding as measured by the Force and Motion Conceptual Evaluation (FMCE) following a first-semester, calculus-based introductory physics course, which features significant active engagement in both lecture and recitation. By administering the FMCE at the end of a first-semester physics course and again at the beginning of the subsequent second-semester physics course, we examine students' knowledge retention over time periods ranging from 1-15 months. We find that the shift in students' FMCE scores between these two courses is positive but corresponds to a small effect size, indicating that students retained effectively all of their conceptual learning (as measured by the FMCE). This finding largely persists even as the length of the gap between the two courses increases. We also find that, when breaking out students' performance on individual questions, the majority of students maintain their score on individual questions. Averaged over all questions, roughly a fifth of the students switched their answers from right to wrong or wrong to right on any given item.

Exploring the structure of misconceptions in the Force and Motion Conceptual Evaluation with modified module analysis

Investigating student learning and understanding of conceptual physics is a primary research area within Physics Education Research (PER). Multiple quantitative methods have been employed to analyze commonly used mechanics conceptual inventories: the Force Concept Inventory (FCI) and the Force and Motion Conceptual Evaluation (FMCE). Recently, researchers have applied network analytic techniques to explore the structure of the incorrect responses to the FCI identifying communities of incorrect responses which could be mapped on to common misconceptions. In this study, the method used to analyze the FCI, Modified Module Analysis (MMA), was applied to a large sample of FMCE pretest and post-test responses ($N_{pre}=3956$, $N_{post}=3719$). The communities of incorrect responses identified were consistent with the item groups described in previous works. As in the work with the FCI, the network was simplified by only retaining nodes selected by a substantial number of students. Retaining nodes selected by 20\% of the students produced communities associated with only four misconceptions. The incorrect response communities identified for men and women were substantially different, as was the change in these communities from pretest to post-test. The 20% threshold was far more restrictive than the 4% threshold applied to the FCI in the prior work which generated similar structures. Retaining nodes selected by 5% or 10% of students generated a large number of complex communities. The communities identified at the 10\% threshold were generally associated with common misconceptions producing a far richer set of incorrect communities than the FCI; this may indicate that the FMCE is a superior instrument for characterizing the breadth of student misconceptions about Newtonian mechanics.

Quantitatively ranking incorrect responses to multiple-choice questions using item response theory

Research-based assessment instruments (RBAIs) are ubiquitous throughout both physics instruction and physics education research. The vast majority of analyses involving student responses to RBAI questions have focused on whether or not a student selects correct answers and using correctness to measure growth. This approach often undervalues the rich information that may be obtained by examining students' particular choices of incorrect answers. In the present study, we aim to reveal some of this valuable information by quantitatively determining the relative correctness of various incorrect responses. To accomplish this, we propose an assumption that allow us to define relative correctness: students who have a high understanding of Newtonian physics are likely to answer more questions correctly and also more likely to choose better incorrect responses, than students who have a low understanding. Analyses using item response theory align with this assumption, and Bock's nominal response model allows us to uniquely rank each incorrect response. We present results from over 7,000 students' responses to the Force and Motion Conceptual Evaluation.

Multidimensional item response theory and the force and motion conceptual evaluation

Many studies have examined the structure and properties of the Force Concept Inventory (FCI); however, far less research has investigated the Force and Motion Conceptual Evaluation (FMCE). This study applied Multidimensional Item Response Theory (MIRT) to a sample of N=4528 FMCE post-test responses. Exploratory factor analysis showed that 5, 9, and 10-factor models optimized some fit statistics. The FMCE uses extensive blocking of items into groups with a common stem; these blocks factored together in most models. A confirmatory analysis, which constrained the MIRT models to a theoretical model constructed from expert solutions, produced a model requiring only 8 principles, fundamental reasoning steps. This was substantially fewer than the 19 principles identified in the FCI by a previous study. Correlation analysis also demonstrated that the two instruments were very dissimilar. The reduced number of principles and the repetition of items using a single principle allowed the extraction of eight single-principle subscales, seven with Cronbach’s alpha greater than the 0.7 required for acceptable internal consistency. The differences between the FCI and the FMCE suggest that the two instruments could provide complementary, but different, information about student understanding of Newton’s laws with the FCI measuring an integrated Newtonian force concept and the FMCE measuring details of that force concept.Received 6 June 2019DOI:https://doi.org/10.1103/PhysRevPhysEducRes.15.020141Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI.Published by the American Physical SocietyPhysics Subject Headings (PhySH)Research AreasAssessmentConcepts & principlesInstructional materials developmentResearch methodologyPhysics Education Research

Adapting RealTime Physics for Distance Learning with the IOLab

The IOLab is a versatile and inexpensive data acquisition device in a cart that can roll on its three wheels. It has numerous sensors for a variety of physical quantities. We adapted RealTime Physics, Module 1: Mechanics active learning labs for use with the IOLab. We tested these labs both on campus and with distance learners at Portland State University and Chemeketa Community College for three years, consistently obtaining significant conceptual learning gains on the Force and Motion Conceptual Evaluation (FMCE). Student attitudes towards the labs, the device, and distance learning—as measured by post-course evaluations—were generally very positive.

Critical Thinking and Physics Concepts: Indicators for General Education Science Course Assessment

Abstract Demonstrating knowledge of major concepts and critical thinking are common learning outcomes for general education courses, especially in the sciences. Although numerous valid and reliable measures of conceptual understanding exist, the determination of critical thinking level is often more difficult, subjective, and domain-specific. This study investigates the relationship between domain-specific critical thinking and the learning of force and motion concepts within laboratory activities in a first semester, non-calculus college physics course. Regression analyses were used while controlling for age, prior physics coursework, and the conceptual understanding pretest score. Conceptual understanding was measured using the valid and reliable Force and Motion Conceptual Evaluation (FMCE). Critical Thinking (CT) was determined using the Holistic Critical Thinking Scoring Rubric (HCTSR) and the Structure of the Observed Learning Outcomes (SOLO) Taxonomy. The results indicate that students' CT has a significant correlation with their learning of force and motion concepts. In other words, students who demonstrate a Newtonian view of force and motion also demonstrate higher levels of CT. Thus, FMCE post-test scores could be used as a proxy for students' CT measurements within a first semester physics course. Additionally, the preference of the SOLO Taxonomy is based on ease of use and estimates of inter-rater reliability.

Rasch Analysis of the Force and Motion Conceptual Evaluation Test: Validity and Reliability in Measuring Force and Motion Understanding of Students

Rasch Analysis of the Force and Motion Conceptual Evaluation Test: Validity and Reliability in Measuring Force and Motion Understanding of Students

Partitioning the gender gap in physics conceptual inventories: Force Concept Inventory, Force and Motion Conceptual Evaluation, and Conceptual Survey of Electricity and Magnetism

Very little of the gender gap on multiple choice conceptual inventories can be explained by differences in academic performance between men and women.

The Influence of Teaching Approach on Students' Conceptual Learning in Physics.

Physics is fundamental to secure future needs for scientific and technological competence (Angell et al., 2004), but many countries experience a drop in students’ performances in international assessments (Organisation for Economic Co-operation Development [OECD], 2018), as well as in rates of enrolment in undergraduate programs in scientific disciplines (STEM). Socio-constructivist theories have produced a reforming movement in several educational systems, in particular in the area of sciences, but teacher often consider them an idealistic view of education and do not consider themselves metacognitively competent enough to foster thinking in the classroom. In this study, we investigated the efficacy of different teaching methods on high-school students’ conceptual knowledge of physics, after the effect of science-related beliefs and critical thinking skills was controlled. We adopted a mixed-method with sequential design, in which quantitative and qualitative data flow are inter-mixed. In specific, we interviewed four high school physics teachers to identify teaching approaches (qualitative approach) and compared them in terms of efficacy on students’ performances (quantitative approach). Four teachers and 77 10th grade students participated. Teachers were interviewed during the school years and asked questions about their teaching experience, their teaching approach (Kang and Wallace, 2005) and their epistemic beliefs (Tsai, 2002). Students performances in Science-related beliefs (Conley et al., 2004), critical thinking (Cornell Critical Thinking Test Level X, Millman et al., 2005), and conceptual knowledge in physics (The Force and Motion Conceptual Evaluation, Ramlo, 2002) were evaluated twice, at the beginning and at the end of the school year. The independent-sample t-tests on pre-test variables did not reveal any statistically significant difference between groups. Results from the complex samples GLM revealed statistically significant differences on post-test scores in conceptual knowledge in physics, after the effect of covariates was controlled. Overall, the study contributes to our understanding on current teaching practices in school, and their effect on students’ conceptual understanding of physics concepts.

Item-level gender fairness in the Force and Motion Conceptual Evaluation and the Conceptual Survey of Electricity and Magnetism

An item-level analysis of the Force and Motion Conceptual Evaluation and Conceptual Survey of Electricity and Magnetism instruments shows that there are relatively few items that show gender unfairness.

Use of item response curves of the Force and Motion Conceptual Evaluation to compare Japanese and American students’ views on force and motion

Item response curves can be used as part of the validation process when extending multiple-choice instruments to new contexts, such as languages and nationalities.

Secondary analysis of teaching methods in introductory physics: A 50 k-student study

Physics education researchers have developed many evidence-based instructional strategies to enhance conceptual learning of students in introductory physics courses. These strategies have historically been tested using assessments such as the Force Concept Inventory (FCI) and the Force and Motion Conceptual Evaluation (FMCE). We have performed a review and analysis of FCI and FMCE data published between 1995 and 2014. We confirm previous findings that interactive engagement teaching techniques are significantly more likely to produce high student learning gains than traditional lecture-based instruction. We also establish that interactive engagement instruction works in many settings, including those with students having a high and low level of prior knowledge, at liberal arts and research universities, and enrolled in both small and large classes.

Experimentally Building a Qualitative Understanding of Newton's Second Law

Newton's second law is one of the cornerstones of the introductory physics curriculum, but it can still trouble a large number of students well after its introduction, hobbling their ability to apply the concept to problem solving1 and to related concepts, such as momentum, circular motion, and orbits. While there are several possibilities for lab activities addressing the functional relationship among net force, mass, and acceleration, the qualitative understanding of the connection between forces and acceleration can still be lacking,2 leading to poor performance in problem solving and in assessments such as the Force Concept Inventory3 and Force and Motion Conceptual Evaluation.4 There is a need for strong conceptual understanding of the relationships between net force and acceleration and between acceleration and velocity in order to effectively address common force-motion misconceptions;5 there is a large literature concerning student understanding of force and motion.6

Validating the Japanese translation of the Force and Motion Conceptual Evaluation and comparing performance levels of American and Japanese students

Validating the Japanese translation of the Force and Motion Conceptual Evaluation and comparing performance levels of American and Japanese students

Applying model analysis to a resource-based analysis of the Force and Motion Conceptual Evaluation

Model analysis provides more information than normalized gain for understanding class performance on multiple-choice conceptual evaluations.