Force Concept Inventory Research Articles

Assessing AI’s problem solving in physics: Analyzing reasoning, false positives and negatives through the force concept inventory

Assessing AI’s problem solving in physics: Analyzing reasoning, false positives and negatives through the force concept inventory

ChatGPT's quality: Reliability and validity of concept inventory items.

The recent advances of large language models (LLMs) have opened a wide range of opportunities, but at the same time, they pose numerous challenges and questions that research needs to answer. One of the main challenges are the quality and correctness of the output of LLMs as well as the overreliance of students on the output without critically reflecting on it. This poses the question of the quality of the output of LLMs in educational tasks and what students and teachers need to consider when using LLMs for creating educational items. In this work, we focus on the quality and characteristics of conceptual items developed using ChatGPT without user-generated improvements. For this purpose, we optimized prompts and created 30 conceptual items in kinematics, which is a standard topic in high-school level physics. The items were rated by two independent experts. Those 15 items that received the highest rating were included in a conceptual survey. The dimensions were designed to align with the ones in the most commonly used concept inventory, the Force Concept Inventory (FCI). We administered the designed items together with the FCI to 172 first-year university students. The results show that ChatGPT items have a medium difficulty and discriminatory index but they overall exhibit a slightly lower average values as the FCI. Moreover, a confirmatory factor analysis confirmed a three factor model that is closely aligned with a previously suggested expert model. In this way, after careful prompt engineering, thorough analysis and selection of fully automatically generated items by ChatGPT, we were able to create concept items that had only a slightly lower quality than carefully human-generated concept items. The procedures to create and select such a high-quality set of items that is fully automatically generated require large efforts and point towards cognitive demands of teachers when using LLMs to create items. Moreover, the results demonstrate that human oversight or student interviews are necessary when creating one-dimensional assessments and distractors that are closely aligned with students' difficulties.

Conceptual Coherence and Dominant Misconceptions in the Concept of Force among Higher Education Students through the Administration of the Force Concept Inventory

The paper reported here is a part of a larger 8000 student survey to determine the common misconceptions regarding Newtonian mechanics among Bhutanese students. The force concept inventory was used as a diagnostic instrument to probe students’ misconceptions regarding the concept of force in physics. The objective of the research was to determine the extent to which student scores demonstrated conceptual coherence, Newtonian thinking, and identify dominant misconceptions. Data was collected from university students majoring in physics (N = 40), year I and year IV pre-service teachers (N=155), by administering the questionnaire face-to-face. Data was analysed using descriptive statistics. Results suggests that students are not able to consistently apply Newtonian mechanics, and none have reached the Newtonian thinking threshold. Dominant misconceptions were identified in relation to kinematics, Newton’s first, second, and third laws of motion, superimposition principle, and kinds of force. Pre-service teacher educators, university lecturers, and curriculum developers should invest time and resources in identifying and addressing the misconceptions.

Understanding Newton's Third Law: A Study of Prospective Physics Teachers' Knowledge Structure

This research aims to explore the ability to understand concepts and the characteristics of the knowledge structure of prospective physics teachers regarding Newton's Third Law. This research was conducted on 26 Physics Education students in their first year who had received material on Newton's Laws. The instrument used is the Force Concept Inventory (FCI) test instrument. Using four multiple-choice questions accompanied by reasons, the quantitative data source comes from students' choice of answers. In contrast, the qualitative data source comes from students' reasons for choosing answers, which are used to see the characteristics of knowledge structures. The data analysis technique used is descriptive statistics to present a picture of the ability to understand concepts and the level of character of the knowledge structure of prospective teachers. The results show that the concept understanding ability of prospective physics teachers has an average of 67.31 and is classified as sufficient. The knowledge structure characteristics of prospective physics teachers at the expert level is 62.50%, the beginner level is 14.42%, and the intermediate level is 24.04%. These results show that the ability to understand the concept of Newton's Third Law and the characteristics of knowledge structures have a coherent influence. Prospective teachers with the characteristics of an expert knowledge structure can solve physics problems using correct physics principles. These findings require further research to explore the factors influencing understanding concepts and characteristics of prospective teachers' knowledge structures and more effective teaching strategies.

Biophysics Concept Inventory Survey: An Assessment in Biophysical Undergraduate Education

ABSTRACT Since the Force Concept Inventory in 1992, many concept inventories have been developed to cover classical scientific fields. However, there is a lack of concept inventories for interdisciplinary fields, such as biophysics. We introduce a Biophysics Concept Inventory Survey (BCIS), a 20-question, multiple-choice survey to measure student gains in biophysical concepts. The BCIS contains 5 question classifications: remember, understand, apply, analyze, and create, as well as question concepts divided into primarily physics or primarily biology questions. We administered the BCIS to 3 cohorts of students over 4 years. Each cohort participated in a 10-week summer Research Experience for Undergraduates (REU) in biophysics. We compared the presurvey (before REU) and postsurvey (after REU) scores to determine the fraction of the maximum possible gain or loss realized. Our analysis of the results suggests that the BCIS shows no biases based on sex or ethnicity. Further, we used the BCIS to show that 69% of the REU participants showed gains in biophysics concepts, with most of the total participant mean of gain occurring at higher levels of Bloom’s taxonomy: create and analyze. Overall, participants obtain slightly higher scores in physics (8% increase) than biology (5% increase) when comparing the pre- and postscores. The coronavirus disease 2019 pandemic allows a splitting of prepandemic and postpandemic cohorts, with the postpandemic cohort showing significantly larger gains than the prepandemic students. These results show that the BCIS, with question classifications and concepts, probes the students’ ability to apply knowledge to various biophysical science topics without underlying biases and enables instructors to obtain answers to important questions about the effectiveness of the educational programs. The BCIS fills a gap for interdisciplinary concept inventories.

Assessment of preservice physics teachers’ knowledge of student understanding of force and motion

In physics education, a number of studies have developed assessments of teachers’ knowledge of student understanding (KSU) of specific physics concepts with modified versions of existing concept inventories, in which teachers were asked to predict the popular incorrect answers from students. The results provide useful but indirect information to make inferences about teachers’ knowledge of the misconceptions that students may be using in answering the questions. To improve the assessment of teachers’ KSU, a new instrument is developed using a three-tier item design. The items were adapted from 17 questions from the Force Concept Inventory on force and motion. Each item was designed in three tiers, with tier 1 asking for teachers’ own answers to the question to test their content knowledge, tier 2 asking for teachers’ predictions of popular students’ incorrect answers, and tier 3 asking for teachers’ explanations of students’ incorrect answers in an open-ended form. The three-tier design captures teachers’ content knowledge, predictions, and explanations in a single item to allow explicit measures of teachers’ own content knowledge and their KSU on students’ misconceptions. The instrument was validated with preservice physics teachers, who were master-level graduate students in a normal university in China. The assessment results also suggest that the preservice teachers’ KSU of force and motion was only moderately developed, and their content knowledge was uncorrelated with their KSU. In addition, a four-level progression scale of KSU was also developed, which categorized the preservice teachers into five proficiency groups. Published by the American Physical Society 2024

Misconceptions of High School Students on Motion and Force Using the Force Concept Inventory (FCI)

This research aims to identify high school students' misconceptions regarding motion and force using the Force Concept Inventory (FCI) instrument. This study used a quantitative method with respondents of 108 students in class XI science. Misconception data was obtained using FCI in the form of multiple-choice questions. There are 30 questions with five options. The results showed that a total of 46 out of 108 students (19.44%) answered 8 to 11 questions correctly, 41 students (37.96%) answered 5 to 7 questions correctly and the other 21 students (42.59%) only answered 2 to 4 questions correctly. The conclusion is students had misconceptions about the concepts of motion and force, especially on topics such as kinematics, Newton's laws, the principle of superposition, and kinds of force. This research implies the importance of understanding students' misconceptions in developing physics curricula and learning strategies in high schools. It is hoped that the research results can provide information to teachers about students' conceptual understanding so that appropriate learning strategies can be developed to correct students' misconceptions in learning.

Persistence of Conceptual Errors in First-Year University Physics Course and Its Possible Relationship with Learning Styles

In this work, we study the persistence of some fundamental previous ideas in physics in a group of freshmen at the Universidad Politécnica de Madrid (Spain). For this purpose, we analyze the answers to a questionnaire consisted of 24 multiple-choice items, most of them borrowed from the Force Concept Inventory (FCI). Our study is performed in two different ways by using, on the one hand, classical test theory and, on the other hand, the Pearson product–moment correlation. The survivance of some of the previous ideas at the end of the course is assessed by comparing and critically analyzing the answers of the students to the same test at the beginning and the end of the term. A possible connection with Honey–Alonso learning styles (LS) is also discussed. The results yielded by our study demonstrate the persistence of some of the initial and previous ideas, no matter the students’ previous qualifications or their current LS.

Exploring gender differences in the Force Concept Inventory using a random effects meta-analysis of international studies

The Force Concept Inventory (FCI) is one of the research-based assessments established by the physics education research community to measure students’ understanding of Newtonian mechanics. Former works have often recorded the notion of gendered mean FCI scores favoring male students notably in the North American (NA) based studies. Nevertheless, these performance gaps remain inconclusive and unexplored outside the NA context. This paper aims to fill this gap by meta-analyzing the mean FCI scores between gender based on the existing physics education research literature internationally. We analyzed the magnitude and direction on the mean FCI scores between gender based on primary international studies published over the last two decades. We also explored the moderating impact of international study characteristics on the meta-analytic findings by performing a subgroup analysis to study the different study regions stratified by two subgroups (NA vs non-NA authors). Thirty-eight studies reporting the mean FCI scores by gender were included in the present meta-analysis. We employed Hedges’ g statistic to estimate to what degree the mean FCI scores may be different between male and female students on each study. Under a random effects model, we meta-analyzed the findings and conducted a subgroup analysis to answer the research questions. In summary, our meta-analysis indicated a significantly positive and moderate amount of gendered mean FCI scores in favor of male students both in NA- and non-NA based regions, and the performance gaps were wider in the NA-based studies. Suggestions are discussed while interpreting the mean FCI scores between gender for teaching, learning, and forthcoming studies. Published by the American Physical Society 2024

Methodological suggestions for creating subquestions for the Force Concept Inventory (and other research-based assessments)

The Force Concept Inventory (FCI) is a research-based assessment that is used internationally to assess student understanding of Newtonian mechanics. The assessment has been investigated from a number of perspectives and many suggestions have been made for its improvement. One suggestion is the creation of subquestions. Subquestions are a tool to reduce false positives (answering a survey item correctly without correct understanding) and false negatives (answering incorrectly despite correct understanding). In this presentation, we will discuss how we created subquestions for two items on the FCI, informed by survey-based interviews with students and the original intended targets of the items. Our research aims are not only to present the subquestions we developed, but to share our methodology of creating these subquestions, as this methodology can be used in general for producing subquestions on other research-based assessments as well.

Appropriate TPACK Components for Overcoming Misconceptions on Force Concept Inventory Test

Misconception is one of the obstacles in the achievement of physics learning and must be immediately corrected so that there is no retention of misconceptions in subsequent knowledge. One way to prevent and reduce misconceptions that occur is with the teachers skill of Technological Pedagogical and Content Knowledge (TPACK). Teachers who integrate TPACK in learning can prevent misconceptions and find solutions to overcome the misconceptions. This study aims to determine the TPACK skill of teachers in overcoming misconceptions and the appropriate TPACK components to reduce misconceptions based on concepts on the three tier test of FCI (Force Concept Inventory). The research method used is mixed method with concurrent embedded design. The main data in this study is TPACK which is qualitative data and supporting data is students misconception which is quantitative data. The results showed a decrease in misconceptions after being given an understanding of the concept using the teacher's TPACK skill. The components used during the teacher's reduction of misconceptions based on the concepts in the FCI three tier test of FCI are TPACK and PCK components. So it can be concluded that the teacher's TPACK skill can reduce the misconceptions that occur in the three tier test of FCI

Analysing the structure of Kazakhstan university undergraduate students’ knowledge about the force concept: findings from a three-tier FCI survey

This mixed method research (quantitative and qualitative) is dedicated to analyse the Structure of Kazakhstani University Students’ knowledge about the force concept. For this purpose we extend the existing, well known instrument Force Concept Inventory (FCI), applying a three-tier test approach, by asking students to write an explanation of scientific reasons for choosing a certain answer of FCI in the second tier and checking their confidence level in the third tier. Also, the survey was translated into Kazakh language and validated for the Kazakhstan population. The results showed the test has high content and construct validity and high reliability. When analysing the results of the survey, it was found that considering the FCI as the three-tier test significantly affected the number of identified misconceptions and the assessment of scientific knowledge, which implied higher reliability in the results obtained. The obtained data suggest that some incorrect answers should be specified as a lack of knowledge rather than misconceptions, and some correct answers should be considered as a lucky guess rather than scientific knowledge. In the study, we showed the most common misconceptions of the Kazakhstani University students population and detected a gender gap in conceptual understanding of Mechanics, which attracts the attention of researchers for their future studies.

A three-year comparative study of dominant misconceptions among first-year physics students at a South African university

This article discusses a three-year study (2020–2022) of dominant misconceptions (DMs) for a large cohort of first-year physics course students at the University of Johannesburg, South Africa. Our study considered pre-test scores on the force concept inventory using a graphical method, where we found statistical differences between the mean DM scores for the 2020 cohort, as compared to the 2021 and 2022 cohort; possibly due to the onset of COVID lockdowns. We also compared our data from South Africa with cohorts based in Spain and the Kingdom of Saudi Arabia, where the method of DMs was also applied. From this comparison, we found some differences in the preconception knowledge of the cohorts. Furthermore, we included an analysis of DMs through the ‘gender lens’ for the South African cohort, finding no statistically significant difference between the means for DM scores of students who identify as male or female. Finally, given the diverse language backgrounds and levels of matriculation preparation foruniversity level physics courses, we have also shown how quickly responding to student misconceptions can be efficiently addressed using the method of DMs.

Effect of Interactive Conceptual Instruction Assisted by PhET Simulations on the Student’s Scientific Consistency in Physics

This study aims to determine the effect of Interactive Conceptual Instruction Assisted by PhET Simulations on students' scientific consistency. The research method used was quasi experiment with the matching-only pretest-posttest control group design involving 62 students in one of the high schools in Indonesia. The instrument used was an scientific consistency test modified from the Representational Variant of Force Concept Inventory (R-FCI). The data obtained were analyzed by t-test and N-Gain. The results showed that Interactive Conceptual Instruction Assisted by PhET Simulations can improve students' scientific consistency.

Newton Makes Me Happy: Cycling Emotions during Science Text Reading

Some researchers have theorized that emotions while reading science texts influence learning in a wholistic way, such that overall positive or negative affect leads to different learning outcomes. Other researchers have envisioned that emotions fluctuate during reading such that the degree of cycling impacts the learning outcomes. In this study, participants read both expository and refutational science texts sentence-by-sentence. They described how each sentence made them feel and why. They completed the Force Concept Inventory before and after reading. The amount of positive emotions predicted conceptual growth for both types of texts. Students who tended to switch their emotions more often and had a variety of emotions also tended to learn more than other students. These results suggest that both types of theories could be true. An overall positive affect might be best, but the most effective students cycle through short periods of negative emotions as well.

Evaluation of Newton's Law Learning

The Force Concept Inventory is a test to determine the level of conceptual knowledge of students about mechanical physics and to evaluate the effectiveness of different teaching strategies on the conceptual component of learning. The test was applied with the purpose of knowing the level of conceptualization of the students who study the subject of Physics in a basic and medium educational institution. The results of the pre-test gave to know the students' level of conceptualization and offered information for the elaboration of workshops based on real physical situations to elaborate force diagrams. The results of the post-test allowed estimating Hake's learning gain and showed evidence about the students' conceptual evolution and information to develop future teaching activities about Newton's laws

High school students' perceptions and experiences of using combined RW and labatorials to understand Newton's laws of motion

We discuss the perceptions and experiences of high school students using reflective writing and labatorials to attempt to understand Newtonian concepts of force and motion. The sequencing and content of these activities are centered on targeting students’ key difficulties related to forces and motion. Participants are 210 secondary 5 (grade 11) students, from three private schools in Montreal, who took a physics course during 2017–2018 or 2018–2019. Their ideas and opinions about forces and learning physics were investigated prior to and following the study with (a) three questions from the Force Concept Inventory; (b) a concept map focused on the relations between force and motion, and (c) pre- and post-semi-structured interviews conducted with 12 participants. The gathered data and interviews indicate that the process of combining labatorials with reflective writing improves students’ attitudes towards learning the subject.

Task-based attentional and default mode connectivity associated with science and math anxiety profiles among university physics students

PurposeAttentional control theory (ACT) posits that elevated anxiety increases the probability of re-allocating cognitive resources needed to complete a task to processing anxiety-related stimuli. This process impairs processing efficiency and can lead to reduced performance effectiveness. Science, technology, engineering, and math (STEM) students frequently experience anxiety about their coursework, which can interfere with learning and performance and negatively impact student retention and graduation rates. The objective of this study was to extend the ACT framework to investigate the neurobiological associations between science and math anxiety and cognitive performance among 123 physics undergraduate students. ProceduresLatent profile analysis (LPA) identified four profiles of science and math anxiety among STEM students, including two profiles that represented the majority of the sample (Low Science and Math Anxiety; 59.3% and High Math Anxiety; 21.9%) and two additional profiles that were not well represented (High Science and Math Anxiety; 6.5% and High Science Anxiety; 4.1%). Students underwent a functional magnetic resonance imaging (fMRI) session in which they performed two tasks involving physics cognition: the Force Concept Inventory (FCI) task and the Physics Knowledge (PK) task. FindingsNo significant differences were observed in FCI or PK task performance between High Math Anxiety and Low Science and Math Anxiety students. During the three phases of the FCI task, we found no significant brain connectivity differences during scenario and question presentation, yet we observed significant differences during answer selection within and between the dorsal attention network (DAN), ventral attention network (VAN), and default mode network (DMN). Further, we found significant group differences during the PK task were limited to the DAN, including DAN-VAN and within-DAN connectivity. ConclusionsThese results highlight the different cognitive processes required for physics conceptual reasoning compared to physics knowledge retrieval, provide new insight into the underlying brain dynamics associated with anxiety and physics cognition, and confirm the relevance of ACT theory for science and math anxiety.

What Physics Instructors Can Learn from Astronomy Teaching

A student says, “Even though the physics involved can be basically the same, the setup is what drives me to like astronomy more,” comparing his introductory physics class to his introductory astronomy class. Another student concurs, “Astronomy is more exciting.” What is it about astronomy that appeals to students? Is there a way to harness such enthusiasm within the context of an introductory physics class while retaining the same level of rigor? Here, we describe a study that shows that, compared with an introductory physics course for science and engineering majors, an introductory astronomy course (also for science and engineering majors) had similar gains on the Force Concept Inventory (FCI), but students’ attitudes and approaches to problem solving were significantly better in the latter. Moreover, interviewed students corroborated the survey findings that the astronomy course was significantly more enjoyable and engaging. This suggests that physics instructors can learn from the way astronomy is often taught, in order to better engage students.

Establishing a physics concept inventory using computer marked free-response questions

The study covers the development and testing of the alternative mechanics survey (AMS), a modified force concept inventory (FCI), which used automatically marked free-response questions. Data were collected over a period of three academic years from 611 participants who were taking physics classes at high school and university level. A total of 8,091 question responses were gathered to develop and test the AMS. The AMS questions were tested for reliability using classical test theory (CTT). The AMS computer marking rules were tested for reliability using inter-rater reliability (IRR). Findings from the CTT and IRR studies demonstrated that the AMS questions and marking rules were overall reliable. Therefore, the AMS was established as a physics concept inventory which uses automatically-marked, free-response questions. The approach used to develop and test the AMS could be used in further attempts to develop concept inventories which make use of automatically-marked, free-response questions.