Contribution: This article proposes a new definition of conceptual understanding (CU) specific to engineering. It then measures CU of signals and systems (S&S) in senior undergraduate students and describes how students approach conceptual problems. Background: Previous studies across multiple engineering subjects show students have low CU at the end of courses. However, little is known about CU semesters after a course. Research Questions: What is the CU of S&S concepts among electrical engineering senior students? Methodology: This mixed method study uses quantitative concept inventory data <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$(n=467)$ </tex-math></inline-formula> and think-aloud interviews <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$(n={12})$ </tex-math></inline-formula> to measure CU. The results come from two universities. Findings: Seniors’ scores on the concept inventory are typical of scores presented at the end of an S&S course. Many struggled with the concept of linearity, made a common error when finding the maximum value in graphical convolution, and had low confidence on relating frequencies in time to a Fourier transform representation, but seniors had relatively high CU of time invariance and filtering.

Examination of my students’ individual gains on the Force Concept Inventory (FCI) over the course of several semesters led to the realization that student pretest knowledge on certain key questions appeared to be correlated to enhanced gain during the class. Acting under the hypothesis that early intervention aimed at helping the class perform better on those select key questions might result in enhanced test-wide gains, I developed two class activities targeted at just one of the key questions. I have tested those activities over the course of four semesters and there is evidence that they significantly increased the class-wide normalized gains on the FCI.

The Force Concept Inventory (FCI) has been used as a research instrument and to evaluate the conceptual understanding of students since it was first published in 1992. It is commonly used to assess the formation of conceptual structures in the minds of students who are learning Newtonian dynamics. For this reason it is vital that both researchers and teachers are assured that such conceptual coherence does, in fact, appear in FCI data, and are aware of the differences between Newtonian concepts as they appear in experts, and as they appear in students. The research presented here provides evidence that this conceptual coherence exists in FCI response data. This evidence is the result of a careful factor analysis of such data and we present the factor structure found in this analysis. This factor structure does not correspond exactly to the conceptual structure proposed by the authors of the FCI and thus is evidence of the difference between expert and novice conceptualisations of Newtonian Mechanics. Furthermore, we also provide an item response analysis of FCI data which is able to provide us with a better understanding of the specific abilities developed by our students and the interactions between these abilities as student obtain mastery of Newtonian ideas.

The Force Concept Inventory was designed to poll the Newtonian conception of force. While there are many in-depth studies analyzing response data that look at the structure of the correct answers, we believe that the incorrect answers also carry revealing information about the students’ worldview. The inventory was originally designed so that the “distractors” in each question reflected commonly held misconceptions, and thus the rate at which students guess the correct answer is very low. Students select incorrect answers that correspond to the misconception that they hold and there are very few responses which appear obviously wrong to students. A side effect of this approach is that the incorrect responses reflect the non-Newtonian worldviews held by students. These non-Newtonian worldviews are coherent and robust, and this, at least in part, helps to explain why these misconceptions are so resistant to instruction. In this study we focus once more on the misconception data in Force Concept Inventory responses, particularly on the linkages between these misconceptions. We hypothesize that there are distinct groupings of distractor items formed by the strength of the association between these items. The two largest groupings are associated with the “impetus” world view in which the motion of an object is determined by the quantity of impetus which that object contains. We find that certain central items hold particularly important places in these groupings and also that individual groupings may be connected to each other by “connector” items. We finally suggest that, on the basis of this study, that these non-Newtonian worldviews might best be dismantled by addressing these key central and connector items.

The Force Concept Inventory is one of the most popular and most analyzed multiple-choice concept tests used to investigate students’ understanding of Newtonian mechanics. The correct answers poll a set of underlying Newtonian concepts and the coherence of these underlying concepts has been found in the data. However, this inventory was constructed after several years of research into the common preconceptions held by students and using these preconceptions as distractors in the questions. Their sole purpose is to deflect non-Newtonian candidates away from the correct answer. Alternatively, one can argue that the responses could also be treated as polling these preconceptions. In this paper we shift the emphasis of the analysis away from the correlation structure of the correct answers and look at the latent traits underlying the incorrect responses. Our analysis models the data employing exploratory factor analysis, which uses regularities in the data to suggest the existence of underlying structures in the cognitive processing of the students. This analysis allows us to determine whether the data support the claim that there are alternate non-Newtonian worldviews on which students’ incorrect responses are based. The existence of such worldviews, and their coherence, could explain the resilience of non-Newtonian preconceptions and would have significant implications to the design of instruction methods. We find that there are indeed coherent alternate conceptions of the world which can be categorized using the results of the research that led to the construction of the Force Concept Inventory.

We describe Module Analysis for Multiple Choice Responses (MAMCR), a new methodology for carrying out network analysis on responses to multiple choice assessments. This method is used to identify modules of non-normative responses which can then be interpreted as an alternative to factor analysis. MAMCR allows us to identify conceptual modules that are present in student responses that are more specific than the broad categorization of questions that is possible with factor analysis and to incorporate non-normative responses. Thus, this method may prove to have greater utility in helping to modify instruction. In MAMCR the responses to a multiple choice assessment are first treated as a bipartite, student X response, network which is then projected into a response X response network. We then use data reduction and community detection techniques to identify modules of non-normative responses. To illustrate the utility of the method we have analyzed one cohort of postinstruction Force Concept Inventory (FCI) responses. From this analysis, we find nine modules which we then interpret. The first three modules include the following: Impetus Force, More Force Yields More Results, and Force as Competition or Undistinguished Velocity and Acceleration. This method has a variety of potential uses particularly to help classroom instructors in using multiple choice assessments as diagnostic instruments beyond the Force Concept Inventory.10 MoreReceived 13 January 2016DOI:https://doi.org/10.1103/PhysRevPhysEducRes.12.020131This article is available under the terms of the Creative Commons Attribution 3.0 License. Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI.Published by the American Physical SocietyPhysics Subject Headings (PhySH)Research AreasAssessmentCollective behavior in networksCommunity structureInstructional materials developmentPhysics Education ResearchNetworks

In this paper we present a series of item response models of data collected using the Force Concept Inventory. The Force Concept Inventory (FCI) was designed to poll the Newtonian conception of force viewed as a multidimensional concept, that is, as a complex of distinguishable conceptual dimensions. Several previous studies have developed single-trait item response models of FCI data; however, we feel that multidimensional models are also appropriate given the explicitly multidimensional design of the inventory. The models employed in the research reported here vary in both the number of fitting parameters and the number of underlying latent traits assumed. We calculate several model information statistics to ensure adequate model fit and to determine which of the models provides the optimal balance of information and parsimony. Our analysis indicates that all item response models tested, from the single-trait Rasch model through to a model with ten latent traits, satisfy the standard requirements of fit. However, analysis of model information criteria indicates that the five-trait model is optimal. We note that an earlier factor analysis of the same FCI data also led to a five-factor model. Furthermore the factors in our previous study and the traits identified in the current work match each other well. The optimal five-trait model assigns proficiency scores to all respondents for each of the five traits. We construct a correlation matrix between the proficiencies in each of these traits. This correlation matrix shows strong correlations between some proficiencies, and strong anticorrelations between others. We present an interpretation of this correlation matrix.

We perform a factor analysis on a ``Force Concept Inventory'' (FCI) data set collected from 2109 respondents. We address two questions: the appearance of conceptual coherence in student responses to the FCI and some consequences of this factor analysis on the teaching of Newtonian mechanics. We will highlight the apparent conflation of Newton's third law with Newton's first law in one of the FCI questions and suggest an approach to teaching that may avoid this issue. We also note the absence of a distinct factor interpretable as relating specifically to kinematics. Furthermore, we identify and discuss some of the technical difficulties which may be encountered when performing factor analysis on categorical data sets, such as is the case with FCI data sets.

This essay describes how the use of a concept inventory has enhanced professional development and curriculum reform efforts of a faculty teaching community. The Host Pathogen Interactions (HPI) teaching team is composed of research and teaching faculty with expertise in HPI who share the goal of improving the learning experience of students in nine linked undergraduate microbiology courses. To support evidence-based curriculum reform, we administered our HPI Concept Inventory as a pre- and postsurvey to approximately 400 students each year since 2006. The resulting data include student scores as well as their open-ended explanations for distractor choices. The data have enabled us to address curriculum reform goals of 1) reconciling student learning with our expectations, 2) correlating student learning with background variables, 3) understanding student learning across institutions, 4) measuring the effect of teaching techniques on student learning, and 5) demonstrating how our courses collectively form a learning progression. The analysis of the concept inventory data has anchored and deepened the team's discussions of student learning. Reading and discussing students' responses revealed the gap between our understanding and the students' understanding. We provide evidence to support the concept inventory as a tool for assessing student understanding of HPI concepts and faculty development.