Multiple-choice Items Research Articles

Exploratory Assessment of GPT-4's Effectiveness in Generating Valid Exam Items in Pharmacy Education.

Exploratory Assessment of GPT-4's Effectiveness in Generating Valid Exam Items in Pharmacy Education.

Digital Linguistic Markers of Emotions during the September 2024 Hostilities in Lebanon: A Case Study in Communication Arts Course at a University Level in Southern Lebanon

Language and emotions are intricate systems that have several interactions. Amid the expeditious spread of digitalization and the upswing of various international issues, this case study seeks to explore how students at the Lebanese University in the South utilize language on social media to convey their emotions as of the 2024 clashes and airstrikes in Lebanon. Further, it aims to identify digital linguistic markers that reflect key emotional states and social identity. Drawing on discourse analysis, which acknowledges language as a dynamic system influenced by shared experiences, values, and beliefs, and notably, by Conceptual Act Theory, which explores how meaning is constructed from experiences and perceptions, and on sentiment analysis, a mixed-methods approach is employed. A convenient sampling of (191) university EFL students taking the Communication Arts module during the fall of 2024-2025 completed an online survey of 15 multiple-choice and Likert scale items. Twelve students joined a focus group discussion, and 155 out of 191 participants interacted on the web-based application Padlet. The findings show that utilizing language in the context of social media platforms through words, emojis, hashtags, slogans, or posting articles, images, or videos plays a vital role not only in channeling positive and negative sentiments but also in signaling belonging to a collective identity during times of crisis in Lebanon. The overall analysis indicated that using various linguistic practices on social media platforms communicates positive and negative emotions and upholds social norms and ideologies.

Comparison of Psychometric Properties of Turkish as a Foreign Language Listening Tests Composed of Independent and Common-Stem Items

This study comparatively examined the psychometric properties of Turkish listening tests composed of independent and common-stem multiple-choice items that measure the same construct in the context of teaching Turkish as a foreign language. Additionally, students’ perceptions of these two different item formats were evaluated. The research was designed as a descriptive study using a relational survey model. The study group consisted of 201 international students enrolled at the B1 level in Turkish Language Teaching Centers during the 2024–2025 academic year. The assessment tools were developed on the Concerto platform, and the tests were administered online in a single session across three stages. According to the findings, comparisons of item difficulty indices revealed statistically significant differences and moderate to large effect sizes in some item pairs, indicating that item format may influence student performance. Similarly, item discrimination indices showed significant differences in several item pairs, although both tests overall consisted of highly discriminative items. Analysis of test mean scores showed that the independent item test yielded significantly higher performance at the 0.05 level. The reliability coefficients of both tests were high, with no statistically significant difference between them. Based on students’ perceptions, the independent item test was found to be more advantageous in terms of time management and online usability. In contrast, the common-stem item format was perceived as more cognitively demanding. Both item types received similarly positive feedback regarding item clarity and ease of answering listening questions. These results suggest that in teaching Turkish as a foreign language, independent items may contribute to higher student achievement and improved item discrimination in listening assessments.

Item analysis of multiple choice questions from assessment of health sciences students, Tigray, Ethiopia

BackgroundRapid expansion of higher education and graduation rate in Ethiopia, doesn’t tell much about quality of core processes like teaching and learning. Despite the fact that preparation of a good item is essential, little attention has been paid to examining characteristics of exam items such as difficulty, discrimination, and distractor efficiency. Unless these characters are examined results may lead to biased evaluation. To address the current issue, this study aimed to carry out item analysis of multiple choice questions (MCQs) in undergraduate health sciences students of Mekelle University, Tigray, Ethiopia in order to evaluate correlations between item difficulty, item discrimination and distraction efficiency.MethodsInstitution based cross sectional study was conducted from March to April 2020. A total of 189 exam papers with 539 multiple choice items and 2224 distracters were analyzed in terms of difficulty, discrimination, distractor efficiency and internal consistency reliability. SPSS version 25 was used for analysis & findings presented using tables & figures. Pearson’s correlation & one-way analysis of variance were used.ResultsIn this study mean difficulty ranged from 0.57–0.81, and discrimination from 0.27–0.52. Difficulty and discrimination were negatively correlated. Six departments achieved excellent reliability, indicating r>0.9. However, the reliability score of Medicine and Pharmacy departments was poor (r<0.5), suggesting areas for improvement. The remaining two departments exhibited acceptable reliability. More than three fourth of the items had non-functional distractors.ConclusionsThe internal consistency of the exams from six departments demonstrated excellent reliability. However, the exams from the Medicine and Pharmacy departments showed areas for improvement. In order to enhance assessment effectiveness, it's crucial to incorporate items that exhibit both desirable difficulty and high discrimination along with functionally realistic distracters. This approach ensures assessments accuracy.

Improving the Measurement of Students’ Composite Ability Score in Mixed-Format Assessments

Mixed-format assessments, which include both multiple-choice (MC) and constructed-response (CR) items, often produce separate scoring scales, with MC items scored dichotomously and CR items scored polytomously. Conventional methods for estimating composite ability scores, such as weighting or summing, rely on subject matter expertise but overlook the information embedded in MC item scores. While recent progress takes advantage of empirical Bayes analysis for estimating composite ability scores, it may also introduce biases because it relies solely on point estimates without accounting for the variability in unknown ability distributions and other parameters. To address these gaps, this study introduces a practical and easily implementable method, empirical fully Bayesian, that leverages MC item scores to derive empirical priors, leading to more accurate composite score estimates. It has been found that MC scores have effectively captured students’ achievement in the assessment domain, and they could provide valuable information for final scoring. Through empirical analyses of students in Grades 3 to 10 and two additional simulation studies based on real-world data, we demonstrate that this approach enhances composite ability score reliability, reduces reporting biases, and provides a valuable empirical evaluation tool for mixed-format assessments.

On a Reparameterization of the MC-DINA Model.

The MC-DINA model is a cognitive diagnosis model (CDM) for multiple-choice items that was introduced by de la Torre (2009). The model extends the usual CDM in two basic ways: it allows for nominal responses instead of only dichotomous responses, and it allows skills to affect not only the choice of the correct response but also the choice of distractors. Here it is shown that the model can be re-expressed as a multinomial logit model with latent discrete predictors, that is, as a multinomial mixture model; a signal detection-like parameterization is also used. The reparameterization clarifies details about the structure and assumptions of the model, especially with respect to distractors, and helps to reveal parameter restrictions, which in turn have implications for psychological interpretations of the data and for issues with respect to statistical estimation. The approach suggests parsimonious models that are useful for practical applications, particularly for small sample sizes. The restrictions are shown to appear for items from the TIMSS 2007 fourth grade exam.

ASSESSING THE EFFECT OF NUMBER OF OPTIONS ON PSYCHOMETRIC PROPERTIES OF WASSCE MULTIPLE CHOICE MATHEMATICS PAPER IN KATSINA ZONAL EDUCATION QUALITY ASSURANCE, KATSINA STATE, NIGERIA

This paper assessed the effect of number of options on psychometric properties of the West African Senior School Certificate Examinations (WASSCE) multiple choice mathematics paper in Katsina Zonal Education Quality Assurance Katsina State-Nigeria. The instrument for data collection was 2023 WASSCE Objectives test item. The study employed a repeated measures research design. The total population of this study comprised of seventeen thousand, two hundred and fifty-seven (17,257) mathematics students. A sample size of 378 SSSII students were selected. Simple random sampling technique was employ with proportional sampling technique and Krejcie and Morgan (1970) table for the study. The data collected were analyzed using descriptive statistics and cronbach alpha reliability. The instrument did not go through validation and reliability tests as WASSC examinations are standardized and must have gone through those processes. The Mathematics teachers in the respective sampled schools helped in the administration of the questions to the sampled students. The scored scripts of the students were exposed to item analysis to determine the difficulty, discrimination and distracter indices of those items. The findings showed that the students’ performance is below average, and their academic performance in Mathematics is low. It further revealed that the t-test calculated value is 1.84 while its p-value is 0.67 at alphan level of 0.05 which showed that there is no significant difference in the academic performance of students (involved in item analysis) in WASSCE May/June 2023 multiple choice items Mathematics Examination in Katsina Zonal Education Quality Assurance, Katsina State- Nigeria based on gender, school types and school location. Based on the results, it was recommended that the Government should constitute a joint examination evaluation team to ensure that examination items constructed by different examination bodies but for same purposes are of comparable standard and the WASSCE and other examination bodies should ensure that only quality items are developed for assessment.

Assessing the Effect of Number of Options on Psychometric Properties of WASSCE Multiple Choice Mathematics Paper in Katsina Zonal Education Quality Assurance, Katsina State, Nigeria

This paper assessed the effect of number of options on psychometric properties of the West African Senior School Certificate Examinations (WASSCE) multiple choice mathematics paper in Katsina Zonal Education Quality Assurance Katsina State-Nigeria. The instrument for data collection was 2023 WASSCE Objectives test item. The study employed a repeated measures research design. The total population of this study comprised of seventeen thousand, two hundred and fifty-seven (17,257) mathematics students. A sample size of 378 SSSII students. Simple random sampling technique was employ with proportional sampling technique and Krejcie and Morgan (1970) table for the study. The data collected were analyzed using descriptive statistics and cronbach alpha reliability. The instrument did not go through validation and reliability tests as WASSC examinations are standardized and must have gone through those processes. The Mathematics teachers in the respective sampled schools helped in the administration of the questions to the sampled students. The scored scripts of the students were exposed to item analysis to determine the difficulty, discrimination and distracter indices of those items. The findings showed that the students’ performance is below average, their academic performance in Mathematics is low. It further revealed that the t-test calculated value is 1.84 while its p-value is 0.67 at alphan level of 0.05 which showed that there is no significant difference in the academic performance of students (involved in item analysis) in WASSCE May/June 2023 multiple choice items for Mathematics Examination in Katsina Zonal Education Quality Assurance, Katsina State- Nigeria based on gender, school types and school location. Based on the results, it was recommended that the Government should constitute a joint examination evaluation team to ensure that examination items constructed by different examination bodies but for same purposes are of comparable standard and the WASSCE and other examination bodies should ensure that only quality items are developed for assessment.

Relating Decision-Making Styles, Examination Questions Types, Sleep Deprivation and Academic Performance of College Students in Tanzania

This study examined the relationship between performance in examination questions types, decision-making styles, sleep deprivation and performance in general performance average in a sample of 598 students in the Community Development Training Institutes in Tanzania. The study responded to the quests as to whether or not could maximizers and satisficers differ in reporting sleep deprivation and Grade Performance Average; Could maximizers and satisficers differ in terms of performance with type of examination questions such as multiple choice items (MC), matching items (MI), true/false items (T/F), short answers (SA), or Essay; and what could be the relationship between sleep deprivation and academic performance in terms of GPA. Participants concurrently responded to the Sleep Deprivation scale (SDS) and Maximization Scale (MS). Then, performance scores in the examination questions types were traced directly to the examination scripts of the Community Psychology course purposely selected to match the researcher’s area of specialization. Data were analyzed using MANOVA and Pearson’s Moment Correlation Coefficient with an assistance of the Statistical Package for Social Sciences (SPSS). It was found that only 6.5% of respondents reported being maximizers while 93.3% reported being satisficers. Further, while there was a significant difference between Maximizers and satisficers in sleep deprivation, F (1, 593) = 6.50, p = 0.011, partial eta squared = 0.011; no significant difference between maximizers and satisficers was found in performance in GPA or in the examination questions types. In addition, there were from low to strong positive correlations between examination question types and performance in GPA. Although psychological factors such as sleep deprivation and decision-making have a reciprocal influence, their influence to students’ GPA and scores in question types is not as clear as it is between scores in the question types and GPA, being an indication of malleability of maximization with domain and context specificity.

Social Media Influence on Investment Decisions: Insights from Nepal's Capital Market

This study investigates the influence of social media and online community groups on individual investors’ decision-making processes, with capital market literacy serving as a mediating factor. The research employed a descriptive and causal-comparative design, utilizing a purposive sampling method to select 384 respondents. Data collection was conducted through a structured questionnaire comprising multiple-choice and Likert-scale items. The reliability of the instrument was evaluated using Cronbach’s alpha, and data analysis was performed using descriptive and regression techniques. The findings indicate that Facebook is the predominant platform utilized by investors, who are primarily between 26 and 35 years of age, hold bachelor’s degrees, and self-identify as having intermediate investment experience. The analysis reveals a statistically significant positive correlation between social media, online community groups, and investment decisions, with capital market literacy partially mediating this relationship. These results underscore the increasing impact of online platforms on investment behaviour and emphasize the necessity for enhancing capital market literacy. Future research should explore the long-term effects of social media on investment performance and examine the influence of various social media types on different investment strategies across demographic groups.

Development of standardized mathematics achievement test instruments for measuring senior secondary school students’ learning outcomes

This study developed a Standardized Mathematics Achievement Test (SMAT) instrument for measuring senior secondary school students’ learning outcomes in Dala education directorate, Kano State, Nigeria. The study adopted an instrumentation design. Three research questions were raised to guide the study. The population of the study comprised all 6,462 SS 3 students of the Dala education directorate. The sample size of the study was 132 SS 3 students drawn from the target population in 7 schools with a total number of 248 students using purposive sampling. Fifty multiple-choice items of the SMAT was developed by the researchers. The content validity of the SMAT was determined using a table of specifications. The reliability of the instrument was calculated using Kuder-Richardson formula 20 (KR-20). The result indicated that the content validity as presented in the table of specification of the SMAT items covered the six main sections of the Mathematics scheme. The result indicated that the internal consistency index of the SMAT was found to be a highly reliable instrument with a reliability coefficient of 0.79. The result also indicated that the difficulty index (DifI) of the SMAT items is moderately difficult, and the discriminatory power indicated that the test items distinguish between intelligent and weaker students. The study recommended that mathematics teachers and researchers should always determine the difficulty and discrimination indices of the test items before using such instruments as a tool for measuring students’ learning outcomes.

Influence of Mind Wandering and Increased Attentional Demands on Multitasking and Implicit Learning.

The goal of the current study was to replicate resent findings that suggest mind wandering is associated with impaired explicit learning but not implicit learning, and to extend those finding by investigating whether explicit learning is impaired under attentional load, but implicit learning is not. We used a sequential learning task, specifically a serial reaction task (SRT), to determine if mind wandering would interfere with learning a task that does not require attentional resources (implicit learning). Participants completed the serial reaction time task while watching a 13-minute video lecture. At the end of the video participants answered 10 multiple-choice questions regarding the content presented in the video. At specific intervals during the task, participants responded to mind wandering probes. The probes required participants to report where their attention was in the moments before the probe appeared. Implicit learning was measured by decreased reaction time over the course of several blocks of trials of the SRT. In two experiments, it was observed that participants implicitly learned a sequence of 12 items, regardless of their performance on the multiple-choice item regarding the concurrent video content. Even those who appeared to actively engage with the video and performed well on the multiple-choice questions showed improved performance on the implicit learning task (SRT). These results suggest implicit learning can occur when one is engaged in a concurrent explicit learning task.

A descriptive analysis of emergency medicine residency scholarly tracks faculty workforce.

Scholarly tracks typically consist of longitudinal subspecialty-specific curricula and mentorship for residents. Roughly one in five emergency medicine (EM) residency programs offer scholarly tracks, allowing residents to explore a niche and develop skills to prepare them for their future careers. There is limited information on the faculty workforce that leads scholarly tracks. Our objective was to understand the workforce characteristics of EM track leaders, specifically their effort and compensation. We performed a cross-sectional survey study of EM scholarly track leaders who were identified by reviewing program websites and directly contacting program coordinators. Participants completed an electronic survey consisting of multiple-choice and completion items. We calculated descriptive statistics. There were 112 of 276 (40%) track leaders who completed the survey, including 63 (57.3%) males and 47 (42.7%) females. The mean (±SD) age was 42.6 (±7.5) years. A majority (62.5%) had completed fellowship. Participants supervise a mean of 2.6 residents per year and spend a median (range) of 7.6 (4-10.7) hours per month on track activities. A total of 57.1% of participants received no compensation for their role. Nonmonetary benefits included career satisfaction (87.6%), intellectual stimulation (76.4%), departmental recognition (41.6%), and increased scholarly productivity (33.7%). Overall, 71.7% of faculty track leaders would recommend the role to a colleague. This study characterizes the current EM scholarly track leader workforce in terms of demographics, activities, time and effort, and compensation. It sheds light on this key educational role and on the opportunity to provide better support to faculty track leaders. These results may inform decision making of current and would-be scholarly track leaders as well as institutional leadership.

Content and Item Response Theory Analysis of ChatGPT-4-Generated Multiple-Choice Items

Background Multiple-choice item (MCI) assessments are burdensome for instructors to develop. Artificial intelligence (AI, e.g., ChatGPT) can streamline the process without sacrificing quality. The quality of AI-generated MCIs and human experts is comparable. However, whether the quality of AI-generated MCIs is equally good across various domain- and task-specific prompts remains to be determined. Therefore, we ask whether AI can generate high-quality MCIs to assess learning outcomes from a psychology textbook chapter reading. Objective In an exploratory study, we enlist Item Response Theory analysis and expert reviewers to assess MCIs generated by ChatGPT-4 from a psychology textbook chapter. Method We submitted a prompt and textbook chapter to ChatGPT-4 requesting 20 MCIs. One hundred ninety undergraduate participants read the chapter before responding to the MCIs. Expert reviewers assessed the MCIs for learning outcome alignment and quality. Results ChatGPT-4-generated MCIs were low in difficulty and high in discrimination. Expert reviewers found that nearly all items were logically sound, aligned with learning objectives, and met prevailing standards of MCI quality. Conclusion When carefully prompted, ChatGPT-4 can rapidly generate high-quality MCIs to test comprehension of a psychology textbook chapter. However, due to the uniformly low difficulty of the items, we recommend enlisting ChatGPT-4 to write MCIs for formative, but not summative, assessments.

When Systems Fall Short: Employee Perspectives on Knowledge Management and Enterprise Social Networks

This study examines the limitations and challenges of knowledge management systems (KMS) and enterprise social networking systems (ESNS) in a multinational professional services firm. While digital tools like MS Teams and SharePoint are critical for knowledge sharing and collaboration, their effectiveness varies significantly depending on employee group and career stage. To fill an important gap in the literature, this study illuminates how employees in different positions perceive and interact with these systems and demonstrates their impact – or lack thereof – on the distribution of knowledge within the organization. A structured online questionnaire was used to collect responses from 66 employees, combining Likert scales, multiple choice items and numerical ratings. Statistical analysis using IBM SPSS revealed correlations, mean differences, and trends in system usage and satisfaction. The results show that KMS and ESNS have deficits in critical areas: new employees have difficulty integrating knowledge effectively, while experienced employees are dissatisfied with the efficiency and functionality of the systems. In particular, MS Teams proved to be the most effective tool, while MS SharePoint was considered problematic. These findings suggest that targeted KMS improvements and tailored onboarding strategies are critical to closing knowledge gaps and improving collaboration between different organizational levels and regions.

Comparing two psychometric approaches: The case of item analysis for a classroom test in mathematics

Multiple-choice and constructed-response item formats are widely used in Philippine classrooms where tests are primarily summative. Conducting item analysis provides teachers the professional possibility to independently improve their test items while also allowing them to make informed decisions about test improvement and ensure that their assessments provide meaningful scores. Comparisons of item analysis approaches have been extensively used in standardized tests but not so much in small-scale testing such as classroom assessments. Hence, this research reports an empirical study where a summative test for classroom use is analyzed using psychometric techniques from the Classical Test Theory (CTT) and the Item Response Theory (IRT). This study employs an action research design where the primary author being a teaching practitioner-researcher, reports how an item analysis of his teacher-made summative test was conducted to improve the test items. Results from this study show that interpretations of the item parameter estimates from both psychometric approaches, CTT and IRT agree on most of the recommendations for the items. The items where the two approaches provide contrasting recommendations call for a further qualitative evaluation of the flagged items putting the teacher in a judicious position to discern which recommendations to follow regarding the concerned items. In the context of this study, an example of a guiding insight in deciding would be reflecting on the learning objectives measured by the test and the context in which the test was created.

Examining the Instructional Sensitivity of Constructed-Response Achievement Test Item Scores.

Inferences about student learning from large-scale achievement test scores are fundamental in education. For achievement test scores to provide useful information about student learning progress, differences in the content of instruction (i.e., the implemented curriculum) should affect test-takers' item responses. Existing research has begun to identify patterns in the content of instructionally sensitive multiple-choice achievement test items. To inform future test design decisions, this study identified instructionally (in)sensitive constructed-response achievement items, then characterized features of those items and their corresponding scoring rubrics. First, we used simulation to evaluate an item step difficulty difference index for constructed-response test items, derived from the generalized partial credit model. The statistical performance of the index was adequate, so we then applied it to data from 32 constructed-response eighth-grade science test items. We found that the instructional sensitivity (IS) index values varied appreciably across the category boundaries within an item as well as across items. Content analysis by master science teachers allowed us to identify general features of item score categories that show high, or negligible, IS.

학교상담자의 학교부적응 학생상담 어려움과 지원 방안 연구

Objectives The purpose of this study is to examine the challenges faced by school counselors when providing counseling to students experiencing school maladjustment, and to explore support strategies to address these difficulties. Methods From August 1 to September 3, 2023, a survey was conducted with a sample of 54 school counselors selected through random sampling. The survey data were analyzed using SPSS WIN 22.0, with frequency (N) and percentage (%) calculated for the multiple-choice items. Additionally, six participants were recruited for in-depth one-on-one interviews conducted via the online platform ZOOM. The qualitative data were analyzed following Krippendorff’s (2003) content analysis procedures, resulting in the conceptualization and categorization of partic ipants’ experiences into 2 domains, 8 categories, and 15 subcategories. This study employed a mixed-methods approach, integrating quantitative and qualitative research for an in-depth analysis. Results The content analysis identified 2 main domains, 8 categories, and 16 subcategories. The major challenges in counseling students with school maladaptation are as follows: First, institutional limitations in the school coun seling system make it difficult to provide flexible responses to specific situations, and the lack of awareness among school staff about counseling hinders the counseling process. Second, maintaining the counselor's pro fessional competence while managing psychological burdens is a significant issue, as the concentration of re sponsibility and unrealistic expectations of counseling outcomes lead to feelings of helplessness among counselors. Third, the complexity and diversity of students' issues require tailored approaches and responses to involuntary counseling, with a lack of cooperation from teachers and parents being a further concern. Strengthening counselor support systems and establishing collaborative networks are essential to addressing these challenges. Conclusions This study holds significance in exploring the challenges faced by school counselors and proposing support strategies through institutional improvements and the need for external resource linkage. These findings may serve as practical improvements to enable school counselors to more effectively support students struggling with school adjustment and to alleviate their workload.

The Impact of the COVID-19 Pandemic on Work-Life Integration of Physician Assistants in Oncology

Introduction: The COVID-19 pandemic led to rapid changes in the delivery of oncology care. Studies examining the impact of the pandemic on the oncology workforce are largely limited to oncologists and nurses. This study was conducted to explore how the COVID-19 pandemic impacted the oncology physician assistant (PA). Methods: A survey of oncology PAs was conducted in the fall of 2020. Multiple choice items and two open-ended questions explored how the COVID-19 pandemic may have impacted clinical practice. Burnout was assessed using the Maslach Burnout Inventory. Results: Most participants worked in an academic center (63.7%), in medical oncology (73.3%), and in the outpatient setting (70.5%). Telemedicine was performed by 77.5% of PAs, and 34.7% of PAs reported being assigned to help cover other departments/specialties. Physician assistants performing telemedicine were found to have higher rates of burnout compared to those that did not perform telemedicine (47.3% vs. 15.6%; p = .0013). Surprisingly, burnout was significantly lower for PAs who were redeployed during the pandemic compared to those who were not (28.0% vs. 46.8%; p = .0285). There was no correlation in the rates of burnout based on changes in hours worked, base pay, bonus pay, continuing medical education funding, or working remotely. Conclusion: The COVID-19 pandemic resulted in significant operational and workforce changes, which dramatically impacted the oncology PA. As the health-care landscape continues to adjust following the COVID-19 pandemic, future research should focus on the delivery of telemedicine to help identify opportunities to optimize this aspect of clinical practice and minimize the risk of burnout.

Assessing misconceptions in astronomy: The use of ordered multiple-choice items

This study pilots a unique test item known as the ordered multiple-choice (OMC) item. These OMC items were administered to two high school astronomy classrooms participating in a &lt;i&gt;NASA classroom of the future: Astronomy village&lt;/i&gt; program. The OMC items were included on a pre- and post-test to assess common misconceptions in astronomy, as part of study employing a quasi-experimental design. Each answer choice in an OMC item is linked to varying levels of student understanding, allowing for diagnostic interpretation of student responses. Results from the items indicated that although students did improve in their levels of understanding at the end of the program, there were still gaps in the students’ knowledge. We anticipate these items will allow for a unique and comprehensive assessment of student understanding of science.