General Chemistry Exam Research Articles

Statistical Comparison between In-Person and Online General Chemistry Exam Outcomes: A COVID-Induced Case Study

Statistical Comparison between In-Person and Online General Chemistry Exam Outcomes: A COVID-Induced Case Study

Investigating the Use of an Artificial Intelligence Chatbot with General Chemistry Exam Questions

Investigating the Use of an Artificial Intelligence Chatbot with General Chemistry Exam Questions

Evaluating and Improving Questions on an Unproctored Online General Chemistry Exam

Evaluating and Improving Questions on an Unproctored Online General Chemistry Exam

General Chemistry in Just One Semester for All Majors

We have successfully developed a one-semester general chemistry course that meets the primary learning goals of the traditional two-semester sequence. It is appropriate for physical, life, and earth science majors. Since implementation, 88% of students who start general chemistry complete it, an increase from 72% in our prior traditional curriculum. Scores on a full-year ACS general chemistry exam are almost the same, as are attitudes as measured by the CLASS-Chemistry instrument. Thus, one-semester general chemistry is more efficient, allowing students to move on to additional STEM courses quicker without impacting learning of core content. By increasing completion rates through general chemistry, we have removed a key barrier for students majoring in STEM fields. As well, students now can proceed through organic chemistry and biochemistry by the end of their second year, consistent with the increased emphasis on the chemistry of biological systems for the life sciences and in admissions to medical school. This reform can be implemented by schools that value the traditional quantitative and broad content of general chemistry without modifying later courses or disrupting other departments.

Evaluation of Subset Norm Stability in ACS General Chemistry Exams

The ability to assess students’ content knowledge and make meaningful comparisons of student performance is an important component of instruction. ACS exams have long served as tools for standardized assessment of students’ chemistry knowledge. Because these exams are designed by committees of practitioners to cover a breadth of topics in the curriculum, they may contain items that an individual instructor may not cover during classroom instruction and therefore chooses not to assess. For the instructor to make meaningful comparisons between his or her students and the national norm sample, the instructor needs norms that are generated upon the basis of the subset of items he or she used. The goal of this project was to investigate the effects of norm stability when items were removed from ACS General Chemistry Exams. This was achieved by monitoring the average change in percentile for students as items were removed from the exam and noting when average change crossed a specified threshold. An exploration of subset norm stability for three commonly used ACS General Chemistry Exams is presented along with implications for research and instruction.

Development and use of a multiple-choice item writing flaws evaluation instrument in the context of general chemistry

Multiple-choice (MC) exams are common in undergraduate general chemistry courses in the United States and are known for being difficult to construct. With their extensive use in the general chemistry classroom, it is important to ensure that these exams are valid measures of what chemistry students know and can do. One threat to MC exam validity is the presence of flaws, known as item writing flaws, that can falsely inflate or deflate a student's performance on an exam, independent of their chemistry knowledge. Such flaws can disadvantage (or falsely advantage) students in their exam performance. Additionally, these flaws can introduce unwanted noise into exam data. With the numerous possible flaws that can be made during MC exam creation, it can be difficult to recognize (and avoid) these flaws when creating MC general chemistry exams. In this study a rubric, known as the Item Writing Flaws Evaluation Instrument (IWFEI), has been created that can be used to identify item writing flaws in MC exams. The instrument was developed based on a review of the item writing literature and was tested for inter-rater reliability using general chemistry exam items. The instrument was found to have a high degree of inter-rater reliability with an overall percent agreement of 91.8% and a Krippendorff Alpha of 0.836. Using the IWFEI in an analysis of 1019 general chemistry MC exam items, it was found that 83% of items contained at least one item writing flaw with the most common flaw being the inclusion of implausible distractors. From the results of this study, an instrument has been developed that can be used in both research and teaching settings. As the IWFEI is used in these settings we envision an improvement in MC exam development practice and quality.

Investigation of the Effect of Peer-Led Team Learning Model on University Students’ Exam Achievement in General Chemistry

The purpose of this study is to compare freshmen engineering students’ exam achievement in general chemistry course using peer-led team learning model and traditional instruction and to investigate the effect of the methods on different achieving level students. A total of 128 students who were instructed by the same professor participated in the study. Both in PLTL and traditional class consisted of different engineering disciplines. Throughout the semester, PLTL model was implemented in the experimental group while traditional instruction was used in the control group. 14 peer leaders who have successfully completed general chemistry course in previous semester have been trained on basic communication and pedagogy issues and they have supported their teams in problem-solving throughout the semester. Both experimental and control group students took the same mid-terms and final exam which are constructed by the related department. The data were analyzed by two-way analysis of variance (ANOVA). Results indicated that low and medium achieving students’ general chemistry exam grades are statistically higher in PLTL group than those of traditional group students; however, there is no statistically significant mean difference in high achieving students’ grades across the groups.

ConfChem Conference on Select 2016 BCCE Presentations: Radical Awakenings—A New Teaching Paradigm Using Social Media

This Communication summarizes one of the invited papers to the Select 2016 BCCE Presentations ACS CHED Committee on Computers in Chemical Education online ConfChem held from October 30 to November 22, 2016. The ConfChem paper (included within Supporting Information) focuses on the results of one instructor’s incorporation of social media into her classroom and her teaching life as both a communication tool and a method for building a community of learners among students. The instructor implemented multivariate statistical analyses to determine the correlational relationship between students’ social media usage and their performance shown by their earned grade on a 40-question final exam similar to the ACS Exams Institute First Semester General Chemistry exam. Initial findings indicated no statistically significant change in pass rate or persistence between models in introductory General Chemistry I classes; however, the potential for a significant difference between models exists in next level (General Ch...

Analyzing the Role of Science Practices in ACS Exam Items

Recent efforts to reform K–12 science curricula, embedded within the NRC Framework for K–12 Science Education and the Next Generation Science Standards, have focused on unifying core disciplinary content with crosscutting concepts that span across science disciplines and scientific practices. With these reforms comes the challenge of creating “three-dimensional” assessments that measure beyond basic content knowledge mastery to incorporate measures of science practices and crosscutting concepts. Adding measures of science practices to traditional forms of assessments may prove to be challenging. The work herein investigates how science practices have been inherently incorporated into previously released ACS exams through analysis and classification of ACS general chemistry exam items with a developed rubric. The findings may help to inform the creation of new assessment items that are more explicit in their incorporation of science practices by building upon the implicit incorporation of science practices...

Impact of the Flipped Classroom on Student Performance and Retention: A Parallel Controlled Study in General Chemistry

Despite much recent interest in the flipped classroom, quantitative studies are slowly emerging, particularly in the sciences. We report a year-long parallel controlled study of the flipped classroom in a second-term general chemistry course. The flipped course was piloted in the off-semester course in Fall 2014, and the availability of the flipped section in Spring 2015 was broadly advertised prior to registration. Students self-selected into the control and flipped sections, which were taught in parallel by the same instructor; initial populations were 206 in the control section, 117 in the flipped. As a pretest, we used the ACS first-term general chemistry exam (form 2005), given as the final exam across all sections of the first-term course. Analysis of pretest scores, student percentile rankings in the first-term course, and population demographics indicated very similar populations in the two sections. The course designs required comparable student effort, and five common exams were administered, in...

Design, Implementation, and Evaluation of a Flipped Format General Chemistry Course

Research has consistently shown that active problem-solving in a collaborative environment supports more effective learning than the traditional lecture approach. In this study, a flipped classroom format was implemented and evaluated in the chemistry majors’ sequence at Purdue University over a period of three years. What was formerly lecture material was presented online outside of class time using videos of PowerPoint lectures with a voice-over. In-class time was devoted to collaborative problem solving based on the Student Centered Active Learning Environment with Upside-Down Pedagogies (SCALE-UP) approach, which has been shown to increase student exam scores and passing rates. The purpose of this proof-of-principle study was to look at quantitative measures of student performance in the flipped versus the traditional lecture approach of this chemistry course. Three years of results using ACS standardized exams showed that students’ ACS general chemistry exam scores in the flipped class were significa...

Building a Database for the Historical Analysis of the General Chemistry Curriculum Using ACS General Chemistry Exams as Artifacts

As a discipline, chemistry enjoys a unique position. While many academic areas prepared “cooperative examinations” in the 1930s, only chemistry maintained the activity within what has become the ACS Examinations Institute. As a result, the long-term existence of community-built, norm-referenced, standardized exams provides a historical artifact about the nature of content coverage in courses that stretches over decades. This work reports efforts to capture information and formulate it into a database about general chemistry content coverage over the past 20 years. Roughly 2000 items have been characterized in several ways, including (i) content, using an Anchoring Concepts Content Map; (ii) item construct, such as the presence of symbolic or visual information; and (iii) cognitive processing required, in terms of recall, algorithmic, or conceptual thinking.

Problem Solving Videos for General Chemistry Review: Students’ Perceptions and Use Patterns

We examined the use of problem solving videos (PSVs) as a substitute for general chemistry exam review sessions. We investigated student perceptions of course aspects regarding usefulness for supporting their learning of chemistry content. We also examined how students used the PSVs to further their learning. Students ranked the PSVs as more useful than other course components such as the textbook and exam review sessions. Weekly lectures, back tests, and online homework assignments were also ranked highly. A majority of our students viewed the PSVs, multiple views were common, and PSVs were mainly viewed for exam preparation. YouTube statistics corroborated this finding in that numbers of PSV views were maximal 4 days prior to and through each exam day. Students found step-by-step explanations and ability to control their own learning to be more helpful features of the PSVs. Shifting content to out-of-class video viewing frees up in-class time for tasks that promote learning at the higher levels of Bloom...

Examining Evidence for External and Consequential Validity of the First Term General Chemistry Exam from the ACS Examinations Institute

Validity of educational research instruments and student assessments has appropriately become a growing interest in the chemistry education research community. Of particular concern is an attention to the consequences to students that result from the interpretation of assessment scores and whether those consequences are swayed by invalidity within an assessment. This study examines external and consequential validity of a first-term general chemistry exam from the ACS Examinations Institute. The measure was used as a final exam in the intended course at the research setting. As a result, student performance on the exam contributed to the grading decision in the course. Owing to the prerequisite relationship, this grading decision then informed the decision to permit student enrollment in second-semester general chemistry. This study evaluates the appropriateness of the first-term general chemistry exam to inform this decision. Results indicate that the exam offers meaningful information into students' performance in the follow-on course with a consistent decision process across student subgroups.

Adaptation of an Instrument for Measuring the Cognitive Complexity of Organic Chemistry Exam Items

Experts use their domain expertise and knowledge of examinees’ ability levels as they write test items. The expert test writer can then estimate the difficulty of the test items subjectively. However, an objective method for assigning difficulty to a test item would capture the cognitive demands imposed on the examinee as well as be assignable by any domain expert familiar with the examinee group. One such instrument for assigning objective complexity of general chemistry exam items has already been reported. A revised instrument for assigning objective complexity of organic chemistry exam items is presented including the reliability and validity studies of the rubric.

Analysis of Student Performance on Multiple-Choice Questions in General Chemistry

The percentage of students choosing the correct answer (PSCA) on 17 multiple-choice algorithmic questions taken from general chemistry exams is analyzed. PSCAs for these questions varied from 47 to 93%, and a decrease of 4.5% in PSCA was observed with each additional step in the algorithm required for solving the problem (R2 = 0.80). Variants of a subset of these questions were also examined and reveal the effect of making changes to a particular algorithmic question on the ability of students to choose the correct answer.

An expanded framework for analyzing general chemistry exams

This paper describes an expanded framework to aid chemical educators in constructing exams for their courses. The framework has three primary levels: definition, algorithmic, and conceptual. These primary levels have often been used in chemical education research to analyze and describe exam questions, but in this study the definition, algorithmic, and conceptual primary levels have expanded secondary levels.

Investigating the relationship between faculty cognitive expectations about learning chemistry and the construction of exam questions

We have investigated chemistry faculty’s cognitive expectations about learning chemistry and their influence upon the construction of exam questions in a general chemistry curriculum. Faculty cognitive expectations for learning chemistry were measured using QUIMX. Learning objectives and exam questions for a year-long general chemistry sequence were classified according to Bloom’s Cognitive Taxonomy and the Expanded Framework for Analyzing General Chemistry Exams. Analyses revealed correlations between learning objectives and the type of exam questions. Faculty cognitive expectations correlate with the algorithmic or conceptual nature of exam. The findings suggest that incorporating more conceptual assessment requires faculty to articulate explicitly their cognitive expectations and integrate them into the writing of learning objectives.

Does the Answer Order Matter on Multiple-Choice Exams?

Surprising version-dependent differences are noted in student performance on certain questions in a standardized general chemistry exam. The exam in question has two versions, on which both questions and answers are ordered differently. For the questions suspected of answer-order bias, the performance is better in ten of twelve cases when students see the correct answer earlier in the list of four choices. Expansion of the study to more abundant data for the same exam from the ACS DivCHED Examinations Institute supports the interpretation of these observations as a primacy effect, characteristic of guessing, but with an aversion for the first choice. However, the single problem that shows the greatest disparity between versions in both data sets is one most chemists would view as so simple as to make guessing unnecessary, hence unlikely as an explanation.

Predicting at-risk students in general chemistry: comparing formal thought to a general achievement measure

This study is an investigation into the ability of pre-assessment measures of formal thought ability and general achievement to predict students at-risk of poor performance in college- level general chemistry. Over a three year period, data on formal thought ability (as measured by the Test of Logical Thinking, or TOLT) and/or general achievement (as measured by the Scholastic Aptitude Test, or SAT) was collected from over 3000 students as they entered a general chemistry course. The outcome measure was an American Chemical Society general chemistry exam at the end of the course. Findings indicate that both the formal thought and the general achievement measure can successfully identify at-risk students in this setting, with neither measure being superior in doing so. The presence of distinct groups of students correctly predicted to be at-risk by only one of the measures demonstrates that formal thought ability and general achievement each represent an independent hindrance to success in chemistry. Therefore, efforts to help at-risk students should include a focus on the development of formal thought as well as a content review. [Chem. Educ. Res. Pract., 2007, 8 (1), 32-51]