Classroom Observation Protocol For Undergraduate STEM Research Articles

University students dataset related to achievement, classroom practices, perceptions and attitudes of multimedia-based learning quantum physics.

This dataset presents data collected to assess teaching and learning of quantum physics at the University of Rwanda - College of Education (UR-CE), Rwanda. Data were collected between August and November 2019 as the baseline, and between January and April 2022 under a quasi-experimental design. Three sets of data were collected. The first set was about students' performance and conceptual understanding collected before and after teaching intervention (lecture method or multimedia-aided approach) using mainly Quantum Physics Conceptual Survey (QPCS). The second set documented classroom practices during teaching and learning using the Classroom Observation Protocol for Undergraduate STEM (COPUS). The last set is comprised of the data related to lecturers' and students' perceptions before teaching and learning quantum physics and students' attitudes after learning Quantum physics. The Quantum Physics Attitude Test (QPAT) was mainly used to collect these data. The dataset is important to education stakeholders because university managers can visualize the status of teaching and learning outcomes, lecturers can reflect on the study, and researchers can use the data to analyze various independent variables.

Iterating toward change: Improving student-centered teaching through the STEM faculty institute (STEMFI).

One of the primary reasons why students leave STEM majors is due to the poor quality of instruction. Teaching practices can be improved through professional development programs; however, several barriers exist. Creating lasting change by overcoming these barriers is the primary objective of the STEM Faculty Institute (STEMFI). STEMFI was designed according to the framework established by Ajzen's Theory of Planned Behavior. To evaluate its effectiveness, the Classroom Observation Protocol for Undergraduate STEM (COPUS) tool was used before and after an intensive year-long faculty development program and analyzed using copusprofiles.org, a tool that classifies each COPUS report into one of three instructional styles: didactic, interactive lecture, and student-centered. We report the success of our program in changing faculty teaching behaviors and we categorize them into types of reformers. Then, thematically coded post-participation interviews give us clues into the characteristics of each type of reformer. Our results demonstrate that faculty can significantly improve the student-centeredness of their teaching practices in a relatively short time. We also discuss the implications of faculty attitudes for future professional development efforts.

University students dataset related to achievement, classroom practices, perceptions and attitudes of multimedia-based learning quantum physics

This dataset presents data collected to assess teaching and learning of quantum physics at the University of Rwanda - College of Education (UR-CE), Rwanda. Data were collected between August and November 2019 as the baseline, and between January and April 2022 under a quasi-experimental design. Three sets of data were collected. The first set was about students' performance and conceptual understanding collected before and after teaching intervention (lecture method or multimedia-aided approach) using mainly Quantum Physics Conceptual Survey (QPCS). The second set documented classroom practices during teaching and learning using the Classroom Observation Protocol for Undergraduate STEM (COPUS). The last set is comprised of the data related to lecturers' and students' perceptions before teaching and learning quantum physics and students' attitudes after learning Quantum physics. The Quantum Physics Attitude Test (QPAT) was mainly used to collect these data. The dataset is important to education stakeholders because university managers can visualize the status of teaching and learning outcomes, lecturers can reflect on the study, and researchers can use the data to analyze various independent variables.

Instructors and Students’ Practices and Behaviours during a Quantum Physics class at the University of Rwanda: Exploring the Usage of Multimedia

This study was aimed at exploring the usage of multimedia during a quantum physics class. Five instructors and 385 undergraduate students at the University of Rwanda, College of Education (UR-CE) were observed and surveyed. Thus, the study employed experimental and survey designs. A standardized classroom observation protocol for undergraduate STEM (COPUS) and a validated online survey were used. Classroom observation data were analysed quantitatively using an M.S. Excel spreadsheet, and interpreted descriptively. Likewise, survey data were analysed qualitatively using a note-pencil, and interpreted narratively. The class in which a multimedia method was used, showed more active learning compared to one in which lecturing was used. The findings indicated that instructors (lecturers) were guiding students, and the students were working in the multimedia class. In the lectured class, instructors were found presenting the content to students and students received information passively. Instructors identified the mathematical background as the trigger to students’ negative attitude towards learning quantum physics when they were encouraged to learn through animations, PhET simulations, and YouTube videos. The study recommends the use of multimedia technologies in teaching quantum physics-related concepts.

COPUS, PORTAAL, or DART? Classroom Observation Tool Comparison From the Instructor User’s Perspective

Classroom observation tools are used to evaluate teaching and learning activities, and to provide constructive feedback to instructors. To help instructors with selecting a suitable tool based on their needs and available resources, in this study, a group of observers assessed lectures of an introductory biology course using three, broadly cited classroom assessment tools in the STEM field: the Classroom Observation Protocol for Undergraduate STEM (COPUS); the Practical Observation Rubric to Assess Active Learning (PORTAAL); and the Decibel Analysis for Research in Teaching (DART). From a user’s perspective, we evaluated 1) the type and extent of information each tool provides, and 2) the time investment and difficulty of working with each tool. The assessment result of each tool was compared, with a list of expected outcomes generated by surveying a group of college instructors and with the result of a self-teaching assessment tool, Teaching Practices Inventory (TPI). Our findings conclude that each tool provided valuable assessment with a broad range of outcomes and time investment: PORTAAL offered the most detailed information on the quality of teaching practices and students’ engagement, but it demanded the greatest time investment. DART provided a basic estimation of active learning proportion with the least effort. The level of assessment outcome and the time investment when using COPUS was found to be less than PORTAAL, and more than DART. The TPI self-assessment outcome was found to be slightly optimistic regarding the proportion of active learning practices used in the studied course. This comparative study can help instructors in selecting a tool that suits their needs and available resources for a better assessment of their classroom teaching and learning.

Are Faculty Changing? How Reform Frameworks, Sampling Intensities, and Instrument Measures Impact Inferences about Student-Centered Teaching Practices.

Although recent studies have used the Classroom Observation Protocol for Undergraduate STEM (COPUS) to make claims about faculty reform, important questions remain: How should COPUS measures be situated within existing reform frameworks? Is there a universal sampling intensity that allows for valid inferences about the frequency of student-centered instruction within a semester or across semesters of a course? These questions were addressed using longitudinal COPUS observations (128 classes, three faculty, 4 years). COPUS behaviors were used to categorize classes into didactic, interactive lecture, or student-centered instructional styles. Sampling intensities (one to 11 classes) were simulated (1000 times) within a course and across semesters. The sampling intensities required for generating valid inferences about 1) the presence of student-centered instruction and 2) the proportion of instructional styles in a course and through time were calculated. Results indicated that the sampling intensity needed to characterize courses and instructors varied and was much higher than previously recommended for instructors with: 1) rare instances of student-centered classes, 2) variability in instructional style, and 3) longitudinal changes in instructional patterns. These conditions are common in early reform contexts. This study highlights the risks of broad, decontextualized sampling protocol recommendations and illustrates how reform frameworks, sampling intensities, and COPUS measures interact to impact inferences about faculty change.

How Pre-service Teachers Learn Microbiology using Lecture, Animations, and Laboratory Activities at one Private University in Rwanda

Observing classroom practices and checking the effect of instructional methods on academic achievement are crucial in the teaching and learning process. The present study was aimed at discovering the dominating pre-service biology teachers’ and instructors’ activities in microbiology classes and their respective effects when animations–based instructions and small-group laboratory activities are used. An equivalent time-series design was applied using a small group of participants in year two biology education (N=30, 16 female and 14 males), and a pre-test was used as a pre-intervention comparison test, while a post-test alternated with interventions. Classroom Observation Protocol for Undergraduate STEM (COPUS) was used to record classroom activities. Before using its inter-rater agreement reached 80%. Pre-service Biology Teachers Achievement Test (PBTAT) with a Pearson’s r reliability of .51 served to measure instructional methods' effect on academic achievement. It was found that the main teaching methods were activities, lectures and animation classes, while group work and instructors moving among the students and guiding them characterized small-group laboratory activity classes. All interventions improved pre-service biology teachers’ academic achievement; however, a statistically significant difference (df=28, p<.05) existed between interventions where small group laboratory activities proved a considerable effect size (d=3.86). No statistically significant difference (df=1, p>.05) was found regarding gender after interventions. However, females scored better than males after the lecture and laboratory methods, while the opposite happened after animation-based instruction. Therefore, we recommend using small-group laboratory activities that promote active learning through student small-group work to improve pre-service biology teachers’ academic achievement in biology.

Interpret with Caution: COPUS Instructional Styles May Not Differ in Terms of Practices That Support Student Learning.

There is a growing need for valid and reliable measures to monitor the efficacy of undergraduate science, technology, engineering, and mathematics (STEM) reform initiatives. The Classroom Observation Protocol for Undergraduate STEM (COPUS) is a widely used tool originally designed to measure the presence of overt instructor and student behaviors. It has subsequently been used to characterize instruction along a continuum from didactic to student centered, and more recently to categorize instruction into one of three styles. Initiatives focused on professional development often support instructors’ progression from didactic to student-centered styles. There is a need to examine COPUS instructional styles in terms of behaviors that research has shown to improve student learning. Formative assessment is a research-based practice that involves behaviors accounted for by the COPUS (e.g., posing a question). We qualitatively compared the formative assessment behaviors in 16 biology class sessions categorized into each of the three COPUS styles. We were unable to detect differences in formative assessment behaviors between the COPUS styles. Caution should be taken when interpreting COPUS data to make inferences about the effects of reform efforts. This study underscores the need for additional measures to monitor national reform initiatives in undergraduate STEM.

A case study on chemistry classroom practices in the Rwandan secondary schools

This study aims at assessing the classroom practices of Rwandan chemistry teachers in secondary schools, the second grade (S2), age range between 14 to 16 years old. The classroom observation conducted using the Classroom Observation Protocol for Undergraduate STEM (COPUS). In general, the best practice of group work or collaborative learning and students' engagement was observed. The analysis of the COPUS data reveals that active learning in chemistry classes is dominating, 54% against 42% of passive learning; found high, statistically significant over passive learning at p < .01 (tCritical = 1.89, df = 7, p = .003). The active students' practice is taking 82% of a 2-min time interval across 30 lessons observed, as one class period has 40 min to make 20 intervals of 2 min. Teachers are engaging their students in collaborative learning by assigning them various tasks in groups. These results established the current instructional practices in chemistry classes to draw conclusions and recommendations.

Study Of Faculty Instructors In Undergraduate Classroom And Planetarium Learning Environments

A mixed-methods study exploring the undergraduate planetarium learning environment was conducted during the 2019--2020 academic year at a western American university. Survey responses from university faculty, observational data using the Classroom Observation Protocol for Undergraduate STEM (COPUS), and faculty interview responses were collected and analyzed to investigate how and why collegiate undergraduates were being instructed in a planetarium environment and how this environment compared to a traditional classroom counterpart. Results suggest that planetarium use is viewed by instructors as an integrated learning experience with the classroom environment, with affective learning outcomes in the planetarium complemented by cognitive learning outcomes in the classroom. COPUS observations of planetarium instruction show broad similarity to classroom instruction; however, reductionsin active-learning behavior archetypes measured in the planetarium environment suggest a trade-off between interactive learning strategies and visually immersive content presentation. Implications concerning the collegiate planetarium environment and future work are discussed.

Measuring institutional transformation: a multifaceted assessment of a new faculty development program

PurposeThis study aims to understand the extent to which a faculty development program that includes a week-long course design experience followed by sustained support changes new faculty's perceptions, beliefs and teaching practices. The authors employed the teacher professional knowledge and skill (TPK&amp;S) framework and characteristics of effective educational development interventions to drive the program development, implementation and assessment.Design/methodology/approachThis study utilized a mixed methods approach. Data sources include pre-/mid-/post-program responses to a validated survey, pre-/post-program course syllabi analyzed using a validated rubric and pre-/post-classroom observations collected using the Classroom Observation Protocol for Undergraduate STEM (COPUS) instrument.FindingsFindings indicate transformative effects for participants' beliefs about their teaching and changes to their instructional practices. Significant and practical effects were observed across different portions of the program for increases in participants' self-efficacy, endorsement of a conceptual change approach toward teaching and perceptions of institutional support. Participants produced more learning-focused syllabi and many moved toward more student-centered instructional approaches in their teaching practices.Research limitations/implicationsDue to the voluntary nature of the new faculty development program, this study may have been limited by participant self-selection bias and differential sample sizes for the study's individual measures. Future research should consider designs which maximize faculty participation in measurement across all data sources.Originality/valueThis study addresses shortcomings in prior studies which utilized limited data sources to measure intervention impact and answers the call for more rigorous research to obtain a more complete picture of instructional development in higher education.

Comparison of Cluster Analysis Methodologies for Characterization of Classroom Observation Protocol for Undergraduate STEM (COPUS) Data

The Classroom Observation Protocol for Undergraduate STEM (COPUS) provides descriptive feedback to instructors by capturing student and instructor behaviors occurring in the classroom. Due to the increasing prevalence of COPUS data collection, it is important to recognize how researchers determine whether groups of courses or instructors have unique classroom characteristics. One approach uses cluster analysis, highlighted by a recently developed tool, the COPUS Analyzer, that enables the characterization of COPUS data into one of seven clusters representing three groups of instructional styles (didactic, interactive, and student centered). Here, we examine a novel 250 course data set and present evidence that a predictive cluster analysis tool may not be appropriate for analyzing COPUS data. We perform a de novo cluster analysis and compare results with the COPUS Analyzer output and identify several contrasting outcomes regarding course characterizations. Additionally, we present two ensemble clustering algorithms: 1) k-means and 2) partitioning around medoids. Both ensemble algorithms categorize our classroom observation data into one of two clusters: traditional lecture or active learning. Finally, we discuss implications of these findings for education research studies that leverage COPUS data.

Using the Reasoned Action Approach to Predict Active Teaching Behaviors in College STEM Courses

Active learning can have a substantial impact on STEM student learning outcomes. This study used the reasoned action approach to predict the presence of active teaching techniques. Instructor attitudes, perceived social norms, perceived behavioral control and intentions concerning active learning strategies were assessed before and after a teaching workshop, in addition to retrospectively. Behavioral teaching practices were recorded using the Classroom Observation Protocol for Undergraduate STEM (COPUS). Results indicated that instructor attitudes toward and perceived behavioral control over implementing active learning significantly contributed to STEM instructor intentions to use active learning. Further, instructor intentions assessed after the workshop and retrospectively were predictive of instructor behavior. Instructors with high intentions exhibited an increase in ‘active’ instructor behavior (group work) and a decrease in ‘passive’ instructor behavior (lecture). These results suggest that whether instructors view active learning as favorable or unfavorable, and whether instructors feel they have control over implementing active learning strategies, is critical to increasing the presence of active teaching intentions and behaviors. Professional development programs wishing to create instructional change should target instructor attitude and control beliefs, as well as discuss normative teaching practices in aim of contributing to instructors’ understanding of and future intentions to implement active learning.

Analysis of Teaching Techniques and Scheme of Work in Teaching Chemistry in Rwandan Secondary Schools

This study aims at determining the effect of the pedagogical technique most frequently used while implementing the competency-based curriculum (CBC) in teaching chemistry in senior two (S2) classes (age range 14-16) and the teachers’ alignment with the scheme of work. It was conducted in 10 selected Rwandan secondary schools using a teacher survey, teacher interviews and the Classroom Observation Protocol for Undergraduate STEM (COPUS) in the period from 27 May to 8 July 2019. The result of the analysis reveals that teaching chemistry using group work-related techniques that engage more learners does not have any relationship to teachers moving slowly compared to the scheme of work of S2 chemistry. Therefore, this study recommends more support for teachers to use active learning techniques in teaching chemistry in S2 for the effective implementation of the CBC.

I’ve Been Given COPUS (Classroom Observation Protocol for Undergraduate STEM) Data on My Chemistry Class... Now What?

Classroom observation data using tools such as the Classroom Observation Protocol for Undergraduate STEM (COPUS) are frequently collected in instructional settings for large-scale research studies ...

Development and Validation of Poksd-Porsd Protocol Assessment of Engineering in Elementary Classroom

The aim of this study was to develop and validate classroom observation instruments designed to reveal the emergence of engineering activities in primary school teachers in project-based learning. The instruments developed included the elementary school classroom observation protocol sheet (POKSD) and the elementary school engineering observation protocol assessment (PORSD). Task items were arranged based on indicators adapted from COPUS (Classroom Observation Protocol for Undergraduate STEM) items. The initial design of the instrument was consulted with three experts based on learning objectives. The instrument was then validated by three experts in the field of basic education. The instrument test was conducted on teachers and 5th-grade students of UPI Bandung Laboratory (N = 1). POKSD and PORSD were assessed by three raters. Scores from the three raters were then analyzed using two-way ANOVA. The results showed that the intra-class correlation of performance assessment instruments was adequate (ICC = 0.773). The findings of this study demonstrated that the instrument was reliable and could be used for the emergence of engineering activities in elementary school teachers.

Sustained mentorship promotes the development of active learning strategies in undergraduate biology classrooms: Evidence gained from the Promoting Active Learning and Mentoring (PALM) Network

Here we report on the work of the Promoting Active Learning and Mentoring Network (PALM), a National Science Foundation‐supported project started in 2016 and dedicated to leveraging the expertise found in life science professional societies to provide long‐term mentorship in developing active learning techniques in the classroom. PALM mentees work one‐on‐one with mentors that are seasoned in active learning strategies and travel to their mentor's institution to observe classes, plan modifications for their own classes, and strategize the best active learning techniques that are suited to their class' particular needs. A monthly online journal club and regular opportunities to meet in conjunction with the Society for the Advancement of Biology Education Research (SABER) and ASCB meetings provide a dynamic teaching and learning community for mentees and mentors alike. After their mentorship, mentees are encouraged to work with their colleagues on active learning strategies and to become a PALM mentor to further the spread of effective active learning strategies to an ever‐wider audience.We have assessed the efficacy of PALM mentorship through the use of the Classroom Observation Protocol for Undergraduate STEM (COPUS) protocol (Smith et al., 2013) on pre‐mentorship and post‐mentorship videos of mentee teaching. The COPUS results have revealed a consistent shift in mentee teaching practices from a strict focus on lecture to a more diffuse combination of classroom activities in which the student is more directly active and engaged in the class and the instructor is more directly engaging students in a more personalized manner. While the specific changes in classroom practice vary between individual mentees due to the distinct needs and environments of each class, there is a consistent picture of more student‐centered classrooms that are more engaging and dynamic for both student and instructor. Overall, the COPUS data suggests that the long‐term mentoring strategies brought forth by PALM are an effective and sustainable means to promote the development of active learning techniques among undergraduate biology instructors.Support or Funding InformationNational Science Foundation #1624200This abstract is from the Experimental Biology 2019 Meeting. There is no full text article associated with this abstract published in The FASEB Journal.

What Types of Instructional Shifts Do Students Experience? Investigating Active Learning in Science, Technology, Engineering, and Math Classes across Key Transition Points from Middle School to the University Level

Despite the need for a strong Science, Technology, Engineering, and Math (STEM) workforce, there is a high attrition rate for students who intend to complete undergraduate majors in these disciplines. Students who leave STEM degree programs often cite uninspiring instruction in introductory courses, including traditional lecturing, as a reason. While undergraduate courses play a critical role in STEM retention, little is understood about the instructional transitions students encounter upon moving from secondary to post-secondary STEM courses. This study compares classroom observation data collected using the Classroom Observation Protocol for Undergraduate STEM (COPUS) from over 450 middle school, high school, introductory-level university, and advanced-level university classes across STEM disciplines. We find similarities between middle school and high school classroom instruction, which are characterized by a large proportion of time spent on active-learning instructional strategies, such as small-group activities and peer discussion. In contrast, introductory and advanced university instructors devote more time to instructor-centered teaching strategies, such as lecturing. These instructor-centered teaching strategies are present in classes regardless of class enrollment size, class period length, or whether or not the class includes a separate laboratory section. Middle school, high school, and university instructors were also surveyed about their views of what STEM instructional practices are most common at each educational level and asked to provide an explanation of those perceptions. Instructors from all levels struggled to predict the level of lecturing practices and often expressed uncertainty about what instruction looks like at levels other than their own. These findings suggest that more opportunities need to be created for instructors across multiple levels of the education system to share their active learning teaching practices and discuss the transitions students are making between different educational levels.

Correction for The Classroom Observation Protocol for Undergraduate STEM (COPUS): A New Instrument to Characterize STEM Classroom Practices

Correction for The Classroom Observation Protocol for Undergraduate STEM (COPUS): A New Instrument to Characterize STEM Classroom Practices

The Classroom Observation Protocol for Undergraduate STEM (COPUS): A New Instrument to Characterize University STEM Classroom Practices

Instructors and the teaching practices they employ play a critical role in improving student learning in college science, technology, engineering, and mathematics (STEM) courses. Consequently, there is increasing interest in collecting information on the range and frequency of teaching practices at department-wide and institution-wide scales. To help facilitate this process, we present a new classroom observation protocol known as the Classroom Observation Protocol for Undergraduate STEM or COPUS. This protocol allows STEM faculty, after a short 1.5-hour training period, to reliably characterize how faculty and students are spending their time in the classroom. We present the protocol, discuss how it differs from existing classroom observation protocols, and describe the process by which it was developed and validated. We also discuss how the observation data can be used to guide individual and institutional change.