Classroom Observation Protocol Research Articles

An exploration of the relationship between active learning and student motivation in STEM: a mixed methods study.

Much of the research on science, technology, engineering, and mathematics (STEM) students' motivation measures the relationship between student motivation and academic outcomes, focusing on the student's mindset. Our mixed-methods research takes a different approach and considers the relationship between student motivation and instructional practices. Teaching practices and student motivation were analyzed simultaneously in undergraduate Biology classes using a self-determination theory-based survey to measure students' motivation during courses that were observed using the Classroom Observation Protocol for Undergraduate STEM (COPUS), and observation notes were collected to document instructor and student behaviors. Quantitative data were used to differentiate students' motivational levels, and qualitative data were collected to describe how instructors use specific teaching practices. The results provide a lens into how students' intrinsic motivation varies alongside the instructional practices and interactions in these classes. We found a correlation between higher levels of student motivation in interactive lectures and student-centered teaching profiles. This study highlights how the same practice can be implemented by multiple instructors with varying student motivation scores, pointing out the importance of fidelity to evidence-based instructional practice methods. The results of this study are discussed in the context of published empirical studies examining evidence-based instructional practices that are conceptually supportive of autonomy, competence, and relatedness. Active learning practices observed in this study correlated to positive learning outcomes are discussed and may serve as a guide for instructors interested in implementing specific active learning practices. Recommendations for instructors and departments that are interested in flexible methods to monitor progress toward active learning practices in biology and other STEM disciplines by combining the COPUS and self-determination survey results are presented.NEW & NOTEWORTHY This study uses a novel combination of instruments to describe students' intrinsic motivation in response to teaching practices. Findings demonstrate that active learning methods may support higher student motivation. Recommendations drawn from the study include using a variety of active learning methods, using evidence-based instructional methods with fidelity, and monitoring the students' affective response to those methods. Alignment of active learning practices to the components of self-determination may result in higher quality student motivation in science, technology, engineering, and mathematics (STEM) courses.

Undergraduate Students’ Opinion on Digital Learning in a Post-pandemic Era: A Case Study

"The current article aims to investigate learners’ post-pandemic attitude towards digital learning in Higher Education with a specific focus on a group of undergraduate Law students at Babes-Bolyai University from Cluj-Napoca, Romania. To achieve its purpose, the article focuses on answering two research questions: Do post pandemic undergraduate students recognize the benefits of digital learning? Is e-learning currently a reasonable choice for them? The research is designed as an exploratory case study that uses three data collection tools: a questionnaire, an interview, and a classroom observation protocol. The article links information from literature related to the use of digital learning during the pandemic with results revealed by the data collected in the case study. The results suggest a steady trend towards a growing acceptance rate of digital learning by the targeted student population and a lower approval rate of e-learning. Keywords: higher education, digital learning, online learning, pandemic, good teaching practices."

Seeing Through the Eyes of Classroom Observers: The Case of Rating Contrasted Groups of Lessons With Classroom Observation Measures

Classroom observations are commonly employed to assess quality of instruction in research and practice in mathematics education. However, there is more to be learned about how sensitive classroom observation protocols are to exemplars of strong mathematics instruction, and continuous refinements to observation protocols or rating processes that may be warranted. In this study, we use the public-released mathematics videos from the Third International Mathematics and Science Study (TIMSS) to examine how classroom observers, using two contemporary classroom observation instruments, rate a set of lessons whose instructional quality is in theory expected to differ, also referred to as contrasted groups. We find that descriptively, the pattern of findings is distinct from prior studies’ conclusions about the relative instructional quality reflected in the TIMSS video pool. We provide qualitative examples to illustrate the findings, and discuss implications for future research. We point to the potential value of exploring classroom observation rubrics’ performance using ‘contrasted groups’ of lesson videos, as a tool to broaden our understanding of how observation instruments are functioning.

University students dataset related to achievement, classroom practices, perceptions and attitudes of multimedia-based learning quantum physics.

This dataset presents data collected to assess teaching and learning of quantum physics at the University of Rwanda - College of Education (UR-CE), Rwanda. Data were collected between August and November 2019 as the baseline, and between January and April 2022 under a quasi-experimental design. Three sets of data were collected. The first set was about students' performance and conceptual understanding collected before and after teaching intervention (lecture method or multimedia-aided approach) using mainly Quantum Physics Conceptual Survey (QPCS). The second set documented classroom practices during teaching and learning using the Classroom Observation Protocol for Undergraduate STEM (COPUS). The last set is comprised of the data related to lecturers' and students' perceptions before teaching and learning quantum physics and students' attitudes after learning Quantum physics. The Quantum Physics Attitude Test (QPAT) was mainly used to collect these data. The dataset is important to education stakeholders because university managers can visualize the status of teaching and learning outcomes, lecturers can reflect on the study, and researchers can use the data to analyze various independent variables.

Structural Modeling and Measuring Impact of Active Learning Methods in Engineering Education

<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Contribution:</i> This article presents a novel approach that demonstrates how students’ learning can be improved by increasing classroom adherence to active learning (AL) application, in a typical engineering education (EE) environment. It does that by using classroom observation protocol data and student assessment grades analyzed by a statistical tool, representing seven engineering programs. <p xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <i>Background:</i> AL applications in EE have been growing in recent years and there have been relevant discussions about their effect on students learning. This research is grounded on a 2.5-year-long data collection process through objective measures, following a strict experimental design project. It differs from other studies where surveys are used for data collection. <p xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <i>Research Question:</i> Can differences in students’ learning between traditional teaching and AL methods be distinguished by means of classroom observation protocol measures coupled with partial least-square structural equation modeling? <p xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <i>Methodology:</i> An experimental research design was conducted in an Engineering Higher Education Institution, by taking independent measures of latent constructs’ indicators, such as student grades and AL classroom adherence levels. The data were subject to a partial least-squares structural equation modeling (PLS-SEM) approach for modeling, analysis, and validation. <p xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <i>Findings:</i> The results suggested a nonlinear positive cause-and-effect relationship between AL adherence and learning, validated by several performance indexes, such as a learning prediction relevance <inline-formula> <tex-math notation="LaTeX">$Q^{2}~\mathrm{predict}$</tex-math> </inline-formula> <inline-formula> <tex-math notation="LaTeX">$=$</tex-math> </inline-formula> 0.453 and goodness of fit <inline-formula> <tex-math notation="LaTeX">$=$</tex-math> </inline-formula> 0.588. The model demonstrated a learning score improvement from 45.89 to 74.90 as an effect of the adherence to AL score increase from 0 to 35.97.

Elementary science teachers’ engineering integration after long-term in-service training program with and without curriculum material support

ABSTRACT Background and Purpose As a response to the need for effective engineering integration in K-12 science classrooms, teachers should be supported with rich professional development (PD) with diverse components (e.g. curriculum material support). Grounded in the framework for quality K-12 engineering education, this study explores how elementary science teachers integrated engineering into science lessons after participating in PD and the nature of engineering integration with additional curriculum material support (CMS). Design/Method In this single-case study with multiple embedded units of analysis, three teachers’ 79 classroom video recordings were analyzed using the Classroom Observation Protocol for Engineering Design with the engineering design process and engineering habits of mind components. Findings Analysis showed that teachers spent time on each design component in both PD Year and CMS Year, with an increase for at least half of the components in the CMS Year. Across all the observations in both years, the most common engineering design process component was researching possible solutions to the engineering challenge. Compared to the PD Year, teachers’ engineering habits of mind integration improved greatly in the CMS Year. Regarding engineering design and science concepts integration, observations became more balanced, moving away from the independent ends of the continuum in the CMS Year. Conclusions Long-term PD was effective in supporting teachers’ integration of engineering design components. Providing CMS had the greatest impact on the integration of engineering habits of mind components. Along with PD, curriculum support with explicit indication of how and when to incorporate engineering with habits of mind components was beneficial for teachers.

Iterating toward change: Improving student-centered teaching through the STEM faculty institute (STEMFI).

One of the primary reasons why students leave STEM majors is due to the poor quality of instruction. Teaching practices can be improved through professional development programs; however, several barriers exist. Creating lasting change by overcoming these barriers is the primary objective of the STEM Faculty Institute (STEMFI). STEMFI was designed according to the framework established by Ajzen's Theory of Planned Behavior. To evaluate its effectiveness, the Classroom Observation Protocol for Undergraduate STEM (COPUS) tool was used before and after an intensive year-long faculty development program and analyzed using copusprofiles.org, a tool that classifies each COPUS report into one of three instructional styles: didactic, interactive lecture, and student-centered. We report the success of our program in changing faculty teaching behaviors and we categorize them into types of reformers. Then, thematically coded post-participation interviews give us clues into the characteristics of each type of reformer. Our results demonstrate that faculty can significantly improve the student-centeredness of their teaching practices in a relatively short time. We also discuss the implications of faculty attitudes for future professional development efforts.

Using an Observation Protocol To Evaluate Student Argumentation Skills in Introductory Biology Laboratories.

Argumentation is vital in the development of scientific knowledge, and students who can argue from evidence and support their claims develop a deeper understanding of science. In this study, the Argument-Driven Inquiry instruction model was implemented in a two-semester sequence of introductory biology laboratories. Student's scientific argumentation sessions were video recorded and analyzed using the Assessment of Scientific Argumentation in the Classroom observation protocol. This protocol separates argumentation into three subcategories: cognitive (how the group develops understanding), epistemic (how consistent the group's process is with the culture of science), and social (how the group members interact with each other). We asked whether students are equally skilled in all subcategories of argumentation and how students' argumentation skills differ based on lab exercise and course. Students scored significantly higher on the social than the cognitive and epistemic subcategories of argumentation. Total argumentation scores were significantly different between the two focal investigations in Biology Laboratory I but not between the two focal investigations in Biology Laboratory II. Therefore, student argumentation skills were not consistent across content; the design of the lab exercises and their implementation impacted the level of argumentation that occurred. These results will ultimately aid in the development and expansion of Argument-Driven Inquiry instructional models, with the goal of further enhancing students' scientific argumentation skills and understanding of science.

COPUS-TA: An "Entry-Level" Peer Observation Tool to Support Teaching Assistant Professional Pedagogical Development.

When coordinating teams of teaching assistants (TAs) in our courses, it can be time and resource prohibitive to provide mentorship for individual professional development of their teaching. Peer observation of teaching is a useful and effective approach for professional development and for forming a community of practice that TAs can engage in. However, structured peer observation can require substantial training - which may make it infeasible for large teams of TAs with variable teaching expertise and limited contract hours. This article describes the development of an observation protocol, adapted for the TA context, from the Classroom Observation Protocol for Undergraduate STEM (COPUS; by Smith et al. 2013). We have used this successfully, with very minimal training (15-min discussion plus one practice observation). I include the modified form (COPUS-TA) for practical and immediate use in supporting the development of TAs' teaching skills.

Developing an Integrated STEM Classroom Observation Protocol Using the Productive Disciplinary Engagement Framework.

The online version contains supplementary material available at 10.1007/s11165-023-10110-z.

Development and Validation of an Integrated STEM Teacher Classroom Observation Protocol

This study aimed to develop a K-12 classroom observation protocol to assess K-12 teachers’ implementation of science, technology, engineering, and mathematics (STEM) integration. The intended purpose of the observation protocol is for researchers to examine how K-12 teachers implement the STEM integrated curriculum. Based on research on STEM integration, the protocol covers eight categories: Define, Learn, Plan, Try, Test, Decide, Improve, and Instructional Strategies. The protocol was reviewed by a panel of integrated STEM education experts, engineering design experts, and educational assessment experts (n = 15) and revised accordingly. To test the reliability of using the protocol, two raters applied the protocol to videos of 12 lower-level elementary teachers located at seven schools who implemented an integrated STEM curriculum over a total of 196 class periods. The inter-rater reliability Gwet AC1 coefficient was 0.73. The inter-rater reliability of each category’s coding was also calculated and reported. We discuss the implications and limitations of the protocol.

University students dataset related to achievement, classroom practices, perceptions and attitudes of multimedia-based learning quantum physics

This dataset presents data collected to assess teaching and learning of quantum physics at the University of Rwanda - College of Education (UR-CE), Rwanda. Data were collected between August and November 2019 as the baseline, and between January and April 2022 under a quasi-experimental design. Three sets of data were collected. The first set was about students' performance and conceptual understanding collected before and after teaching intervention (lecture method or multimedia-aided approach) using mainly Quantum Physics Conceptual Survey (QPCS). The second set documented classroom practices during teaching and learning using the Classroom Observation Protocol for Undergraduate STEM (COPUS). The last set is comprised of the data related to lecturers' and students' perceptions before teaching and learning quantum physics and students' attitudes after learning Quantum physics. The Quantum Physics Attitude Test (QPAT) was mainly used to collect these data. The dataset is important to education stakeholders because university managers can visualize the status of teaching and learning outcomes, lecturers can reflect on the study, and researchers can use the data to analyze various independent variables.

Indicating Classroom Observation Protocols in PBL Studies

This paper investigates whether the classroom observation protocol can be adopted in PBL studies, to explore the appropriate instruments for further observation-based studies in PBL obtain. By systematically searching and concluding previous literature, the observation protocols are given special focus, leading to the main results of this review:1) There is one formal observation tool developed for PBL settings in the area of STEM (science, technology, engineering and mathematics) education; 2) Apart from the only PBL-aimed protocol, there are other studies using tools specially designed to demonstrate teachers’ behavior and students’ engagement in real school settings; 3) Classroom observation could offer practical evidence for further exploration in a study using a mixed methodology, but formal observation tools should be adopted in these studies. In PBL studies, classroom observation could be an innovative method for further research, and more observation protocols are encouraged to be adopted or developed in the PBL obtain.

Uncovering Core Dimensions of K-12 Integrated STEM

Abstract To address the lack of a classroom observation protocol aligned with integrated STEM, the author team developed one to measure the degree of integrated STEM instruction implemented in K-12 science and engineering classrooms. This study demonstrates how our instrument can be used to uncover the dimensions of integrated STEM instruction practiced in K-12 classrooms and to determine which protocol items are associated with each of these dimensions. This article reports on the results of a principal component analysis (PCA) using 2030 K-12 classroom observation videos. PCA revealed two core dimensions of integrated STEM education. Real-world problem-solving includes 21st century skills and STEM practices necessary for developing solutions to real-world problems. Nature of Integrated STEM includes items that promote integration between the real-world context, students’ personal experiences, STEM careers, and STEM content. The authors’ analysis also suggests the possibility of an additional dimension of integrated STEM involving technology practices in STEM.

Observing Instructor Behaviour in an Active Learning Classroom: A Case Study of an Undergraduate Calculus Course

Active learning classrooms (ALCs) are spaces explicitly designed to encourage collaborative learning, often through the use of technology. To learn more about teaching activity in ALCs, a study was designed to observe an engineering calculus course during the winter 2020 term. A large-scale active learning classroom was selected for classroom observation using the extended Teaching Dimensions Observation Tool (TDOP+). The TDOP+ is a descriptive classroom observation protocol based on the Teaching Dimensions Observation Tool, enhanced with elements from the Active Learning Classroom Observation Tool (ALCOT). &#x0D; This case study compares the class orchestration of different instructors teaching two different sections of the same course at a large, public university. Twenty class sessions were coded for this study: 10 for each section (5 for each instructor). The coded instructor behaviour was analyzed using a conceptual framework described by Nocera (i.e., a version of Activity Theory), focusing on mediating artifacts and instructor goals. &#x0D; While we observed differences in the frequency and duration of active learning activities and in the type and number of tools used in each class session, the results from this case study suggest that flexible space design enables instructors with the same lesson plan (and content) to create different technological frames to achieve their varied pedagogical goals, while encouraging increased adoption of new tools.

Evidence-based decision-making in nursing – development and piloting of a train-the-trainer concept for teachers at schools for health care professionals: A pilot study

BackgroundTrain-the-trainer concepts (TTT concepts) to impart evidence-based practice competences aim at promoting the implementation of evidence-based practice. The aim of this study was to develop and pilot-test a TTT training program for teachers at schools for health care professionals. The program is based on the core curriculum “Evidence-based Decision-making” set up by the German Network for Evidence-based Medicine e.V. (DNEbM). MethodThe study was conducted according to the UKMRC framework for complex interventions and consisted of Phase 1 (Development) and Phase 2 (Piloting). In Phase 1, the intervention was modelled on the basis of a systematic literature search. In Phase 2, the TTT concept was piloted with teachers at two schools for health care professionals. For the evaluation, short interviews with the participants were carried out after each module; classroom observations took place during the training and were followed by a focus group interview. In addition, baseline characteristics were collected prior to the training and descriptively evaluated. Critical health competence was surveyed using the Critical Health Competence Test (CHC test) before and after the course. The questionnaires, interviews and classroom observation protocols were processed according to Mayring’s qualitative content analysis. Person parameters ±SD were calculated to determine the level of competence. ResultsPiloting took place in September 2018 and from September to November 2019. Eight teachers from two training institutions participated. The evaluation of the focus group interviews and the classroom observation protocols resulted in a category system comprising 6 main and 17 subcategories. The feasibility of the materials and methods could be demonstrated in both pilot tests. Reactivation and extension of prior knowledge were confirmed by the participants. The small group size was appreciated; the participants took an active part during the lessons and were stimulated to plan their own lessons. The evaluation of the CHC test revealed average person parameters of 432 ± 45 (pretest, n=8) and 512 ± 65 (posttest, n=8). DiscussionUsing mixed methods, the results show the feasibility of the curriculum. ConclusionFurther evaluation in a randomized controlled trial (RCT) is needed to prove the efficacy of the curriculum.

Instructors and Students’ Practices and Behaviours during a Quantum Physics class at the University of Rwanda: Exploring the Usage of Multimedia

This study was aimed at exploring the usage of multimedia during a quantum physics class. Five instructors and 385 undergraduate students at the University of Rwanda, College of Education (UR-CE) were observed and surveyed. Thus, the study employed experimental and survey designs. A standardized classroom observation protocol for undergraduate STEM (COPUS) and a validated online survey were used. Classroom observation data were analysed quantitatively using an M.S. Excel spreadsheet, and interpreted descriptively. Likewise, survey data were analysed qualitatively using a note-pencil, and interpreted narratively. The class in which a multimedia method was used, showed more active learning compared to one in which lecturing was used. The findings indicated that instructors (lecturers) were guiding students, and the students were working in the multimedia class. In the lectured class, instructors were found presenting the content to students and students received information passively. Instructors identified the mathematical background as the trigger to students’ negative attitude towards learning quantum physics when they were encouraged to learn through animations, PhET simulations, and YouTube videos. The study recommends the use of multimedia technologies in teaching quantum physics-related concepts.

Recent Developments in Classroom Observation Protocols for Undergraduate STEM: An Overview and Practical Guide

Recent Developments in Classroom Observation Protocols for Undergraduate STEM: An Overview and Practical Guide

Perceptions and Challenges in the Implementation of the In-In-Out Programme in the Selected Colleges of Education in Ghana

The purpose of the study was to assess the perceptions and challenges facing in the implementation process of the IN-IN-OUT programme in selected colleges of education in the Ashanti Region of Ghana. The descriptive cross-sectional survey was employed as the research design in this study. The population was made up of supervisors, mentors, and mentees from St. Louis and Wesley Colleges of education who have been posted to various communities in the Ashanti Region of Ghana to practice teaching. Purposive and random Sampling techniques were used to select tutors, teacher-trainees, mentors and assemblymen in the practicing communities. The study's main data collection tools were questionnaires and interview and classroom observation protocols. The data collected were scored and analyzed using the Statistical Package for Social Scientists (SPSS) computer software. Frequencies, percentages, and other measures of central tendencies were calculated. The study revealed that teacher-trainees, mentors, and assemblymen in the two (2) communities saw the IN-IN-OUT programme as useful to a large extent, even though they had varied opinions. While some say the program was merely useful, some see it as very useful and others saw the program as extremely useful. The study also revealed that the challenges faced by the various teacher trainees, mentors, and supervisors include social problems, financial difficulties, and time constraints. It is recommended that teacher trainees are posted to the rural and peri-urban communities, one can be sure that after their certification they will be fully prepared to face the challenges ahead of them. It is also recommended that the government and stakeholders should as a matter of urgency commit more money to this programme.

Predicting implementation of active learning by tenure-track teaching faculty using robust cluster analysis

BackgroundThe University of California system has a novel tenure-track education-focused faculty position called Lecturer with Security of Employment (working titles: Teaching Professor or Professor of Teaching). We focus on the potential difference in implementation of active-learning strategies by faculty type, including tenure-track education-focused faculty, tenure-track research-focused faculty, and non-tenure-track lecturers. In addition, we consider other instructor characteristics (faculty rank, years of teaching, and gender) and classroom characteristics (campus, discipline, and class size). We use a robust clustering algorithm to determine the number of clusters, identify instructors using active learning, and to understand the instructor and classroom characteristics in relation to the adoption of active-learning strategies.ResultsWe observed 125 science, technology, engineering, and mathematics (STEM) undergraduate courses at three University of California campuses using the Classroom Observation Protocol for Undergraduate STEM to examine active-learning strategies implemented in the classroom. Tenure-track education-focused faculty are more likely to teach with active-learning strategies compared to tenure-track research-focused faculty. Instructor and classroom characteristics that are also related to active learning include campus, discipline, and class size. The campus with initiatives and programs to support undergraduate STEM education is more likely to have instructors who adopt active-learning strategies. There is no difference in instructors in the Biological Sciences, Engineering, or Information and Computer Sciences disciplines who teach actively. However, instructors in the Physical Sciences are less likely to teach actively. Smaller class sizes also tend to have instructors who teach more actively.ConclusionsThe novel tenure-track education-focused faculty position within the University of California system represents a formal structure that results in higher adoption of active-learning strategies in undergraduate STEM education. Campus context and evolving expectations of the position (faculty rank) contribute to the symbols related to learning and teaching that correlate with differential implementation of active learning.