Student, instructor, and observer agreement regarding frequencies of scientific teaching practices using the Measurement Instrument for Scientific Teaching-Observable (MISTO)

Mary F Durham,Brian A Couch,Emily K Bremers,Alex R Paine,Jameson D Defreece,Jennifer K Knight

doi:10.1186/s40594-018-0128-1

Abstract

BackgroundThe Scientific Teaching (ST) pedagogical framework encompasses many of the best practices recommended in the literature and highlighted in national reports. Understanding the growth and impact of ST requires instruments to accurately measure the extent to which practitioners implement ST in their courses. Researchers have typically relied on students, instructors, or observers to document course teaching practices, but it remains unclear whether and how these perspectives differ from each other. To address this issue, we modified our previously published instrument to generate the Measurement Instrument for Scientific Teaching-Observable (MISTO), which can be completed by students, instructors, and observers, and we investigated the degree of similarity between these three perspectives across 70 undergraduate science courses at seven different institutions in the USA.ResultsWe found that the full MISTO and Active Learning subcategory scores showed the highest correlations among the three perspectives, but the degree of correlation between perspectives varied for the other subcategories. Match scores between students and instructors were significantly higher than observer matches for the full MISTO and for the Active Learning, Inclusivity, and Responsiveness subcategories.ConclusionsWe find that the level and type of agreement between perspectives varies across MISTO subcategories and that this variation likely stems from intrinsic differences in the course access and scoring decisions of the three perspectives. Building on this data, we recommend MISTO users consider their research goals, available resources, and potential artifacts that may arise when deciding which perspective best fits their needs in measuring classroom teaching practices.

Highlights

The Scientific Teaching (ST) pedagogical framework encompasses many of the best practices recommended in the literature and highlighted in national reports
Based on established guidelines for defining correlation levels (Jackson 2013), full Measurement Instrument for Scientific Teaching-Observable (MISTO) scores showed moderate to strong correlations (r = 0.59–0.74, p < 0.001; Fig. 2)
As transformation efforts in undergraduate science education continue, measurements of teaching practices are needed to gauge the status of the field, track how teaching changes over time, and determine the impact of specific strategies (Freeman et al 2014; Gess-Newsome et al 2003; Smith et al 2013; Wieman and Gilbert 2014)

Summary

Introduction

The Scientific Teaching (ST) pedagogical framework encompasses many of the best practices recommended in the literature and highlighted in national reports. Researchers have typically relied on students, instructors, or observers to document course teaching practices, but it remains unclear whether and how these perspectives differ from each other. To address this issue, we modified our previously published instrument to generate the Measurement Instrument for Scientific Teaching-Observable (MISTO), which can be completed by students, instructors, and observers, and we investigated the degree of similarity between these three perspectives across 70 undergraduate science courses at seven different institutions in the USA. ST includes a wide range of research-based instructional strategies organized into three main pillars: Active Learning, Assessment, and Inclusivity (Handelsman et al 2007)

Objectives

Methods

Results

Conclusion