Abstract
In recent decades, the assessment of instructional quality has grown into a popular and well-funded arm of educational research. The present study contributes to this field by exploring first impressions of untrained raters as an innovative approach of assessment. We apply the thin slice procedure to obtain ratings of instructional quality along the dimensions of cognitive activation, classroom management, and constructive support based on only 30 s of classroom observations. Ratings were compared to the longitudinal data of students taught in the videos to investigate the connections between the brief glimpses into instructional quality and student learning. In addition, we included samples of raters with different backgrounds (university students, middle school students and educational research experts) to understand the differences in thin slice ratings with respect to their predictive power regarding student learning. Results suggest that each group provides reliable ratings, as measured by a high degree of agreement between raters, as well predictive ratings with respect to students’ learning. Furthermore, we find experts’ and middle school students’ ratings of classroom management and constructive support, respectively, explain unique components of variance in student test scores. This incremental validity can be explained with the amount of implicit knowledge (experts) and an attunement to assess specific cues that is attributable to an emotional involvement (students).
Highlights
The present study explores the potential of first impressions for assessing instructional quality
We investigated the following research questions: 1. Are the thin slices ratings of instructional quality given by university students, middle school students, and experts reliable? Because of the very strong agreement between adult raters found in previous studies, we expect reliability of thin slices ratings to be comparably high in all samples with Intraclass-Correlation Coefficient (ICC) scores higher than
Are the thin slices ratings of instructional quality given by university students, middle school students, and experts valid for predicting students’ learning? As our previous findings indicate the predictive validity of thin slices ratings of cognitive activation and classroom management, we expect the thin slices ratings of these dimensions to be predictive regarding student learning for all rater samples
Summary
The present study explores the potential of first impressions for assessing instructional quality. A research paradigm used in personality research, the thin slice procedure taps into first impressions by having people rate very short samples of the behavior of target persons (Ambady and Rosenthal 1992; Ambady et al 2000) Applying this technique to classroom observations, prior studies have found evidence for high reliability as well as different indications of validity for thin slices ratings by untrained raters based on only a few seconds of observation (Ambady and Rosenthal 1993; Babad 2005; Begrich et al 2017, 2019). If thin slices ratings reflect an assessment of information specific to and diagnostic for instructional quality, we expect ratings of middle school students as well as educational research experts to have incremental validity in terms of predicting students’ learning
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
