Development of a self-report instrument for measuring in-class student engagement reveals that pretending to engage is a significant unrecognized problem.

Kathryn A Fuller,Betty Exintaris,Jennifer L Short,Nilushi S Karunaratne,Michael D Wolcott,Paul J White,Scott Singleton,Som Naidu,Emmanuel Manalo

doi:10.1371/journal.pone.0205828

Kathryn A Fuller, Betty Exintaris + Show 7 more

Open Access

https://doi.org/10.1371/journal.pone.0205828

Copy DOI

Abstract

Student engagement during classes includes behavioural, cognitive and emotional components, and is a pre-requisite for successful active learning environments. A novel approach to measuring student engagement was developed, involving triangulation of real-time student-self report, observation by trained observers and heart rate measurement. The self-report instrument was evaluated in four separate cohorts (n = 123) at Monash University and the University of North Carolina. The six item self-report demonstrated good reliability (Cronbach’s alpha values ranged from 0.7–0.81). The self-report showed predictive validity in that small group activities were rated as significantly more engaging than didactic lecturing. Additionally, there was significant inter-instructor variability and within-class variability, indicating good discrimination between classroom activities. This self-report may prove useful to academic teaching staff in evaluating and refining their active learning activities. Independent observation was not found to correlate with student self-report, due in part to students who were pretending to engage being rated as engaged by an observer. Strikingly, students reported that they were pretending to engage for 23% of class time, even for highly regarded instructors. Individual participants were rated as engaged for 42 of the 46 intervals for which they reported that they had “pretended to engage”, indicating that the two observers were unable to detect disengagement during periods in which students pretended to engage. Instructors should be aware that student cues such as eye contact and nodding may indicate pretending to engage. One particular self-report item; “I tried a new approach or way of thinking about the content”, correlated positively with heart rates, and a controlled study reproduced this finding during two activities that required students to try a new approach to understanding a concept. Agreement with this item also correlated with superior performance on two in-class written assessment tasks (n = 101, p<0.01). Further use of this tool and related educational research may be useful to identify in-class activities that are engaging and likely to lead to improved student attainment of learning outcomes.

Highlights

Many institutions, including the two pharmacy and pharmaceutical sciences education institutions at which this study was conducted, have transformed their curricula to focus heavily on active learning [1, 2]
This study characterized a method for assessing student engagement during classes
Independent observation was not found to correlate with student self-report, due in part to student’s pretending to engage who were rated as engaged by an observer

Summary

Introduction

Many institutions, including the two pharmacy and pharmaceutical sciences education institutions at which this study was conducted, have transformed their curricula to focus heavily on active learning [1, 2]. Active learning is an instructional approach that involves students as active participants in classes, tackling activities, often in small groups, to gain mastery of learning outcomes by doing rather than by listening. This study focuses on measuring student engagement in the context of active learning; i.e. engagement with an active learning task. Student engagement assumes even greater importance in this context than with traditional lecturing, since engagement with tasks is required for active learning to work. If students engage with a task, they may (or may not) gain desired knowledge or skills. Students may argue that they have the potential to gain the same knowledge or skills through independent study rather than attendance at classes or participation in active learning tasks, . In the current environment of significant change in how we teach, validated methods are required to provide real-time feedback to instructors that will inform a cycle of quality improvement

Objectives

Discussion

Conclusion