Assessing Reflection and Higher-order Thinking in the Clinical Setting Using Electronic Discussion Threads

Abstract

Background and Purpose. Students learn both theory and practical application in the classroom. However, for many, these aspects of clinical practice are not fully comprehended until students encounter real-life situations that require them to analyze, evaluate, modify, and apply what they know. Reflection can help students connect classroom knowledge to clinical practice and enable students to engage in higher-order thinking, moving beyond knowledge and comprehension to analysis, synthesis, and evaluation critical for practice. Integral to effective clinical teaching is the ability to assess outcomes. While the literature offers strategies to facilitate reflection and higher-order thinking, little has been written about the assessment process. The purpose of this study was to design and test a method of assessing reflection and higher-order thinking evident in electronic discussion board threads of students involved in virtual action learning sets in the clinical setting. Subjects. Seven entry-level physical therapist students on their final full-time clinical internship participated in this study. Participants were engaged in a collaborative model of clinical education on an acute rehabilitation unit. Methods. Participants were assigned to 2 groups based on the timing of their internship. Both groups received a 2-hour in-service on reflective practice, submitted critical incidents to an electronic discussion board, and participated in an electronic discussion for 4 weeks. Following the discussion board activity, discussion threads were downloaded and analyzed for the presence of the elements of reflection and levels of higherorder thinking using coding schemas previously reported in the literature. The submissions were coded independently by 3 raters and interrater reliability was assessed by percent agreements and kappa statistics. Results. Moderate to almost perfect levels of agreement between raters were achieved for most of the coding schemas used, with interrater prevalence-adjusted bias-adjusted kappa coefficients that ranged from 0.38 to 0.92. Discussion and Conclusion. This study provides a reliable method of assessing reflective thinking and higher-order processing in the clinical setting through the use of electronic discussion boards.