What Do We Reward in Reflection? Assessing Reflective Writing with the Index for Metacognitive Knowledge

Abstract

Reflection is a staple of contemporary writing pedagogy and writing assessment. Although the power of reflective writing has long been understood in writing studies, the field has not made progress on articulating how to assess the reflective work. Developed at the crossroads of research in reflection and metacognition, the Index for Metacognitive Knowledge (IMK) is designed to help writing researchers, teachers, and students articulate what is being rewarded in the assessment of reflection and to articulate the role of metacognitive knowledge in critical reflective writing. The IMK was used to code final portfolio introductions from first-year writing courses in order to analyze the distribution of the three kinds of metacognitive knowledge (declarative, procedural, and conditional) and to explore the quality and complexity of students’ metacognitive knowledge. Inter-rater reliability testing on the IMK showed that it is highly reliable; the Fleiss’ kappa was 83% (K=.834). The IMK offers researchers, teachers, and students language with which to explore the unique work of reflective writing in order to develop more metacognitively rich observations. It provides a framework to explain the evolving complexity of students’ reflective writing and to assess and describe the impacts of other pedagogical interventions.