Metacognition in mathematics: do different metacognitive monitoring measures make a difference?

Abstract

Metacognitive monitoring in educational contexts is typically measured by calibration indicators, which are based on the correspondence between cognitive performance and metacognitive confidence judgment. Despite this common rationale, a variety of alternative methods are used in the field of monitoring research to assess performance and judgment data and to calculate calibration indicators from them. However, the impact of these methodological differences on the partly incongruent picture of monitoring research has hardly been considered. Thus, the goal of the present study is to examine the effects of methodological choices in the context of mathematics education. To do so, the study compares the effects of two judgment scales (Likert scale vs. visual analogue scale), two response formats (open-ended response vs. closed response format), the information base of judgment (prospective vs. retrospective), and students’ achievement level on confidence judgments. Secondly, the study contrasts measures of three calibration constructs, namely absolute accuracy (Absolute Accuracy Index, Hamann Coefficient), relative accuracy (Gamma, d’), and diagnostic accuracy (sensitivity and specificity). One hundred and nine seventh-grade students completed a set of 20 mathematical problems and rated their confidence in a correct solution for each problem prospectively and retrospectively. Our results show a pervasive overconfidence of students across achievement levels. Monitoring was more precise for retrospective judgments and the visual analogue scale format. Gamma, sensitivity, and specificity proved to be susceptible for boundary values, caused by the general overconfidence in the sample. Measures of absolute accuracy were affected by response format of the task and judgment scale, with higher accuracy found for closed response format and visual analogue scale. We observed substantial correlations within the three calibration constructs and comparably low correlations between indicators of different constructs, confirming three interrelated aspects of monitoring accuracy. The low correlations between corresponding prospective and retrospective calibration indicators suggest different calibration processes. Implications for studies on calibration and mathematics education are discussed.