The Confidence-Accuracy Relationship in Diagnostic Assessment: The Case of the Potential Difference in Parallel Electric Circuits

Abstract

(ProQuest: ... denotes formulae omitted)Research in science education has revealed that students have conceptions that differ from accepted scientific ones in many science subjects (Driver, Guesne, & Tiberghien, 1985; Driver, Squires, Rushworth, & Wood-Robinson, 1994; Duit, 2009). Although several research methods have been employed elicit these alternative ways of reasoning (White & Gunstone, 1992), written diagnostic questions have been primary method for data collection from a large sample in a relatively short time. A diagnostic question analyzes an individuals performance uncover his or her strengths or weaknesses in subject tested. In addition information provided by students' answers questions, knowledge of their confidence in their answers may prove be useful. Confidence judgements provide insights into how well a person evaluates and monitors his or her performance'' (Stankov & Crawford, 1996, p. 971). Students with unwarranted high confidence in their knowledge in a given domain are considered overconfident,'' whereas those with unwarranted low confidence are under-confident.'' Overconfidence in a given area of knowledge indicates that (a) such knowledge is frequently invoked, (b) such knowledge is rarely questioned or checked against other criteria, so that wrong are detected, and (c) such knowledge may be maintained, even in face of explicit counterevidence, so that wrong persist (Reif & Allen, 1992, p. 28). Conversely, under-confidence in a given area indicates that such knowledge may be believed, even if correct, and may be rejected in favor of incorrect conclusions (Reif & Allen, 1992). Research on over/under-confidence has revealed that many people are overconfident in their knowledge (Lichtenstein, Fischhoff, & Phillips, 1982).In research literature, overconfidence and under-confidence are related concept of calibration investigating relationship between confidence in and accuracy of performance. Alexander (2013) defined calibration as the distance between perceived and demonstrated levels of understanding, capability, competence, or preparedness (p. 1). She indicated that there is an increasing interest in topic of calibration among educational researchers and argued that one reason for this is that constructs and processes, as conceptual change, metacognition, and self-regulation, are closely linked learners' ability accurately assess their own capacities. Furthermore, Alexander argued that learners' subsequent efforts and strategic behaviors will be affected by their calibration.Another important concept in research literature on confidence-accuracy relationship is resolution (Mengelkamp & Bannert, 2012). Sharp, Cutler, and Penrod (1988) defined resolution as students' ability to discriminate correct from incorrect by differentially assigning confidence accurate and inaccurate judgments (p. 272). Students who can appropriately discriminate between what they know and what they do (i.e., students with good resolution) may use their study time more effectively than students with poor resolution of confidence.Data on calibration and resolution are considered valuable because findings from research on confidence-accuracy relationship imply that students' confidence in their science knowledge is a part of their understanding of natural phenomena.Confidence Judgements in Science Education ResearchSeveral studies in science education research have employed Certainty of Responses Index (CRI) proposed by Hasan, Bagayoko, and Kelley (1999) distinguish a lack of knowledge from alternative conceptions. In their study, confidence levels of CRI were totally guessed answer, almost a guess, not sure, almost certain, and certain. According Hasan et al. …