Retrospective Confidence Judgments Research Articles

Making Retrospective Confidence Judgments Improves Learners’ Ability to Decide What Not to Study

Predictions about future retrieval success, known as judgments of learning (JOLs), are often viewed as important for effective control over learning. However, much less is known about how retrospective confidence judgments (RCJs), evaluations of past retrieval success, may affect control over learning. We compared participants’ ability to identify items that would benefit from additional study after making either a JOL or an RCJ. Participants completed a cued-recall task in which they made a metacognitive judgment after an initial recall attempt and before making a restudy decision. Participants who made RCJs prior to their restudy decisions were more accurate at identifying items in need of being restudied, relative to participants who made JOLs. The results indicate that having participants assess their confidence in past retrieval success can nudge them toward better utilizing of valid information when deciding which items are in need of further study.

Accuracy of metacognitive judgments as a moderator of learner control effectiveness in problem-solving tasks

A possible explanation for why students do not benefit from learner-controlled instruction is that they are not able to accurately monitor their own performance. The purpose of this study was to investigate whether and how the accuracy of metacognitive judgments made during training moderates the effect of learner control on performance when solving genetics tasks. Eighty-six undergraduate students solved self-selected genetics tasks using either a full learner control or a restricted learner control. Results indicated that learner control effectiveness was moderated by the absolute accuracy (i.e., absolute bias) of metacognitive judgments, and this accuracy was a better predictor of learning performance for full learner control than for restricted learner control. Furthermore, students’ prior knowledge predicted absolute accuracy of both ease-of-learning judgments (EOLs) and retrospective confidence judgments (RCJs) during training, with higher prior knowledge resulting in a better absolute accuracy. Overall, monitoring guided control, that is, EOLs predicted time-on-task and invested mental effort regardless of the degree of learner control, whereas RCJs predicted the total training time, but not the number of tasks selected during training. These results suggest that monitoring accuracy plays an important role in effective regulation of learning from problem-solving tasks, and provide further evidence that metacognitive judgments affect study time allocation in problem solving context.

Rhythmic Influence of Top-Down Perceptual Priors in the Phase of Prestimulus Occipital Alpha Oscillations.

Prior expectations have a powerful influence on perception, biasing both decision and confidence. However, how this occurs at the neural level remains unclear. It has been suggested that spontaneous alpha-band neural oscillations represent rhythms of the perceptual system that periodically modulate perceptual judgments. We hypothesized that these oscillations instantiate the effects of expectations. While collecting scalp EEG, participants performed a detection task that orthogonally manipulated perceptual expectations and attention. Trial-by-trial retrospective confidence judgments were also collected. Results showed that, independent of attention, prestimulus occipital alpha phase predicted the weighting of expectations on yes/no decisions. Moreover, phase predicted the influence of expectations on confidence. Thus, expectations periodically bias objective and subjective perceptual decision-making together before stimulus onset. Our results suggest that alpha-band neural oscillations periodically transmit prior evidence to visual cortex, changing the baseline from which evidence accumulation begins. In turn, our results inform accounts of how expectations shape early visual processing.

Applying Fuzzy Linear Regression to Understand Metacognitive Judgments in a Human-in-the-Loop Simulation Environment

In this paper, a new approach to evaluate metacognitive activities is investigated using fuzzy linear regression analysis. Metacognition shows a broad picture of learning competencies that significantly influences learning processes such as confidence judgment and control of learning. However, it is hard to detect changes in metacognitive judgments because there is no direct way to evaluate metacognition while individuals are learning a new task. We investigated the internal relationship between an individual's metacognitive judgments and task performance. Participants performed a radar monitoring task by playing the role of an antiair warfare coordinator in a human-in-the-loop simulation. In order to measure task performance, participants were given a situation awareness (SA) probe. To measure their metacognitive judgments, we administered a retrospective confidence judgments (RCJ) probe. A fuzzy linear regression model was used to analyze the relationship between RCJ and SA. There were three groups in this experiment. The first group (SA + RCJ feedback) viewed their SA performance with the correct answers to all SA questionnaires and triangular graphs of both SA confidence and SA scores together. The second group (SA feedback) only watched their SA performance with the correct answers to all SA questionnaires. The third group was the control group, and it did not observe any feedback. The results showed that the SA + RCJ feedback screen could significantly influence the participants’ mental state from overconfidence to underconfidence as well as SA accuracy. Using the outcomes of the experiment, we modeled mental state change in metacognitive judgments using fuzzy linear regression.

Differential compromise of prospective and retrospective metamemory monitoring and their dissociable structural brain correlates

Metamemory refers to personal knowledge about one's own memory ability that invokes cognitive processes relevant to monitoring and controlling memory. An impaired monitoring system can potentially result in unawareness of symptoms as can occur in addiction denial. Monitoring processes can be assessed with prospective measures such as Feeling-Of-Knowing (FOK) judgments on prediction of future recognition performance, or retrospective confidence judgments (RCJ) made on previous memory performance. Alcoholic patients with amnesia showed poor FOK but intact RCJ. The neuropsychological continuum from mild to moderate deficits in nonamnesic to amnesic alcoholism raised the possibility that alcoholics uncomplicated by clinically-detectable amnesia may suffer anosognosia for their mild memory deficits. Herein 24 abstinent alcoholics and 26 age-matched controls completed an episodic memory paradigm including prospective FOK and retrospective RCJ monitoring measures and underwent 3T structural magnetic resonance imaging. Alcoholics were less accurate than controls in recognition and in assessing their future recognition performance, which was marked by overestimation, but were as accurate as controls on confidence ratings of actual recognition performance. Examination of brain structure–function relations revealed a double dissociation where FOK accuracy was selectively related to insular volume, and retrospective confidence accuracy was selectively related to frontolimbic structural volumes. Impaired FOK with intact RCJ was consistent with mild anosognosia and suggested evidence for neuropsychological and neural mechanisms of unawareness in addiction.

But I Was So Sure! Metacognitive Judgments Are Less Accurate Given Prospectively than Retrospectively.

Prospective and retrospective metacognitive judgments have been studied extensively in the field of memory; however, their accuracy has not been systematically compared. Such a comparison is important for studying how metacognitive judgments are formed. Here, we present the results of an experiment aiming to investigate the relation between performance in an anagram task and the accuracy of prospective and retrospective confidence judgments. Participants worked on anagrams and were then asked to respond whether a presented word was the solution. They also rated their confidence, either before or after the response and either before or after seeing the suggested solution. The results showed that although response accuracy always correlated with confidence, this relationship was weaker when metacognitive judgements were given before the response. We discuss the theoretical and methodological implications of this finding for studies on metacognition and consciousness.

Building metamemorial knowledge over time: insights from eye tracking about the bases of feeling-of-knowing and confidence judgments.

Metamemory processes depend on different factors across the learning and memory time-scale. In the laboratory, subjects are often asked to make prospective feeling-of-knowing (FOK) judgments about target retrievability, or are asked to make retrospective confidence judgments (RCJs) about the retrieved target. We examined distinct and shared contributors to metamemory judgments, and how they were built over time. Eye movements were monitored during a face-scene associative memory task. At test, participants viewed a studied scene, then rated their FOK that they would remember the associated face. This was followed by a forced choice recognition test and RCJs. FOK judgments were less accurate than RCJ judgments, showing that the addition of mnemonic experience can increase metacognitive accuracy over time. However, there was also evidence that the given FOK rating influenced RCJs. Turning to eye movements, initial analyses showed that higher cue fluency was related to both higher FOKs and higher RCJs. However, further analyses revealed that the effects of the scene cue on RCJs were mediated by FOKs. Turning to the target, increased viewing time and faster viewing of the correct associate related to higher FOKs, consistent with the idea that target accessibility is a basis of FOKs. In contrast, the amount of viewing directed to the chosen face, regardless of whether it was correct, predicted higher RCJs, suggesting that choice experience is a significant contributor RCJs. We also examined covariates of the change in RCJ rating from the FOK rating, and showed that increased and faster viewing of the chosen face predicted raising one’s confidence above one’s FOK. Taken together these results suggest that metamemory judgments should not be thought of only as distinct subjective experiences, but complex processes that interact and evolve as new psychological bases for subjective experience become available.

A-60 * Semantic Memory Confidence: A Correlation with Performance on the Wisconsin Card Sorting Test

Objective: Metamemory, defined as the ability to self-evaluate the capacity, accuracy, and operations of one's memory, is not routinely assessed in the standard neuropsychological examination. Metamemory is distinct from memory proficiency, with a purported neuroanatomical locus in the frontal lobes. The present study evaluated whether overconfidence regarding inaccurate semantic information, which is a type of metamemory impairment, is related to neuropsychological impairment in other domains, such as reasoning, memory, and executive functioning. Method: In the context of a private neuropsychology practice, a consecutive series of examinees with TBI ranging from mild to severe (n = 38) or with brain damage due to other causes (n = 11) were administered the Information subtest of the Wechsler Adult Intelligence Scale-III (WAIS-III) as part of a full neuropsychological assessment. After answering the questions, and without any feedback regarding their response accuracy, examinees were asked to rate their answer confidence in the order of original administration, using a seven-point Likert scale (1 = very unsure to 7 = very sure). Examinees, whose ages ranged from 19 to 71, were excluded if they displayed malingering or had a psychiatric diagnosis. Results: Correct answers were rated with more certainty than incorrect answers (p < .0001), and higher confidence for incorrect answers was significantly correlated with poorer performance on the Wisconsin Card Sorting Test (for categories completed r = −.56, p < .0001; for perseverative errors r = .47, p < .005). Conclusion(s): These findings suggests an association between compromised executive functioning and metamemory abilities in retrospective confidence judgments (RCJ).

Study Time Allocation, Retrospective Confidence, Gender, and Recall Performance: Does MemorySelf-Efficacy Matter?

We examined factors associated with self-regulated learning in a multitrial verbal learning task. Eighty males and 80 females (ages 18–25 years) completed a memory self-efficacy (MSE) questionnaire before being administered a Swahili-English memory task, in which participants received an unlimited amount of time to study each translation and complete cued recall tests across three study-test trials. We hypothesized that MSE, gender, and subsequent study time allocation (STA) based on prior recall performance and retrospective confidence judgments (RCJs) would be related to memory performance. There were gender differences in recall rates on subsequent trials, but no other gender effects were observed. Our results suggested that general beliefs about memory capabilities (MSE), control processes (STA), and monitoring processes (RCJs) are related to recall performance in a multitrial self-regulated learning task, and that the influence of these variables differs by recall trial and whether individuals are in more extreme MSE groups.

Retrospective and prospective metacognitive judgments in rhesus macaques (Macaca mulatta)

A growing body of research suggests that some non-human animals are capable of making accurate metacognitive judgments. In previous studies, non-human animals have made either retrospective or prospective judgments (about how they did on a test or how they will do on a test, respectively). These two types of judgments are dissociable in humans. The current study tested the abilities of two rhesus macaque monkeys to make both retrospective and prospective judgments about their performance on the same memory task. Both monkeys had been trained previously to make retrospective confidence judgments. Both monkeys successfully demonstrated transfer of retrospective metacognitive judgments to the new memory task. Furthermore, both monkeys transferred their retrospective judgments to the prospective task (one, immediately, and one, following the elimination of a response bias). This study is the first to demonstrate both retrospective and prospective monitoring abilities in the same monkeys and on the same task, suggesting a greater level of flexibility in animals' metacognitive monitoring abilities than has been reported previously.

The role of answer fluency and perceptual fluency as metacognitive cues for initiating analytic thinking

Although widely studied in other domains, relatively little is known about the metacognitive processes that monitor and control behaviour during reasoning and decision-making. In this paper, we examined the conditions under which two fluency cues are used to monitor initial reasoning: answer fluency, or the speed with which the initial, intuitive answer is produced (Thompson, Prowse Turner, & Pennycook, 2011), and perceptual fluency, or the ease with which problems can be read (Alter, Oppenheimer, Epley, & Eyre, 2007). The first two experiments demonstrated that answer fluency reliably predicted Feeling of Rightness (FOR) judgments to conditional inferences and base rate problems, which subsequently predicted the amount of deliberate processing as measured by thinking time and answer changes; answer fluency also predicted retrospective confidence judgments (Experiment 3b). Moreover, the effect of answer fluency on reasoning was independent from the effect of perceptual fluency, establishing that these are empirically independent constructs. In five experiments with a variety of reasoning problems similar to those of Alter et al. (2007), we found no effect of perceptual fluency on FOR, retrospective confidence or accuracy; however, we did observe that participants spent more time thinking about hard to read stimuli, although this additional time did not result in answer changes. In our final two experiments, we found that perceptual disfluency increased accuracy on the CRT (Frederick, 2005), but only amongst participants of high cognitive ability. As Alter et al.’s samples were gathered from prestigious universities, collectively, the data to this point suggest that perceptual fluency prompts additional processing in general, but this processing may results in higher accuracy only for the most cognitively able.

The effect of multimedia discrepancies on metacognitive judgments

This study investigated undergraduate students’ metacognitive judgments while learning about complex science topics using multimedia material (text and graph). A within-subjects design was used to examine the effect of discrepancies on study-time allocation, metacognitive judgments and inference generation. There were three types of discrepancies: none, text (between two ideas in the text) and text and graph (between the text and graph). Forty (N=40) participants completed 12 trials where they were asked to provide six judgments: Ease of Learning judgments (EOLs), immediate and delayed Judgments of Learning (JOLs) for both text and graph and Retrospective Confidence Judgments (RCJs). Overall, participants provided significantly lower JOLs for content that contained discrepancies but RCJs remained high across conditions. Discrepancies did not influence study-time allocation, but did significantly influence inference scores. Overall, results suggest that participants’ judgments are different for different aspects of multimedia content and may be based on the text.

Benefits of Order: The Influence of Item Sequencing on Metacognition in Moderate and Severe Traumatic Brain Injury

The ability to appraise one's own ability has been found to have an important role in the recovery and quality of life of clinical populations. Examinee and task variables have been found to influence metacognition in healthy students; however the effect of these variables on the metacognitive accuracy of adults with neurological insult, such as traumatic brain injury (TBI), remains unknown. Twenty-two adults with moderate and severe TBI and a matched sample of healthy adults participated in this study examining the influence of item sequencing on metacognitive functioning. Retrospective confidence judgments were collected while participants completed a modified version of the Matrix Reasoning subtest. Significant influence of item sequence order was found, revealing better metacognitive abilities and performance when participants completed tasks where item difficulty progressed in order from easy to difficult. We interpret these findings to suggest that the sequencing of item difficulty offers "anchors" for gauging and adjusting to task demands.

Confidence judgment in depression and dysphoria: The depressive realism vs. negativity hypotheses

Background and objectivesAccording to the negativity hypothesis, depressed individuals are over-pessimistic due to negative self-concepts. In contrast, depressive realism suggests that depressed persons are realistic compared to their nondepressed controls. However, evidence supporting depressive realism predominantly comes from judgment comparisons between controls and nonclinical dysphoric samples when the controls showed overconfident bias. This study aimed to test the validity of the two accounts in clinical depression and dysphoria. MethodsSixty-eight participants, including healthy controls (n = 32), patients with DSM-IV major depression (n = 20), and dysphoric participants with CDC-defined chronic fatigue syndrome (n = 16) performed an adjective recognition task and reported their item-by-item confidence judgments and post-test performance estimate (PTPE). ResultsCompared to realistic PTPE made by the controls, patients with major depression showed significant underconfidence. The PTPE of the dysphoric participants was relatively accurate. Both the depressed and dysphoric participants displayed less item-by-item overconfidence as opposed to significant item-by-item overconfidence shown by the controls. LimitationsThe judgment-accuracy patterns of the three groups need to be replicated with larger samples using non-memory task domains. ConclusionThe present study confirms depressive realism in dysphoric individuals. However, toward a more severe depressive emotional state, the findings did not support depressive realism but are in line with the prediction of the negativity hypothesis. It is not possible to determine the validity of the two hypotheses when the controls are overconfident. Dissociation between item-by-item and retrospective confidence judgments is discussed.

Accuracy of confidence judgments: Stability and generality in the learning process and predictive validity for learning outcome

One aspect of metacognition is the monitoring of memory or comprehension measured with retrospective confidence judgments after test taking. The research questions of the present study were whether different measures for the accuracy of such confidence judgments are stable over learning time, whether they generalize over two different tests, and whether they predict the learning outcome. In order to answer these questions, a study was conducted in which university students (N = 113) learned about the basic concepts of operant conditioning for 30 min. Knowledge tests with confidence judgments presented after each item were obtained before learning, after 10 min, and at the end of the learning session. Bias and absolute bias were calculated as absolute measures of accuracy, and gamma, Pearson's r, and d(a) were calculated as relative measures of accuracy. The results showed that the absolute measures were stable, but that the relative measures were not. Furthermore, absolute bias, gamma, and Pearson's r obtained 10 min after the beginning of the learning process predicted the learning outcome. The results are discussed with regard to research on different measures of accuracy for confidence judgments.

Solve the problem first: Constructive solution strategies can influence the accuracy of retrospective confidence judgments.

Two experiments tested whether differences in problem-solving strategies influence the ability of people to monitor their problem-solving effectiveness as measured by confidence judgments. On multiple choice problems, people tend to use either a constructive matching strategy, whereby they attempt to solve a problem before looking at the response options, or a response elimination strategy, whereby they work backward from response options trying to find one that fits as a solution. Constructive matching gives rise to different cues that may enhance confidence monitoring. Experiment 1 showed that spontaneous constructive matching in nonverbal spatial reasoning problems was associated with better confidence calibration and resolution than response elimination. We manipulated strategy in Experiment 2 by requiring constructive matching and found improved monitoring. Implications for research on monitoring, overconfidence, and the association between skill and monitoring are discussed.

Neural Correlates of Metamemory: A Comparison of Feeling-of-Knowing and Retrospective Confidence Judgments

Metamemory refers to knowledge and monitoring of one's own memory. Metamemory monitoring can be done prospectively with respect to subsequent memory retrieval or retrospectively with respect to previous memory retrieval. In this study, we used fMRI to compare neural activity during prospective feeling-of-knowing and retrospective confidence tasks in order to examine common and distinct mechanisms supporting multiple forms of metamemory monitoring. Both metamemory tasks, compared to non-metamemory tasks, were associated with greater activity in medial prefrontal, medial parietal, and lateral parietal regions, which have previously been implicated in internally directed cognition. Furthermore, compared to non-metamemory tasks, metamemory tasks were associated with less activity in occipital regions, and in lateral inferior frontal and dorsal medial prefrontal regions, which have previously shown involvement in visual processing and stimulus-oriented attention, respectively. Thus, neural activity related to metamemory is characterized by both a shift toward internally directed cognition and away from externally directed cognition. Several regions demonstrated differences in neural activity between feeling-of-knowing and confidence tasks, including fusiform, medial temporal lobe, and medial parietal regions; furthermore, these regions also showed interaction effects between task and the subjective metamemory rating, suggesting that they are sensitive to the information monitored in each particular task. These findings demonstrate both common and distinct neural mechanisms supporting metamemory processes and also serve to elucidate the functional roles of previously characterized brain networks.

Transfer of Metacognitive Skills and Hint Seeking in Monkeys

Metacognition is knowledge that can be expressed as confidence judgments about what one knows (monitoring) and by strategies for learning what one does not know (control). Although there is a substantial literature on cognitive processes in animals, little is known about their metacognitive abilities. Here we show that rhesus macaques, trained previously to make retrospective confidence judgments about their performance on perceptual tasks, transferred that ability immediately to a new perceptual task and to a working memory task. We also show that monkeys can learn to request "hints" when they are given problems that they would otherwise have to solve by trial and error. This study demonstrates, for the first time, that nonhuman primates share with humans the ability to monitor and transfer their metacognitive ability both within and between different cognitive tasks, and to seek new knowledge on a need-to-know basis.

Does ADHD in Adults Affect the Relative Accuracy of Metamemory Judgments?

Objective: Prior research suggests that individuals with ADHD overestimate their performance across domains despite performing more poorly in these domains. The authors introduce measures of accuracy from the larger realm of judgment and decision making—namely, relative accuracy and calibration—to the study of self-evaluative judgment accuracy in adults with ADHD. Method: Twenty-eight adults with ADHD and 28 matched controls participate in a computer-administered paired-associate learning task and predict their future recall using immediate and delayed judgments of learning (JOLs). Retrospective confidence judgments are also collected. Results: Groups perform equally in terms of judgment magnitude and absolute judgment accuracy as measured by discrepancy scores and calibration curves. Both groups benefit equally from making their JOL at a delay, and the group with ADHD show higher relative accuracy for delayed judgments. Conclusion: Results suggest that under certain circumstances, adults with ADHD can make accurate judgments about their future memory.

Metamemory for faces following frontal lobe damage

Previous research has provided evidence of metamemory impairments in patients with frontal lobe damage on verbal episodic memory tasks. In the present study, we employed metamemory paradigms to investigate whether patients with frontal lesions show monitoring deficits on semantic memory tasks involving facial stimuli. Patients with frontal lobe damage and healthy control subjects made memory decisions to famous faces in a retrospective confidence judgment task and in a prospective feeling-of-knowing (FOK) task. Results indicated that frontal patients performed worse than controls on the retrospective confidence task, but there were no differences between the groups on the FOK task. These findings suggest that metamemory deficits in frontal patients are not confined to specific stimulus domains (words vs. faces) or memory systems (episodic vs. semantic). In addition, the dissociation between retrospective confidence judgments and FOK accuracy documented in this study and also in a recent report by Schnyer et al. suggesting that metamemory should not be considered a unitary function with a single neuroanatomic substrate.