Judgments Of Learning Accuracy Research Articles

The Relationship between Dispositional Mindfulness and Relative Accuracy of Judgments of Learning: The Moderating Role of Test Anxiety.

Research has demonstrated that metacognition accuracy is far from perfect. The accuracy of judgments of learning (JOLs) is of critical importance in self-regulated learning. To explore what factors constrain JOL accuracy, the current study focused on mindfulness, which is intimately related to metacognition and anxiety. A total of 203 undergraduates (198 valid samples) were recruited to determine the relationships among five dimensions of dispositional mindfulness, test anxiety, and relative accuracy of JOLs. Results revealed that the interaction term for acting with awareness and test anxiety significantly predicted JOL accuracy. Further analyses indicated that for individuals with high test anxiety, but not for those with low test anxiety, acting with awareness positively predicted JOL accuracy. Considering that dispositional mindfulness is modifiable, these results help to inspire researchers to further explore whether mindfulness training can be used as a remedy to improve JOL accuracy.

Memory and Executive Functions Subserving Judgments of Learning: Cognitive Reorganization After Traumatic Brain Injury

ABSTRACT We investigated relationships between psychometric measures of memory, executive functions (EFs), and task-based measures of Judgments of Learning (JOLs) in individuals with and without Traumatic Brain Injury (TBI), to study functional reorganization after TBI. We hypothesized that, in controls, accuracy of immediate JOLs would be associated with short-term memory and EFs (but not long-term memory, LTM), while accuracy of delayed JOLs would be associated primarily with LTM. We hypothesized that those with good recovery from TBI would demonstrate functional reorganization of the relationships between memory, EFs, and JOLs. Eighteen individuals with TBI and 18 matched controls completed eight neuropsychological tests of memory and EFs. They studied word-pairs, made immediate and delayed JOLs, and took a cued-recall test for the studied word-pairs. Stepwise regression and Lasso analyses generated a predictive model of JOL accuracy for each group. Accuracy of immediate JOLs in controls was predicted by short-term memory, inhibition, switching, and cognitive fluency, while accuracy of delayed JOLs was predicted by verbal recall. In individuals with TBI, inhibition predicted immediate JOL accuracy and switching predicted delayed JOL accuracy. Cognitive reorganization in those with good long-term recovery from TBI may account for the ability to make JOLs with accuracy similar to controls.

The Influence of Prior Knowledge on Memory and Metamemory for Famous Names and Faces

Abstract. Two experiments examined the effect of prior knowledge on memory and metamemory for names and faces using famous 1960s and 2000s actors as the manipulation of prior knowledge. In Experiment 1, 66 participants studied the names of famous actors, half presented with their faces, with instructions to remember only the names. In Experiment 2, 56 participants studied the faces of these actors, half presented with their names, with instructions to remember only the faces. In both experiments, participants made immediate Judgments of Learning (JOLs) for each to-be-remembered stimulus followed by a test of recognition that used a Recollect/Familiar/No-Memory judgment. We found higher JOLs, recognition memory, and JOL accuracy for 2000s actors. Adding a name to a face or a face to a name increased JOLs while paradoxically decreasing memory. Back-sorting and binning analyses converged on the conclusion that immediate JOLs predicted memories accompanied by recollection but not familiarity.

The Relationship between Metacognitive Ability and Metacognitive Accuracy

Judgments of learning (JOLs), as one type of metacognitive judgments, are assessments that people make about how well they have learned material. The effective use of JOLs depends on various factors, including task-specific variables and the learner’s own metacognitive resources. Little has been known about the relationship between JOL accuracy for memory predictions and metacognitive ability, which is an emerging theory-practice gap in the field of metacognition. The present study investigated the relationship between the absolute accuracy of JOLs and the metacognitive awareness inventory (MAI), using concrete and abstract word pairs through three study-test cycles. We found that participants who scored high on the MAI, also produced a high level of absolute accuracy on each study-test cycle. The results from mediation analyses yielded that the impact of the MAI on absolute accuracy on cycle 3 was completely mediated by absolute accuracy on the first two cycles for both concrete and abstract word pairs. In addition, the same pattern of results was obtained even when a subset of the MAI (either knowledge or regulation of cognition) was used. These indicate that trait-based metacognitive abilities can well explain the correspondence between JOLs and recall performance on the first test. However, their impact is reduced after study-test practice, suggesting that experience-based factors become critical to improving metacognitive accuracy.

The deceptive nature of associative word pairs: the effects of associative direction on judgments of learning.

The accuracy of judgments of learning (JOLs) in forecasting later recall of cue-target pairs is sensitive to associative direction. JOLs are generally well calibrated for forward associative pairs (e.g., credit-card), but recall accuracy is often overestimated for backward pairs (e.g., card-credit). The present study further examines the effect of associative direction on JOL accuracy by comparing forward and backward pairs to unrelated pairs and symmetrical associates (e.g., salt-pepper)-a novel comparison. The correspondence between initial JOLs and recall accuracy was examined when study was either self-paced with concurrent JOLs (Experiment 1), when study/JOL duration was equated across pair types (Experiment 2), when JOLs were made immediately following study (Experiment 3), and when JOLs were made after a delay (Experiment 4). Across experiments, JOLs accurately estimated correct recall for forward pairs, but overestimated recall for symmetrical, backward, and unrelated pairs-an overestimation that was particularly robust for backward pairs. Calibration plots depicting JOL ratings against their corresponding recall accuracy indicated overestimations occurred for all pair types, though overestimations only occurred at high JOL ratings for symmetrical and forward pairs, a qualitative difference that was not captured in standard analyses of mean JOL and recall rates.

Transcranial Direct Current Stimulation over the Prefrontal Cortex Alters Encoding and Judgments of Learning Based on Fluency.

Past research has shown that judgments of learning (JOLs), subjective confidence judgments made at study about later memorability, are inferential in nature and based on cues available during encoding. Participants tend to use fluency as a cue and give higher JOLs to more fluently encoded items, despite having better recognition memory for disfluently encoded items, which leads to poor JOL accuracy. Research has implicated the dorsolateral prefrontal cortex (DLPFC) and anterior prefrontal cortex (aPFC) in JOL and encoding processes, but no studies to date have tested how the roles of these regions vary with the information on which JOLs are based. We used high-definition transcranial direct current stimulation to test the causal roles of DLPFC and aPFC in encoding success, JOL ratings, and JOL accuracy. Participants studied and made JOLs about words that varied in fluency (i.e., frequency and orientation). High-definition transcranial direct current stimulation over the DLPFC impaired encoding, as evidenced by an increase in subsequent false alarms. For words that were less fluently encoded, aPFC stimulation improved JOL accuracy, perhaps making participants more aware of encoding failures under conditions of disfluency. Conversely, DLPFC and aPFC stimulation decreased JOL accuracy for high-frequency words, suggesting the roles of these regions in JOLs vary with the cognitive bases of the judgments. These results contribute to our understanding of the causal roles of prefrontal regions in objective and subjective memory processes and how their contributions to metamemory accuracy vary with information on which subjective assessments are based.

Accuracy of primary school children’s immediate and delayed judgments of learning about problem-solving tasks

Accuracy of students’ judgments of learning (JOLs) plays an important role in self-regulated learning. Most studies on JOL accuracy have focused on learning word pairs and text but problems-solving tasks are also very important in education. This study investigated whether children in grade 3 could differentiate in their JOLs between problem-solving tasks that varied in complexity. Participants (N = 76, 8–10 years old) engaged in solving four arithmetic problems, rated mental effort invested in each problem, gave either immediate or delayed JOLs, and completed a test containing isomorphic problems. The negative correlation that was found between invested mental effort and JOLs suggested that children's JOLs are sensitive to differences in complexity of the problem-solving tasks. Results on the relative and absolute accuracy of JOLs showed that immediate JOLs were numerically higher than delayed JOLs, and relative accuracy of immediate JOLs was moderately accurate, whereas delayed JOLs were not.

Why are children overconfident? Developmental differences in the implementation of accessibility cues when judging concept learning

Children are often overconfident when monitoring their learning, which is harmful for effective control and learning. The current study investigated children’s (N=167, age range 7–12years) judgments of learning (JOLs) when studying difficult concepts. The main aims were (a) to investigate how JOL accuracy is affected by accessibility cues and (b) to investigate developmental changes in implementing accessibility cues in JOLs. After studying different concepts, children were asked to generate novel sentences and then to make JOLs, select concepts for restudy, and take a final test. Overconfidence for incorrect and incomplete test responses was reduced for older children in comparison with younger children. For older age groups, generating a sentence led to greater overconfidence compared with not being able to generate a sentence, which indicates that older children relied more on accessibility cues when making JOLs. This pattern differed in the youngest age group; younger children were generally overconfident regardless of whether they had generated sentences or not. Overconfidence was disadvantageous for effective control of learning for all age groups. These findings imply that instructions to encourage children to avoid metacognitive illusions need to be adapted to children’s developmental stage.

Effects of Problem Solving after Worked Example Study on Primary School Children's Monitoring Accuracy

SummaryResearch on expository text has shown that the accuracy of students' judgments of learning (JOLs) can be improved by instructional interventions that allow students to test their knowledge of the text. The present study extends this research, investigating whether allowing students to test the knowledge they acquired from studying a worked example by means of solving an identical problem, either immediately or delayed, would enhance JOL accuracy. Fifth grade children (i) gave an immediate JOL, (ii) a delayed JOL, (iii) solved a problem immediately and then gave a JOL, (iv) solved a problem immediately and gave a delayed JOL, or (v) solved a problem at a delay and then gave a JOL. Results show that problem solving after example study improved children's JOL accuracy (i.e., overestimation decreased). However, no differences in the accuracy of restudy indications were found. Results are discussed in relation to cue utilization when making JOLs. Copyright © 2014 John Wiley & Sons, Ltd.

Completion of partially worked-out examples as a generation strategy for improving monitoring accuracy

Students’ Judgments of Learning (JOLs) are often inaccurate: students often overestimate their future test performance. Because of the consequences that JOL inaccuracy can have for regulating study activities, an important question is how JOL accuracy can be improved. When learning texts, JOL accuracy has been shown to improve through ‘generation strategies’, such as generating keywords, summaries, or concept maps. This study investigated whether JOL accuracy can also be improved by means of a generation strategy (i.e., completing blank steps in the examples) when learning to solve problems through worked example study. Secondary education students of 14–15years old (cf. USA 9th grade) either studied worked examples or completed partially worked examples and gave JOLs. It was found that completion of worked examples resulted in underestimation of future test performance. It seems that completing partially worked-out examples made students less confident about future performance than studying fully worked examples. However, this did not lead to better regulation of study.

Are feeling-of-knowing and judgment-of-learning different? Evidence from older adults

This study aims to assess age differences between Judgments-of-learning (JOLs) and Feeling-of-knowing (FOKs) as they are typically studied. The novel contribution of the present study is a comparison between these two metacognitive judgments in a within subject design. Young and older adults were tested on their JOL accuracy and were asked to predict future recall during learning. All participants were also asked to predict future recognition of unrecalled items (FOK judgments). Results showed that although older adults had similar low levels of memory performance in the JOL task and in the FOK task, metacognitive impairments were only found on the resolution of FOKs. Furthermore, an analysis of covariance showed that age differences on memory performance explained the age effect observed on the FOK, thus supporting the memory constraint hypothesis (Hertzog et al., 2010). Results are discussed in relation to contemporary models of memory.

What you know can hurt you: Effects of age and prior knowledge on the accuracy of judgments of learning.

How do aging and prior knowledge affect memory and metamemory? We explored this question in the context of a dual-process approach to Judgments of Learning (JOLs), which require people to predict their ability to remember information at a later time. Young and older adults (n = 36, mean ages = 20.2 & 73.1) studied the names of actors who were famous in the 1950s or 1990s, providing a JOL for each. Recognition memory for studied and unstudied actors was then assessed using a Recollect/Know/No-Memory (R/K/N) judgment task. Results showed that prior knowledge increased recollection in both age groups such that older adults recollected significantly more 1950s actors than younger adults. Also, for both age groups and both decades, actors judged R at test garnered significantly higher JOLs at study than actors judged K or N. However, while the young showed benefits of prior knowledge on relative JOL accuracy, older adults did not, showing lower levels of JOL accuracy for 1950s actors despite having higher recollection for, and knowledge about, those actors. Overall, the data suggest that prior knowledge can be a double-edged sword, increasing the availability of details that can support later recollection, but also increasing nondiagnostic feelings of familiarity that can reduce the accuracy of memory predictions.

The influence of delaying judgments of learning on metacognitive accuracy: A meta-analytic review.

Many studies have examined the accuracy of predictions of future memory performance solicited through judgments of learning (JOLs). Among the most robust findings in this literature is that delaying predictions serves to substantially increase the relative accuracy of JOLs compared with soliciting JOLs immediately after study, a finding termed the delayed JOL effect. The meta-analyses reported in the current study examined the predominant theoretical accounts as well as potential moderators of the delayed JOL effect. The first meta-analysis examined the relative accuracy of delayed compared with immediate JOLs across 4,554 participants (112 effect sizes) through gamma correlations between JOLs and memory accuracy. Those data showed that delaying JOLs leads to robust benefits to relative accuracy (g = 0.93). The second meta-analysis examined memory performance for delayed compared with immediate JOLs across 3,807 participants (98 effect sizes). Those data showed that delayed JOLs result in a modest but reliable benefit for memory performance relative to immediate JOLs (g = 0.08). Findings from these meta-analyses are well accommodated by theories suggesting that delayed JOL accuracy reflects access to more diagnostic information from long-term memory rather than being a by-product of a retrieval opportunity. However, these data also suggest that theories proposing that the delayed JOL effect results from a memorial benefit or the match between the cues available for JOLs and those available at test may also provide viable explanatory mechanisms necessary for a comprehensive account.

The effects of summary production and encoding condition on children’s metacognitive monitoring

Two studies were conducted to investigate whether context variations were suitable to improve metacognitive judgments in children in a complex, everyday memory task. In the first phase of each experiment, participants were shown a short event (video) and gave judgments-of-learning (JOLs), that is, rated their certainty that they would later be able to recall specific details correctly. In the second phase of the experiments, participants took part in a memory interview about the memory event and gave confidence judgments (CJs), that is, rated their certainty that the provided answers to the memory questions were correct. Study 1 specifically investigated the potential positive influence of giving a verbal summary before the JOL-interview on metacognitive monitoring, whereas Study 2 had a closer look on the effect of intentional versus non-intentional encoding on JOL and CJ accuracy. Results revealed no significant influence of giving a summary and hardly any effect of encoding condition on metamemory monitoring although children from age 6 on showed adequate monitoring performance. JOL accuracy appears to be a complex process, which is even more difficult to influence in children than in adults.

A revised methodology for research on metamemory: Pre-judgment Recall and Monitoring (PRAM).

A revised methodology is described for research on metacognitive monitoring, especially judgments of learning (JOLs), to investigate psychological processing that previously has been only hypothetical and unobservable. During data collection a new stage of recall occurs just prior to the JOL, so that during data analysis the items can be partitioned into subcategories to measure the degree of JOL accuracy in ways that are more analytic than was previously possible. A weighted-average combinatorial rule allows the component measures of JOL accuracy to be combined into the usual overall measure of metacognitive accuracy. An example using the revised methodology offers a new explanation for the delayed-JOL effect, in which delayed JOLs are more accurate than immediate JOLs for predicting recall.

Metamemory cues and monitoring accuracy: Judging what you know and what you will know.

Three experiments examined metamemory for categorized lists of items. Judgments of learning (JOLs) were obtained from college students either immediately after study or following a brief (at least 30-s) delay. In contrast to past findings (e.g., T.O. Nelson & J. Dunlosky, 1991), no advantage was found for delayed JOLs in Experiment 1, using a standard, prediction-based metamemory cue. In Experiment 2, knowledge-based judgments were elicited, and delayed JOL accuracy improved significantly. The relative efficacy of 4 different metamemory cues was examined in Experiment 3. An interaction between the timing and phrasing of JOL cues was detected: Delayed JOLs were more accurate than immediate JOLs only when knowledge-based cues were used. These results are interpreted in A. Koriat's (1997) cue-utilization framework for JOL accuracy, and they show that the phrasing of metamemory cues can have a substantial impact on monitoring accuracy.

Enhanced metamemory at delays: why do judgments of learning improve over time?

Judgments of learning (JOLs) made after a 5-min delay are almost perfectly accurate: the "delayed-JOL effect" (T. O. Nelson & J. Dunlosky, 1991). The mechanisms underlying this phenomenon have been the subject of debate. This study examined the effects of delays and short-term memory (STM) distraction on memory and metamemory (JOLs). STM distraction (2.5-30 s) immediately following encoding increased both JOL accuracy and mean cued recall. However, JOLs made after longer delays (4-5 min) were even more accurate. In addition, making a JOL at longer delays improved cued-recall performance. Conditional probabilities of cued recall (given successful initial retrieval) also increased over time and with interference, indicating that delayed JOLs may alter what they assess. Finally, increased confidence was associated with shorter JOL latencies only at delays. The results are consistent with an accessibility view of metamemory (e.g., A. Koriat, 1993).

Similarity between the Cue for Judgments of Learning (JOL) and the Cue for Test Is Not the Primary Determinant of JOL Accuracy

Eventual memory performance is predicted more accurately when a person's judgment of learning (JOL) is delayed until shortly after studying an item than when made immediately after studying the item. According to the transfer-appropriate-monitoring hypothesis, this delayed-JOL effect arises because of the contextual similarity between the cue for the JOL and the cue for the memory test. In three paired-associate learning experiments, delayed JOLs were cued by the stimulus alone or by the stimulus-response pair, and the eventual test was associative recognition of stimulus-response pairs. Recognition of stimulus-response pairs was predicted more accurately when JOLs had been cued by the stimulus alone than when they had been cued by the stimulus-response pair, even though the latter was more similar than the former to the cue for the recognition test. Implications of these results, especially the lack of support for the class of theories emphasizing transfer-appropriate monitoring, are discussed for theories of the accuracy of JOLs.

Delaying students' metacognitive monitoring improves their accuracy in predicting their recognition performance.

The accuracy of students' judgments of learning (JOLs) in predicting recognition versus recall was investigated in 4 experiments. Students studied paired associates and made JOLs, which occurred either immediately after an item had been studied or shortly after an item had been studied. Students then received tests of associative recognition or paired-associate recall. JOL accuracy was greater for delayed JOLs than immediate JOLs, and the accuracy of JOLs was lower in predicting recognition than recall. The latter finding occurred (a) regardless of whether students had anticipated a recall test or a recognition test when making JOLs and (b) regardless of whether JOLs had been cued by only the stimulus of an item or by the entire stimulus-response pair. Correct guessing was shown to contribute to the lower accuracy of students' predictions of recognition.