Metacognitive Evaluations Research Articles

Naloxone Coprescribing and the Prevention of Opioid Overdoses: Quasi-Experimental Metacognitive Assessment of a Novel Education Initiative.

Critical evaluation of naloxone coprescription academic detailing programs has been positive, but little research has focused on how participant thinking changes during academic detailing. The dual purposes of this study were to (1) present a metacognitive evaluation of a naloxone coprescription academic detailing intervention and (2) describe the application of a metacognitive evaluation for future medical education interventions. Data were obtained from a pre-post knowledge assessment of a web-based, self-paced intervention designed to increase knowledge of clinical and organizational best practices for the coprescription of naloxone. To assess metacognition, items were designed with confidence-weighted true-false scoring. Multiple metacognitive scores were calculated: 3 content knowledge scores and 5 confidence-weighted true-false scores. Statistical analysis examined whether there were significant differences in scores before and after intervention. Analysis of overall content knowledge showed significant improvement at posttest. There was a significant positive increase in absolute accuracy of participant confidence judgments, confidence in correct probability, and confidence in incorrect probability (all P values were <.05). Overall, results suggest an improvement in content knowledge scores after intervention and, metacognitively, suggest that individuals were more confident in their answer choices, regardless of correctness. Implications include the potential application of metacognitive evaluations to assess nuances in learner performance during academic detailing interventions and as a feedback mechanism to reinforce learning and guide curricular design.

Pharmacological Elevation of Catecholamine Levels Improves Perceptual Decisions, But Not Metacognitive Insight.

Perceptual decisions are often accompanied by a feeling of decision confidence. Where the parietal cortex is known for its crucial role in shaping such perceptual decisions, metacognitive evaluations are thought to additionally rely on the (pre)frontal cortex. Because of this supposed neural differentiation between these processes, perceptual and metacognitive decisions may be divergently affected by changes in internal (e.g., attention, arousal) and external (e.g., task and environmental demands) factors. Although intriguing, causal evidence for this hypothesis remains scarce. Here, we investigated the causal effect of two neuromodulatory systems on behavioral and neural measures of perceptual and metacognitive decision-making. Specifically, we pharmacologically elevated levels of catecholamines (with atomoxetine) and acetylcholine (with donepezil) in healthy adult human participants performing a visual discrimination task in which we gauged decision confidence, while electroencephalography was measured. Where cholinergic effects were not robust, catecholaminergic enhancement improved perceptual sensitivity, while at the same time leaving metacognitive sensitivity unaffected. Neurally, catecholaminergic elevation did not affect sensory representations of task-relevant visual stimuli but instead enhanced well-known decision signals measured over the centroparietal cortex, reflecting the accumulation of sensory evidence over time. Crucially, catecholaminergic enhancement concurrently impoverished neural markers measured over the frontal cortex linked to the formation of metacognitive evaluations. Enhanced catecholaminergic neuromodulation thus improves perceptual but not metacognitive decision-making.

Distinct basal ganglia contributions to learning from implicit and explicit value signals in perceptual decision-making

Metacognitive evaluations of confidence provide an estimate of decision accuracy that could guide learning in the absence of explicit feedback. We examine how humans might learn from this implicit feedback in direct comparison with that of explicit feedback, using simultaneous EEG-fMRI. Participants performed a motion direction discrimination task where stimulus difficulty was increased to maintain performance, with intermixed explicit- and no-feedback trials. We isolate single-trial estimates of post-decision confidence using EEG decoding, and find these neural signatures re-emerge at the time of feedback together with separable signatures of explicit feedback. We identified these signatures of implicit versus explicit feedback along a dorsal-ventral gradient in the striatum, a finding uniquely enabled by an EEG-fMRI fusion. These two signals appear to integrate into an aggregate representation in the external globus pallidus, which could broadcast updates to improve cortical decision processing via the thalamus and insular cortex, irrespective of the source of feedback.

Hot metacognition: poorer metacognitive efficiency following acute but not traumatic stress

Aberrations to metacognition—the ability to reflect on and evaluate self-performance—are a feature of poor mental health. Theoretical models of post-traumatic stress disorder propose that following severe stress or trauma, maladaptive metacognitive evaluations and appraisals of the event drive the development of symptoms. Empirical research is required in order to reveal whether disruptions to metacognition cause or contribute to symptom development in line with theoretical accounts, or are simply a consequence of ongoing psychopathology. In two experiments, using hierarchical Bayesian modelling of metacognition measured in a memory recognition task, we assessed whether distortions to metacognition occur at a state-level after an acute stress induction, and/or at a trait-level in a sample of individuals experiencing intrusive memories following traumatic stress. Results from experiment 1, an in-person laboratory-based experiment, demonstrated that heightened psychological responses to the stress induction were associated with poorer metacognitive efficiency, despite there being no overall change in metacognitive efficiency from pre- to post-stress (N = 27). Conversely, in experiment 2, an online experiment using the same metamemory task, we did not find evidence of metacognitive alterations in a transdiagnostic sample of patients with intrusive memory symptomatology following traumatic stress (N = 36, compared to 44 matched controls). Our results indicate a relationship between state-level psychological responses to stress and metacognitive alterations. The lack of evidence for pre- to post-stress differences in metamemory illustrates the importance for future studies to reveal the direction of this relationship, and consequently the duration of stress-associated metacognitive impairments and their impact on mental health.

Metacognitive phenomena during human–Internet interactions

External information systems often serve as an extended cognitive system and are usually conceived as expansions of the capacity of human cognition. However, the boundaries between our own mind and a powerful cloud mind, like the Internet, are increasingly blurry. This paper discusses recent empirical evidence of various metacognitive phenomena taking place while searching for information on the Internet, against the backdrop of the theory of transactive memory systems and the theory of cognitive offloading. The discussion focuses on the ways our cognitive systems maneuver and adapt their responses to the medium of the Internet by examining its effects on the metacognitive evaluations of oneself as a knower, the novel metacognitive experience of the feeling of findability, and how our metamemory judgments are affected. The conclusion proposes directions for future research and a better understanding of our interaction with the Internet.

Prospective and Retrospective Metacognitive Abilities and Their Association with Impaired Self-awareness in Patients with Traumatic Brain Injury.

Metacognitive impairment often occurs in patients with traumatic brain injury (TBI) and is associated with clinical problems. The aim of this study was to clarify the pathology of metacognitive impairment in TBI patients using a behavioral task, clinical assessment of self-awareness, and lesion-symptom mapping. Metacognitive abilities of TBI patients and healthy controls were assessed using a modified perceptual decision-making task. Self-awareness was assessed using the Patient Competency Rating Scale and the Frontal Systems Behavior Scale. The associations between estimated metacognitive abilities, self-awareness, and neuropsychological test results were examined. The correspondence between metacognitive disabilities and brain lesions was explored by ROI-based lesion-symptom mapping using structural magnetic resonance images. Overall, 25 TBI patients and 95 healthy controls were included in the analyses. Compared with that in healthy controls, the prospective metacognitive ability of TBI patients was lower, with metacognitive evaluations revealing a bias toward overestimating their abilities. Retrospective metacognitive ability showed a negative correlation with self-awareness but not with neuropsychological test results. In the lesion-symptom mapping analysis, the left pFC was associated with lower retrospective metacognitive ability. This study contributes to a better understanding of the pathology of metacognitive and self-awareness deficits in TBI patients and may explain the cause of impaired realistic goal setting and adaptive behavior in these patients.

Retrospective judgments of confidence in a complex span task

Although the study of metamemory monitoring originated in predictions for simple span tasks, the study of metacognition for working memory (WM) has been somewhat neglected in comparison with long-term memory. We aimed to fill this gap by exploring the ability to self-assess WM operations. Thirty-four participants performed 16 series of complex span tasks and rated their confidence in a verbal recall paradigm. We manipulated the cognitive load based on the TBRS model in order to analyze the role of attentional resources on both WM and metacognitive evaluations. As expected, we found that recall is affected by cognitive load and we found standard serial position effects. Interestingly, metacognitive evaluations followed the same pattern, and measures of metacognitive sensitivity suggest that participants are able to make item-by-item retrospective judgments reflective of their performance. We discuss how these results contribute to our understanding of metacognitive access to newly-formed WM contents.

Scientific experiments beyond surprise and beauty

Some experimental results in science are productively surprising or beautiful. Such results are disruptive in their epistemic nature: by violating epistemic expectations they mark the phenomenon at hand as worthy of further investigation. Could it be that there are emotions beyond these two which are also useful for the epistemic evaluation of scientific experiments? Here, I conduct a structured sociological survey to explore affective experiences in scientific experimental research. I identify that learning the results of an experiment is the high emotional point in the experimenting process. Thus, experimental results can be challenging, beautiful, or boring, and they can worry, amuse, make one sad, and so on. They can also drive meta-cognitive evaluations as well as motivate specific research-related actions. From this, I advance two claims: that emotions beyond surprise and disruptive beauty are epistemically useful in empirical research, and that emotions help to overcome specific challenges of reasoning about new experimental results.

Value certainty and choice confidence are multidimensional constructs that guide decision-making

The degree of certainty that decision-makers have about their evaluations of available choice alternatives and their confidence about selecting the subjectively best alternative are important factors that affect current and future value-based choices. Assessments of the alternatives in a given choice set are rarely unidimensional; their values are usually derived from a combination of multiple distinct attributes. For example, the taste, texture, quantity, and nutritional content of a snack food may all be considered when determining whether to consume it. We examined how certainty about the levels of individual attributes of an option relates to certainty about the overall value of that option as a whole and/or to confidence in having chosen the subjectively best available option. We found that certainty and confidence are derived from unequally weighted combinations of attribute certainties rather than simple, equal combinations of all sources of uncertainty. Attributes that matter more in determining choice outcomes also are weighted more in metacognitive evaluations of certainty or confidence. Moreover, we found that the process of deciding between two alternatives leads to refinements in both attribute estimations and the degree of certainty in those estimates. Attributes that are more important in determining choice outcomes are refined more during the decision process in terms of both estimates and certainty. Although certainty and confidence are typically treated as unidimensional, our results indicate that they, like value estimates, are subjective, multidimensional constructs.

Knowing more than we know: metacognition, semantic fluency, and originality in younger and older adults

ABSTRACT We examined age-related similarities and differences in people’s metacognitive awareness of retrieval from semantic long-term memory as well as the originality of their responses. Participants completed several semantic fluency tasks, and before recalling items, made metacognitive predictions of their performance. Additionally, after retrieval, participants made metacognitive evaluations of the originality of their responses. Results revealed that both younger (M age = 24.49) and older adults (M age = 68.31) were underconfident in their performance, despite some metacognitive awareness of their ability to retrieve information from semantic memory. Younger and older adults became more metacognitively aware of their abilities with task experience, but there were no significant differences in participants’ metacognitive predictions and postdictions, although older adults believed that they were less original than younger adults. These findings revealed a “skilled and unaware” effect whereby participants were underconfident on the first trial and became less underconfident on later trials. These patterns may fit with a broader literature that has found a lack of adult age differences in metacognition for verbal skills but shows that older adults may believe that their access to original verbal knowledge may decline in older age.

Confidence at the limits of human nested cognition.

Metacognition is the ability to weigh the quality of our own cognition, such as the confidence that our perceptual decisions are correct. Here we ask whether metacognitive performance can itself be evaluated or else metacognition is the ultimate reflective human faculty. Building upon a classic visual perception task, we show that human observers are able to produce nested, above-chance judgements on the quality of their decisions at least up to the fourth order (i.e. meta-meta-meta-cognition). A computational model can account for this nested cognitive ability if evidence has a high-resolution representation, and if there are two kinds of noise, including recursive evidence degradation. The existence of fourth-order sensitivity suggests that the neural mechanisms responsible for second-order metacognition can be flexibly generalized to evaluate any cognitive process, including metacognitive evaluations themselves. We define the theoretical and practical limits of nested cognition and discuss how this approach paves the way for a better understanding of human self-regulation.

Ketamine increases fronto-posterior functional connectivity during meta-perceptual confidence ratings

Recent advances in the neuropsychopharmacology of metacognition indicate a constituent role of glutamate for the integrity of metamnestic processes. However, the extent to which previous results can be generalized across functional domains to characterize the relationship between glutamate and metacognition remains unclear. Here, in a randomized, double-blind, placebo-controlled, preregistered fMRI study, we tested the effects of a psychotomimetic dose (target plasma concentration 100 ng/mL) of the N-methyl-D-aspartate (NMDA) glutamate receptor antagonist ketamine on metacognition in a perceptual decision-making framework. We collected trial-by-trial metacognitive reports as participants performed a two-alternative forced-choice perceptual task during functional magnetic resonance imaging (fMRI). Results indicated ketamine-induced deterioration in metacognitive performance, whereas no significant effects were observed for perceptual performance, response times and – unexpectedly – metacognitive bias. Whilst there were no detectable ketamine effects on mean BOLD activation, exploratory psychophysiological interaction (PPI) analysis revealed alterations in functional connectivity during metacognitive confidence ratings under ketamine. Specifically, there was increased task-specific connectivity for ketamine compared to placebo between right anterior dorsolateral prefrontal cortex and left middle temporal, supramarginal and precentral gyrus, as well as between right insula/inferior frontal gyrus and left lingual gyrus, possibly indicating re-representations of object-level features supplied for metacognitive evaluations. Overall, these findings contribute towards the emerging picture of the substructures underlying metacognitive operations at the neurotransmitter level and may shed light on a neural pattern characteristic of pharmacologically challenged metacognition.

Confidence and Calibration of Comprehension in Adolescence: Are They Domain-General or Domain-Specific?

Self-regulation of learning is important for reading comprehension. Self-regulation is typically measured by confidence rating before or after a task and calibration calculations. Less is known about the stability of confidence and calibration within and across different tasks. In this study, we examined whether confidence ratings and calibration of adolescence differ by task, skill and time of rating (before, during, and after the task). Eighty-six adolescents took a reading comprehension task and a non-verbal spatial reasoning task (TONI) and were asked to self-evaluate their confidence before, during, and after the tasks. Results within tasks showed that ratings were significantly different as a function of time. Differences were also found across tasks. Consistent with findings in other age groups, good comprehenders were better than poor comprehenders in calibration. Our findings suggest that metacognitive evaluations have domain-specific as well as domain-general aspects and are also influenced by reading comprehension skills.

AKTIVITAS METAKOGNITIF SISWA DALAM MEMECAHKAN MASALAH BANGUN RUANG SISI DATAR

Metacognition is an important aspect in problem solving. This study aims to describe the students' metacognitive activity in solving the problem of building a flat side with a qualitative approach. The research subjects were three students of MTSN Malang 1 with good communication skills in the categories of high, medium, and low mathematical abilities. Data collection was done by giving the problem of flat side space and interviews. In the problem understanding phase, students perform metacognitive awareness when they rethink a given problem marked by rereading the problem by considering and monitoring things that are important to solve the problem. In the planning phase, students perform metacognitive regulation when they rethink the ways that have been linked to previous learning experiences marked by considering clues to problems that have been connected to the learning experience. In the phase of implementing the plan, students carry out metacognitive evaluations when they think back about the solution that is being worked on by changing and correcting the image that is being made and determining another more appropriate way. In the re-examination phase, students carry out metacognitive evaluation and awareness when they rethink the solutions that have been made marked by rereading the problem and checking the completion from the beginning to the end and convincing themselves that the solutions made are correct.

The relation between task-relatedness of anxiety and metacognitive performance

It is well established that anxiety influences a range of cognitive processes such as cognitive control or decision-making. What is less known is how anxiety influences the metacognitive evaluations individuals make about their own performance. The present study explored the importance of task-relatedness in the relation between anxiety and metacognitive awareness. More specifically, we hypothesized that induced anxiety would affect metacognitive performance in opposite directions depending solely on whether the threat is perceived as related or unrelated to the performance. We conducted two experiments to test this hypothesis.In both studies, electric shocks were administered randomly, regardless of participant’s performance. In experiment 1, participants were informed that the shocks would be administered randomly, unrelated to performance. In experiment 2, participants were told that their metacognitive performance would influence the administration of the electric shocks (i.e. fewer electric shocks with better metacognitive performance). Our predictions were confirmed. Threat perceived as unrelated to metacognitive performance (Experiment 1) decreased metacognitive performance. Threat believed to be related to metacognitive performance resulted in improved performance (Experiment 2).These findings confirm that induced anxiety exerts a strong influence on metacognitive awareness while sparing first-order task performance. We furthermore demonstrate that the perceived relatedness of the anxiety determines whether metacognitive performance decreases or improves performance under threat.

Assessment of metacognition in aviation pilot students during simulated flight training of a demanding maneuver

This study adapted the Demand Resource Evaluation Scores (DRES) as a metacognitive indicator in assessing pilot students’ perceptions during simulated training of a novel maneuver. Typically, positive DRES are associated with perceiving a demanding situation as a challenge and with improved performance, while negative DRES are linked to a perception of the situation as a threat, and to poorer performance. The novelty here was to assess DRES before and after the task and across three missions. Overall, students were found to change their perceptions from threat to challenge over time. Also, increased DRES were positively correlated with performance progressing from mission to mission, indicating that the students reflect on their performance as they advance in their training. These findings show that individual metacognitive evaluations of a stressful aviation maneuver might be important for the progress in performance. The results are discussed in terms of flight safety and pilot training.

A blended learning teaching strategy strengthens the nursing students’ performance and self-reported learning outcome achievement in an anatomy, physiology and biochemistry course – A quasi-experimental study

In nursing, bioscience is regarded as one of the cornerstones of nursing practice. However, bioscience disciplines as anatomy, physiology and biochemistry are considered challenging for students and the failure rate is high. In this study we explore a blended learning teaching strategy in an anatomy, physiology and biochemistry course for first year Bachelor nursing students. In the blended learning teaching strategy, short narrated online digital resources of bioscientific terms and concepts were integrated into the teaching design along with digital metacognitive evaluations of learning outcomes. Results show that compared to students receiving traditional face-to-face teaching, the students with a blended learning approach performed better on their national exam with a small to medium effect size (Cohen’s d=0.23). Student course evaluations supported the blended learning delivery with small to medium effect sizes. The students reported that the digital resources supported their learning outcome achievement, that they better understood the teacher’s expectations and that they were more satisfied with their virtual learning environment. This study adds to the growing literature of blended learning effectiveness in higher education, and suggests the use of digital resources as an enrichment of teaching and enhancement of students’ study experience.

From metacognitive beliefs to strategy selection: does fake performance feedback influence cognitive offloading?

The ubiquitous availability of technological aids requires individuals to constantly decide between either externalizing cognitive processes into these aids (i.e. cognitive offloading) or relying on their own internal cognitive resources. With the present research, we investigated the influence of metacognitive beliefs on individuals’ offloading behavior in an experimental setup (N = 159). We manipulated participants’ metacognitive beliefs about their memory abilities by providing fake performance feedback: below-average feedback, above-average feedback, or no feedback (control-group). We then measured offloading behavior, using a pattern copying task in which participants copied a color pattern from a model window into a workspace window. While solving this task, participants could rely either more on an internal memory strategy or more on an offloading strategy. Fake performance feedback affected the participants’ metacognitive evaluations about their memory abilities (below-group < control-group < above-group). Although fake performance feedback did not affect actual offloading behavior, the participants receiving below-average performance feedback reported that they had relied more on an offloading strategy than those participants receiving above-average performance feedback. Furthermore, the participants in the below-group reported lower general memory abilities than the other groups at the end of the experiment. We conclude that while fake performance feedback strongly influenced metacognitive beliefs, this did not transfer into a change of strategy selection, thus not influencing offloading behavior. We propose to consider not only metacognitive beliefs but also metacognitive experiences as potential determinants of cognitive offloading.

The effect of metacognitive training on confidence and strategic reminder setting.

Individuals often choose between remembering information using their own memory ability versus using external resources to reduce cognitive demand (i.e. 'cognitive offloading'). For example, to remember a future appointment an individual could choose to set a smartphone reminder or depend on their unaided memory ability. Previous studies investigating strategic reminder setting found that participants set more reminders than would be optimal, and this bias towards reminder-setting was predicted by metacognitive underconfidence in unaided memory ability. Due to the link between underconfidence in memory ability and excessive reminder setting, the aim of the current study was to investigate whether metacognitive training is an effective intervention to a) improve metacognitive judgment accuracy, and b) reduce bias in strategic offloading behaviour. Participants either received metacognitive training which involved making performance predictions and receiving feedback on judgment accuracy, or were part of a control group. As predicted, metacognitive training increased judgment accuracy: participants in the control group were significantly underconfident in their memory ability, whereas the experimental group showed no significant metacognitive bias. However, contrary to predictions, both experimental and control groups were significantly biased toward reminder-setting, and did not differ significantly. Therefore, reducing metacognitive bias was not sufficient to eliminate the bias towards reminders. We suggest that the reminder bias likely results in part from erroneous metacognitive evaluations, but that other factors such as a preference to avoid cognitive effort may also be relevant. Finding interventions to mitigate these factors could improve the adaptive use of external resources.

Learning From (Test) Experience

Abstract. To date there has been relatively little research within the domain of metamemory that examines how individuals monitor their performance during memory tests, and whether the outcome of such monitoring informs subsequent memory predictions for novel items. In the current study, we sought to determine whether spontaneous monitoring of test performance can in fact help individuals better appreciate their memory abilities, and in turn shape future judgments of learning (JOLs). Specifically, in two experiments we examined recognition memory for visual images across three study-test cycles, each of which contained novel images. We found that across cycles, participants’ JOLs did in fact increase, reflecting metacognitive sensitivity to near-perfect levels of recognition memory performance. This finding suggests that individuals can and do monitor their test performance in the absence of explicit feedback, and further underscores the important role that test experience can play in shaping metacognitive evaluations of learning and remembering.